According to the pip docs, it will use a wheel cache for packages when possible. The url for these packages is part of the cache keying but it appears that what determines if it should download instead of cache is the following:
We've observed that even being explicit with cache location, pip still goes to download the package. I'm not sure if the lack of expire/cache control headers is the culprit yet or if it's the url in some way (i.e. being redirected to a different path).