[RTFACT-12421] PyPi repositories may ignore the the override base url Created: 07/Sep/16  Updated: 05/Aug/18  Resolved: 12/Jun/18

Status: Resolved
Project: Artifactory Binary Repository
Component/s: PyPI
Affects Version/s: 4.11.0, 4.12.0, 4.13.0, 5.3.0
Fix Version/s: 6.1.0

Type: Bug Priority: High
Reporter: Arturo Aparicio Assignee: Dudi Morad (Inactive)
Resolution: Fixed Votes: 6
Labels: None

Issue Links:
Duplicate
is duplicated by RTFACT-14954 Artifactory ignores "X-Artifactory-Ov... Resolved
Sprint: Leap 32

 Description   

PyPi repositories may ignore the the override base url. This can cause replication issues, among others. This is particularly harmful if there is no context URL. This will also convert HTTPS to HTTP. To reproduce:

1. Use a reverse proxy configuration without a public context
2. Create a PyPi repository called pypi-bld-qa-local
3. Curl with verbose to see the redirect URL, for example:
curl -vXGET -u admin:password http://localhost/api/pypi/pypi-bld-qa-local/simple/algn

Notice the header:

    proxy_set_header    X-Artifactory-Override-Base-Url $http_x_forwarded_proto://$host;

Notice a good call:

curl -vXGET -u admin:password http://localhost/pypi-bld-qa-local
...
< HTTP/1.1 302 Found
...
< Location: http://localhost/pypi-bld-qa-local/

Notice the bad PyPi call:

curl -vXGET -u admin:password http://localhost/api/pypi/pypi-bld-qa-local/simple/algn
...
< HTTP/1.1 301 Moved Permanently
...
< Location: http://localhost/artifactory/api/pypi/pypi-bld-qa-local/simple/algn/


 Comments   
Comment by Timothy Lusk [ 08/Feb/17 ]

Having this same issue, it's causing our https to be stripped off for the redirects and remote pypi repos pointing to other artifactory servers to fail to work correctly.

In our case we added this into the nginx config to cause nginx to handle the redirect and not artifactory:

{{rewrite ^/artifactory/api/pypi/([^/]+)/simple$ /artifactory/api/pypi/$1/simple/ permanent;
rewrite ^/artifactory/api/pypi/([^/])/simple/([^/])$ /artifactory/api/pypi/$1/simple/$2/ permanent;}}

Comment by Timothy Lusk [ 08/Feb/17 ]

Bad formatting in my previous comment, here is the nginx configuration again:

rewrite ^/artifactory/api/pypi/([^/]+)/simple$ /artifactory/api/pypi/$1/simple/ permanent;
rewrite ^/artifactory/api/pypi/([^/]+)/simple/([^/]+)$ /artifactory/api/pypi/$1/simple/$2/ permanent;
Comment by Ollie Ready [ 11/Jul/17 ]

WORKAROUND:

I have a header_filter_by_lua_file script that I use for OpenResty which fixes this issue (and bonus, allows for docker repo selection using tags). Here's the script:

--docker fix
host_name = 'FQDN.GOES.HERE'
local locHeader = ngx.header["Location"]
if locHeader then
    if type(locHeader) == "string" then
        local location = ngx.re.match(locHeader, "https?://(.*?/)v2/(.*)", "io")
        if location then
            ngx.header["Location"] =  "https://" .. host_name .. "/v2/" .. location[1] .. location[2]
        end
    end
end

--pypi fix
local locHeader2 = ngx.header["Location"]
if locHeader2 then
    if type(locHeader2) == "string" then
        local location = ngx.re.match(locHeader2, "https?://localhost(.*)", "io")
        if location then
            ngx.header["Location"] =  "https://" .. host_name .. location[1]
        end
    end
end

And then in your OpenResty nginx config, add this:

header_filter_by_lua_file /path/to/lua_includes/header_filter.lua;

The filename can be whatever you want of course, as long as it contains the above code and that config points to it.

Generated at Tue Jan 21 12:23:05 UTC 2020 using JIRA 7.6.16#76018-sha1:9ed376192612a49536ac834c64177a0fed6290f5.