MissingSchema error via requests: fix pip's help or use "http" by default

[blueyed]

pip's help says:

--proxy Specify a proxy in the form [user:passwd@]proxy.server:port.

But when leaving out the scheme, requests complains:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 194, in find_requirement
    page = self._get_page(main_index_url, req)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 568, in _get_page
    session=self.session,
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 670, in get_page
    resp = session.get(url, headers={"Accept": "text/html"})
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 395, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 237, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 305, in send
    conn = self.get_connection(request.url, proxies)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 206, in get_connection
    except_on_missing_scheme(proxy)
  File "/usr/lib/python2.7/dist-packages/requests/utils.py", line 631, in except_on_missing_scheme
    raise MissingSchema('Proxy URLs must have explicit schemes.')
MissingSchema: Proxy URLs must have explicit schemes.

I could imagine fixing the help, or making pip default to using the http scheme.

[dstufft]

I believe this should be fixed in the latest develop now.

[blueyed]

No. Please re-open.

Exception:
Traceback (most recent call last):
  File "…/pip/pip/basecommand.py", line 201, in main
    status = self.run(options, args)
  File "…/pip/pip/commands/install.py", line 317, in run
    requirement_set.prepare_files(finder)
  File "…/pip/pip/req/req_set.py", line 338, in prepare_files
    upgrade=self.upgrade,
  File "…/pip/pip/index.py", line 246, in find_requirement
    page = self._get_page(main_index_url, req)
  File "…/pip/pip/index.py", line 722, in _get_page
    return HTMLPage.get_page(link, req, session=self.session)
  File "…/pip/pip/index.py", line 798, in get_page
    "Cache-Control": "max-age=600",
  File "…/pip/pip/_vendor/requests/sessions.py", line 463, in get
    return self.request('GET', url, **kwargs)
  File "…/pip/pip/download.py", line 284, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "…/pip/pip/_vendor/requests/sessions.py", line 451, in request
    resp = self.send(prep, **send_kwargs)
  File "…/pip/pip/_vendor/requests/sessions.py", line 557, in send
    r = adapter.send(request, **kwargs)
  File "…/pip/pip/_vendor/cachecontrol/adapter.py", line 43, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "…/pip/pip/_vendor/requests/adapters.py", line 329, in send
    conn = self.get_connection(request.url, proxies)
  File "…/pip/pip/_vendor/requests/adapters.py", line 237, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
  File "…/pip/pip/_vendor/requests/adapters.py", line 147, in proxy_manager_for
    **proxy_kwargs)
  File "…/pip/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 265, in proxy_from_url
    return ProxyManager(proxy_url=url, **kw)
  File "…/pip/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 216, in __init__
    'Not supported proxy scheme %s' % proxy.scheme

AssertionError: Not supported proxy scheme localhost

While at it, I've noticed that --proxy does not override the https_proxy env setting, which I've expected it to do (tracked in https://github.com/kennethreitz/requests/issues/2018).

And when setting https_proxy=localhost:3128 (without scheme), it appears to work.

[dstufft]

With the develop branch?

[blueyed]

Yes, just tested it: the exception has changed at least.. ;)

unset https_proxy http_proxy
pip --proxy localhost:3128/ install example

When looking into requests it seemed like the scheme/protocol would not be necessary anymore (https://github.com/kennethreitz/requests/pull/2219), but this appears to be a problem in the code above.

It works without the trailing slash:

pip --proxy localhost:3128 install example

btw: I have created https://github.com/kennethreitz/requests/issues/2220 which would allow for pip to specify a proxy explicitly.

So basically this is just a corner case issue with requests probably, where it fails to prepend the default scheme for HOST:PORT/.

[blueyed]

I've investigated further, added some tests for requests, and it turns out that pip will probably fail with python 2.6. for localhost:3128 already (http://ci.kennethreitz.org/job/requests-pr/433/), while all other python versions fail for localhost:3128/.

[dstufft]

This is fixed for me with the latest pip now.

[xiaohujecky]

@blueyed unset https_proxy http_proxy solved my problem, thank you

[blueyed]

@xiaohujecky
So it wasn't fixed for you still? What value were you using?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.