Suggest alternatives for --process-dependency-links

[rodart]

Environment

• pip version: 19.0
• Python version: 3.5.2
• OS: Ubuntu 16.04

New pip release v19.0 break --process-dependency-links flag. That completely sucks because :

1. there was no information in which exact version this flag will be removed. There was a warning like 'Dependency Links processing has been deprecated and will be removed in a future release.' that we can see for a few years without pointing the exact date or version where it will be actually removed.

2. this feature is still using by a lot of engineers/companies. For example, it broke the whole build pipeline for all our python projects in our company. You just created a headache for thousands of engineers.

3. there is no clear explanation or tutorial about which alternative we should use instead of this flag

I would suggest to:

1. Give detailed explanation what should be used instead --process-dependency-links flag

2. Provide exact version where you are going to deprecate or remove some functionality. Especially in such a global project such as pip. Good examples are numpy and scikit-learn which mention in which versions they will deprecate some features.

[pradyunsg]

Please look at #4187 (comment).

(edit from a future @pradyunsg -- this is a better answer: #5898 (comment))

[pradyunsg]

The above issue was linked in the error message, however, I do think now that more explicit communication would have helped the situation. The deprecation messages should've mentioned the release in which the functionality will be removed.

I guess the ship has sailed here, but I'll go update pip's deprecation helpers so that something like this doesn't happen in the future.

[stinovlas]

It seems that #5898 has not been addressed yet, so there is no way to replace whole dependency links functionality right now, is it? Of course, there is still an option to keep pip < 19 installed until this is resolved.

[fake-name]

This exploded my CI pipeline yesterday/last night too. Maybe have a meaningful depreciation warning in the future?

Last I'd heard, dependency links had been un-depreciated (I don't track issues here when things work), but now passing --process-dependency-links doesn't just not work, but it causes pip to explode.

This sort of thing needs at least 3-6 months of a new warning on installs before it drops.

[pkucmus]

After this I ended up running my own DevPI and I store my private packages there, using it as my main index. Aside from fixing the problem and allowing me to remove --process-dependency-links from all my "build scripts" it gives me other advantages like not being completely reliant on PyPi.

I have a package which references many git repositories as dependency_links in its setup.py file. This package is used in many of my code pipeline (~40 different code libraries using this package to provide common functionality). Pipeline is now totally broken using pip19. Please can someone tell me what the correct way should be to get git repositories installed during a build using pip >=19 ? From what i read so far is this functionality just been removed??

[stinovlas]

@x00x70: If you don't need version specifiers (or you can depend on one specific version of the package), just delete dependency_links arg from setup function call and write the dependencies directly into install_requires (and others) according to PEP 508.

I.e. requests @ https://github.com/requests/requests.git@branch_or_tag

If you need version specifiers, you are out of luck with pip 19.

[jenstroeger]

@stinovlas, I was hopeful that your suggestion would solve my problem, but while the tag v1.3.2 exists in the private repo, foo@ https://token@github.com/jenstroeger/foo.git@v1.3.2 returns

  HTTP error 404 while getting https://token@github.com/jenstroeger/foo.git@v1.3.2
  Could not install requirement […] because of error 404 Client Error: Not Found for url: https://github.com/jenstroeger/foo.git@v1.3.2

The same error when I use @master or nothing at the end of the URL. Are private repos a problem here?

[stinovlas]

@jenstroeger: Private repo requires some form of authentication. pip basically runs git clone, so just copy the same url as when you are cloning your repo (or the output of git remote -v).

In your case foo @ git+ssh://token@github.com/jenstroeger/foo.git@v1.3.2 should work.

[yeraydiazdiaz]

I can confirm the above does work. However, I noticed that changing the git tag in the URL does not install the new version. pip will report the dependency as already met and keep the existing version of the package. Here's a minimal setup.py:

from distutils.core import setup

setup(
    name='pep508_test',
    version='0.0.1',
    install_requires=[
        'requests @ git+ssh://git@github.com/requests/requests.git@v2.20.0',
    ]
)

Running pip install -e . will install version 2.20.0 as expected. However changing the tag to v2.21.0 will not install the new version.

I've tried using requests==2.21.0 @ git... but pip returns the following error:

'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'@ git+ss'"

I'm not sure if it's a problem with pip or the grammar. PEP 508 does not show many examples on URL dependencies, the above being one and name [fred,bar] @ http://foo.com the other.

@pradyunsg any advice?