refactor pip's use of "#egg="

[qwcode]

I suspect pip doesn't really need the"egg=" value (assuming we make some modifications)

setuptools used it in two cases:

• installing single py files (which pip doesn't support I guess?)
• installing from svn urls

pip's support is like so:

1. officially documented for vcs urls (it determines the src or build dir, depending on whether it's editable, and also determine's it's identification in the dependency resolution process.)
2. although not documented, I've seen users (including myself) tack "egg=" onto archive urls (that currently don't get their sdist filenames parsed, as you would expect (see The --build-dir options doesn't work for local archive installs #804), so using "egg=" for them makes them have a set build dir)

3 changes are needed, to make pip not require "egg=":

1. do away with the --no-install/--no-download "workflow" (see refactor --no-install/--no-download/--download (and use mkdtemp build dirs) #906), which assumes fixed build dirs, and then drop fixed build dirs all together.
2. make it a tested feature that non-specifier requirements (i.e. vcs urls, archive urls and paths) get processed first (i.e. unpacked and have their name/version determined), so they can properly participate in the dependency resolution process. this is currently true but not very prominent in the code and not tested afaict.
3. when doing editable intalls from vcs urls, use some other approach to determine the "src" directory. Maybe just unpack it to /tmp, then determine the name, then move it back into src.

even if it turns out #egg=is not used in any way by pip, it may be important to document a convention as visual aid (possibly as #pkg=, see #1265), when the vcs url would otherwise contain no clear indication as to what it is. I know for example that various CM modules for chef and puppet make use of this indicator

[Ivoz]

@qwcode could you link to an example puppet / chef module that makes use of that currently?

[qwcode]

• this PR (Extract egg/package name from URLs. poise/python#85), but it's unmerged
• puppet's core pip provider: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb#L77
• and my company has an internal puppet module (that might go open source) that relies on it.

[qwcode]

cc @coderanger

[coderanger]

To be fair, the reason I've not merged that PR is because I don't want to be parsing pip-specific URL syntax.

[qwcode]

the module we use at work uses a python helper module (that uses pip to handle the parsing), so no rewriting of pip in ruby.

but regardless, the issue is whether the vcs requirement form should require or at least allow an identifier tag like #egg= (or #dist=).

if you're only thinking about what pip needs to do an install, then you could argue it's not necessary (and worse actually misleads pip). you could just say that pip should always be cloning and using the real name and version from setup.py to determine how it gets processed relative to the other requirements.

but for a CM system that might want to support the vcs url form as a valid way to declare a resource (i.e. not just package specifiers 'pkg==version'), then having an identifier like #egg==name-version is pretty handy to quickly validate whether the resource is currently satisfied on the system.

[Ivoz]

So, I've realised that although pip might deprecate making use of that field, there's no-one saying we should disallow the fragment being in the url, right?

[qwcode]

but I'm thinking it's more than just allowing arbitrary fragments.
we should document the convention.

moreover, pip might start using the conventions itself for fast determination of fulfillment:

• if a '@' ref is present that's a literal version/commit, check that before updating the src clone.
• or, if #[egg|dist]=<name>-<version> is present, check that name and version before re-updating the clone.

[dstufft]

I think that this is still an issue, I'm not sure it makes sense to get rid of it, but at the very least we should rename it since we have deprecated eggs and the terminology is confusing to people.

[xavfernandez]

I think it should (very) slowly be deprecated in favor of the PEP-508 blessed pkg_name @ url.
(or am I missing something ?)

[dstufft]

Yea that's likely a reasonable outcome.

[pradyunsg]

So... the actionable part of this would be to deprecate #egg= in favour of the PEP 508 syntax?

Would this be a documentation only change?