[BUG] Setuptools allow arbitrarily wrong configs in `setup.cfg`

[abravalheri]

The issue in #4910 unveiled something much deeper.
It seems that setuptools has been allowing arbitrary fields (like description_file described in #4910) in setup.cfg (possibly without a visible warning -- needs investigation)¹.

Expected outcome

Setuptools should issue a deprecation warning when a invalid field in used in setup.cfg.

Footnotes

1. Funny thing is that issue [BUG] Version 78.0.1 breaks install of ansible-vault package #4910 always involves description-file. I find it hard to believe that it is a coincidence, possibly many packages CTRC+C, CTRL+V the same config all over the place. ↩

[zr40]

description-file in setup.cfg might have originated from Openstack's PBR in 2013 (then called oslo.packaging): https://github.com/openstack/pbr/blob/aa4641f0e6e668d4eb3a0c1f70a3e15e83097c6a/README.rst and they still support this field today, albeit in the underscore form: https://github.com/openstack/pbr/blob/master/pbr/util.py#L321

[fungi]

It goes back even further, PBR (nee oslo.packaging for a brief moment) was a fork of d2to1, which implemented support for description-file because Distutils2 had it.

[aman2454]

I have seen a different one than description-file,
setuptools.errors.InvalidConfigError: Invalid dash-separated key 'force-manifest' in 'sdist' (setup.cfg), please use the underscore name 'force_manifest' instead

[fungi]

The force-manifest entries look like they're for distutils, which allows passing default command-line options through setup.cfg files, and --force-manifest is one such distutils option.

I suppose these examples raise a question: should other packaging tools and SetupTools plugins be allowed to extend setup.cfg with their own directives alongside SetupTools use of the same file?

Apparently some do, and have for years. The heritage of setup.cfg files goes back beyond PBR/d2to1/Distutils2 to the original distutils which relied on it for declarative configuration decades before SetupTools (re)copied that design (you'll find references to setup.cfg files for distutils in documentation from 2001, possibly earlier). Allowing and ignoring unrecognized entries has provided backward-compatibility with existing packages already relying on this functionality for a very long time.

[abravalheri]

I suppose these examples raise a question: should other packaging tools and SetupTools plugins be allowed to extend setup.cfg with their own directives alongside SetupTools use of the same file?

This is something that needs to be part of the investigation for this issue. We don't want to break other tools. I suspect we should allow any keywords defined via distutils.setup_keywords entry-point, but I don't know if that is enough for compatibility.

This issue only applies to the [metadata] and [option] sections of setup.cfg, the rest can be arbitrary.

[abravalheri]

I suspect we should allow any keywords defined via distutils.setup_keywords entry-point, but I don't know if that is enough for compatibility.

It seems this is not enough: https://inspector.pypi.io/project/pbr/6.1.1/packages/47/ac/684d71315abc7b1214d59304e23a982472967f6bf4bde5a98f1503f648dc/pbr-6.1.1-py2.py3-none-any.whl/pbr-6.1.1.dist-info/entry_points.txt#line.4

So we may have to find another alternative.

[herebebeasties]

We don't want to break other tools.

What you are still missing here despite the hundreds of thousands of man hours of disruption you caused globally yesterday is that you also don't want to break your own tool.

Given all the existing packages out there that have config keys that setuptools doesn't recognise, accepting those unknown keys isn't a bug, it is very much a feature.

At the very most you should log a warning that setuptools is ignoring them.

[eli-schwartz]

This is something that needs to be part of the investigation for this issue. We don't want to break other tools. I suspect we should allow any keywords defined via distutils.setup_keywords entry-point, but I don't know if that is enough for compatibility.

It seems evident to me that the answer here is:

setuptools should not attempt to validate fields that don't belong to it. Let them be as right or as wrong as they please, it's not setuptools' place to tell them they are doing something wrong.

Either that, or else setuptools should declare that anyone passing arguments to a setuptools plugin is "in the wrong" and should be punished for it by failure. But setuptools has historically considered at least passing arbitrary plugin-defined kwargs to setuptools.setup() as "expected behavior of the extensibility interface". So I cannot see how e.g. pbr could be considered in the wrong for declaring that they own and will handle arbitrary additional keys in [metadata]. (This is entirely independent of the fact that pbr and friends owned [metadata] and setuptools was the interloper.)