tox4: plugin migration use cases

[gaborbernat]

This issue is used to mark the migration plans for various tox plugins. Shows which plugins and use cases we considered when defining the plugin interface for tox 4.

1. tox-travis
2. tox-poetry
3. tox-ansible
4. tox-pipenv
5. tox-current-env
6. tox-ltt
7. tox-py
8. tox-extra
9. tox-asdf
10. tox-bindep
11. tox-pyo3
12. tox-backticks
13. tox-envreport

Need to fork

1. tox-pyenv

POC done

1. tox-conda see https://github.com/tox-dev/tox-conda/tree/main/src/tox_conda

Done

1. tox-pdm see https://github.com/pdm-project/tox-pdm/blob/main/tox_pdm/plugin4.py
2. tox-gh-actions , recommending https://github.com/tox-dev/tox-gh instead (though upstream started adding support for tox 4)
3. tox-docker - thanks to @dcrosta at https://github.com/tox-dev/tox-docker/tree/tox-4

Moved to core

1. tox-battery via Port pip requirements.txt parser #2009
2. tox-external-wheels - via Support for external packages and builders #2235
3. tox-factor
4. tox-envlist

[hroncok]

We have https://pypi.org/project/tox-current-env/ and I'd happily discuss the use case with you. If new tox could be used as a library, we would be able to drop the plugin entirely and just ask tox for the data we need.

[gaborbernat]

Please do describe what functionality you would like to be exposed to, and why would it be better than a plugin? Thanks!

[hroncok]

Will do tomorrow.

EDIT: Or the day after.

[luzfcb]

I had to do some juggling for my tox-lambda-autodiscovery experimental plugin to work. There is no simple way to defer the processing of commands.

At that time, @gaborbernat had some insights into the improvements needed in a future tox 4. What is happening now.
https://mail.python.org/archives/list/tox-dev@python.org/thread/2B2GXTFWCE6FYMVCMBMKAOQAXEOGXEWR/

[gaborbernat]

There is no simple way to defer the processing of commands.

Can you clarify what do you mean by defer here?

[frostming]

Any chance unify the interface of install deps and install pkgs so that we can change the behavior in one hook.

[gaborbernat]

Any chance unify the interface of install deps and install pkgs so that we can change the behavior in one hook.

Definitely not, these two sections have totally different formats and contracts. It just happens pip can handle both using the same interface, but that's not generally true. If you're using an installer that can handle it the same way you can still define two hooks that calls the same function.

[hroncok]

tox-current-env

What do we want

We want a way to run upstream tests when we build RPM packages for Fedora. We want a standardized way to get a list of test dependencies (by we install them ourselves) and a standardized way to get commands to execute (or a standardized way to execute them in a Python environment we choose).

Why do we use tox

Unfortunately, there is no standard. We've been told that tox is the de-facto standard and that any future standard (if any) is likely to evolve from tox. I don't recall where is this information coming from, but it makes sense to us.

What does our plugin do

When certain command line switch is used, it redefines the virtualenv creation (and fakes it with symbolic links to sys.executable) .
When certain command line switch is used, it redefines the "run the tests" part of tox to print some information parsed by tox from the configuration (related to dependencies).

Why would a library be better

If there was a library, we could "ask" tox: What are the dependencies for this toxenv? What are the commands for this toxenv? (Or alternatively: Execute the commands via subprocess.) I think that this approach is better, because it:

• involves no hacks
• does not break when other plugins are installed
• is lighter
• is possibly faster as it does not involve serializing and deserializing data between our tooling and tox to text files or fds

Ideally this library would not even require virtuelnv, but I understand that this would require packaging it separately and that is a lot to ask.

[gaborbernat]

@hroncok for your use case my recommended solution would be to replace tox environment creation with a no op that just serves to tox the host python, translate pypi dependencies to RPM dependencies, and use apt to install them. All this you could do in a plugin. This would be the recommended way, and safer IMHO. Now if you'd insist on rolling your own pipeline to get the raw dependencies and commands you can use tox --showconfig and interpret the output. In tox 4 you'd be able to do tox c -k deps,commands to get just these.

[hroncok]

replace tox environment creation with a no op that just serves to tox the host python

I thought that we need an actual environment, so we create a fake one. I can have look if this can be simplified.

translate pypi dependencies to RPM dependencies, and use apt to install them

Due to technical limitations, we cannot install stuff directly, we need to echo it on stdout and exit. That is what we do, more or less, via the plugin (except that we do the translation via a wrapper).

This would be the recommended way, and safer IMHO.

OK. We'll keep doing it that way then.

Now if you'd insist on rolling your own pipeline to get the raw dependencies and commands you can use tox --showconfig and interpret the output. In tox 4 you'd be able to do tox c -k deps,commands to get just these.

I'll check it out.

[gaborbernat]

In a tox 4 world, you'd just need to implement the abstract parts of https://github.com/tox-dev/tox/blob/rewrite/src/tox/tox_env/python/api.py#L73, by default we use a virtualenv+pip based https://github.com/tox-dev/tox/blob/rewrite/src/tox/tox_env/python/virtual_env/api.py#L28, but you're free to replace those parts with host python + dnf. Note the API is not stable yet 👍🏻 so don't jump on this bandwagon just yet, but this is the midterm plan.

[obestwalter]

Hi, haven't looked at tox-current-env in detail yet, but it sounds a lot like the plugin, I wrote to facilitate running tests and tasks via tox inside a container that already contains a prepared venv or needs packages installed in that prepared venv: https://pypi.org/project/tox-direct/ circumvents the venv creation and uses the env that tox was started from.

I am just getting acquainted with the rewrite after a long time off from the whole project, but if I understand correctly one reason for the rewrite is to be able to accommodate different run time environments directly, including docker containers, so it would be interesting to find out what and how much of plugins like tox-direct and tox-current-env is actually still needed in tox4.