When using "-n auto", count the number of physical CPU cores instead of logical ones

[utapyngo]

I have a laptop with 6 physical CPU cores and hyper-threading. The code from auto_detect_cpus gives 12:

from os import sched_getaffinity
len(sched_getaffinity(0))
Out[24]: 12

The optimal number of parallel processes though is 6. Tests run slower when using 12 processes.
Did you consider using psutil.cpu_count(logical=False)?

[wei-hai]

+1

[RonnyPfannschmidt]

Psutil has not been considered as dependency yet

[Zac-HD]

For what it's worth this would be very useful for me too, and I wouldn't mind adding a dependency on psutil to support it.

[nicoddemus]

@RonnyPfannschmidt anything against depending on psutil? Seems like a pretty popular and stable package.

Even if we decide to no depend on it explicitly, we can optionally use cpu_count(logical=False) only when psutil is available.

[RonnyPfannschmidt]

There ist nothin against it

[AdrienPensart]

Guys you changed something important there, now, how can we specify $(nproc) in pytest.ini or pyproject.toml ?
If -n auto does not work anymore, the tests are not adapted when ran on different kind of machines :/
If tests are IO-bounded, and not CPU-bounded, it drastically ruin testing time.

maybe a better option like "-n [auto|ncpus|ncores|{n}]" could be nice ?

[nicoddemus]

Hi @AdrienPensart,

If -n auto does not work anymore, the tests are not adapted when ran on different kind of machines :/

-n auto should still be working... can you open an issue if it is not working for you?