cli: Confused on (str, int, List[int]) variants for argparse for --gpus flag?

[EricCousineau-TRI]

🐛 Bug

A colleague (@siyuanfeng-tri) and I sometimes get confused on how the --gpus flag is to be interpreted by argparse. I see the following docs:
https://pytorch-lightning.readthedocs.io/en/1.2.1/advanced/multi_gpu.html#select-gpu-devices

But we're sometimes confused about when argparse interpretation will either assume it's the count of the gpu (int/str) or the device index (List[int]).

Are there docs for this? If not, can that be clarified somehow?

Please reproduce using the BoringModel

see notebook

The main complaint is that gpus=3 implies gpus=[3], while gpus="3" implies gpus=[0,1,2].
Mix that with implicit conversion from argparse from str to int, and you get a kinda weird public interface.

To Reproduce

example notebook: https://colab.research.google.com/drive/1pe9_F2S73-gQ3hOeh_MMiGhmbXmGURDQ?usp=sharing

Expected behavior

Less confusing / more explicit options? (maybe my complaint is with weird implicit behavior of Trainer(gpus=...)?)

Environment

• PyTorch Version (e.g., 1.7.1):
• OS: Ubuntu 18.04
• How you installed PyTorch: pip
• Python version: 3.6.9
• CUDA/cuDNN version: N/A
• GPU models and configuration: N/A
• Any other relevant information: N/A

Additional context

N/A

[awaelchli]

The main complaint is that gpus=3 implies gpus=[3], while gpus="3" implies gpus=[0,1,2].
Mix that with implicit conversion from argparse from str to int, and you get a kinda weird public interface.

This is not true.
gpus=3 implies Trainer(gpus=3) (no matter if string or int) which is equivalent to Trainer(gpus=[0, 1, 2])
only when there is a comma separation does it count as a listed index, i.e.
gpus="3," implies Trainer(gpus=[3])

The table you linked shows all these cases

[EricCousineau-TRI]

no matter if string or int

Er, that isn't what I see on the docs (on my machine at least)?
Here's a screencap (after I delete the other rows):

[awaelchli]

aha! I never noticed that. So the documentation is indeed correct and Trainer differentiates between Trainer(gpus=3) and Trainer(gpus="3"). However, the argparser can't do that, because it's always a string input and then it get's converted to an int. Therefore there is an inconsistency

[tchaton]

Dear @EricCousineau-TRI,

Great catch !
I guess the doc is confusing there and the default behaviour seems wrong to me.

I would expect '3' and 3 to imply [0, 1, 2] and "[3]" or "3," to parsed into "[3]".

However, this would break BC compatibility.
@awaelchli What do you think we should do.

Best,
T.C

[awaelchli]

I guess the doc is confusing there

No it is not the problem of the documentation. In fact, the docs are exactly correctly documenting this behavior the way it is.
The inconsistency is in the parser for the trainer arguments.

I opened a draft PR #6388. I present two options how to inform the user about this change:

a) There are probably not many users that input a string. Implement the change now for 1.3 and warn the user that the gpus flag has changed it's meaning.
b) Announce the change with a deprecation warning. In v1.5 implement the actual API change.

[williamFalcon]

cc @ananthsub