error when using auroc

[stevejpurves]

I may be using auroc incorrectly, but this seems like a bug, which is reproducable using the boring model example for tensors with ndim > 1

🐛 Bug

Using the auroc() metric results in an error:
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

Please reproduce using the BoringModel and post here

https://colab.research.google.com/drive/1Dc5cVu8FZN1C2ZnH4ZH-ouG2II1dt4FE?usp=sharing

Expected behavior

It should work as is?
this line

https://github.com/PyTorchLightning/pytorch-lightning/blob/96785b99df878ebc45bc455eb7902a6dd4661139/pytorch_lightning/metrics/functional/classification.py#L892

should use torch.any()?

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

• PyTorch Version (e.g., 1.0): 1.6.0
• OS (e.g., Linux): mac
• How you installed PyTorch (conda, pip, source): conda
• Build command you used (if compiling from source):
• Python version: 3.8
• CUDA/cuDNN version: none
• GPU models and configuration: none
• Any other relevant information: none

Additional context

also reproducible here:

[github-actions]

Hi! thanks for your contribution!, great first issue!

[ananyahjha93]

@stevejpurves can't access your colab, you need to provide permission. But if its just 1 line which you think is wrong, send in a PR with a test.

[ananyahjha93]

@SkafteNicki

[SkafteNicki]

this happens on multidimensional input:

y = torch.tensor([0, 1, 1, 0]); print(any(y)) # True
y2 = torch.tensor([0,1,1,2]); print(any(y2)) # False
y3 = torch.tensor([[0,1],[1,2]]); print(any(y3)) # error

we will maybe support multidim input in the future (it will still be reshaped into 1d tensor), but for now you would have to do the reshaping yourself.

[stevejpurves]

Thanks, i've updated the colab link which is now shared.

I'll go ahead and flatten my input for my immediate use case.

I considered a PR for such a small change but could find clear "how to contribute" guide/readme? is there one? this page was the closest i found https://github.com/PyTorchLightning/pytorch-lightning/tree/master/tests but that's a pretty high entry point for a PR if i need to run tests + coverage on a 2 GPU machine. Maybe there is a contributor doc somewhere else?

[SkafteNicki]

@stevejpurves that would be a great addition, and you are more than welcome to send a PR :]
Our contribution guidelines are here: https://pytorch-lightning.readthedocs.io/en/latest/CONTRIBUTING.html
In this specific case, you would need to implement test in this file:
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/tests/metrics/functional/test_classification.py
Add tests that would fail currently on master but succeed with your PR.
Try to make sure the test that you add are passing locally (you can run pytest with -k flag for filtering only your own test").

[NumesSanguis]

This issue already requests support for multi-class in AUROC: #3304
And a pull request has also already been made: #4236

[stevejpurves]

ok, looking at that PR is seems distinct from this issue, this is about multi-dim tensors not multi-class support. the issue is still there on that branch https://github.com/PyTorchLightning/pytorch-lightning/blob/19c4b7397614f7680e8c4251ea15b97564e95a95/pytorch_lightning/metrics/functional/classification.py#L892
and independent of the multi class changes it seems. I'll attempt PR for this :]