Trainer.validate crashes for certain validation_step output and hook combinations

[ananthsub]

🐛 Bug

If the validation_step returns a complex output, and if the validation_epoch_end hook is not implemented in the LightningModule, there is a failure in the metrics holder when reducing the outputs.

This occurs on master

cc @carmocca

fyi @simran2905 @ytsheng

Please reproduce using the BoringModel

Repro: https://gist.github.com/ananthsub/5a027e24c91c2fa179b081e959583a00
Stack trace: https://gist.github.com/ananthsub/9b677206c98840f463c3aa9ef9d27fba

Expected behavior

There shouldn't be any crash

Environment

Note: Bugs with code are solved faster ! Colab Notebook should be made public !

• IDE: Please, use our python bug_report_model.py template.

• Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

[carmocca]

The error is not related to validate or test. It's due to logging a tensor with more than one element, which is not supported. From your snippet:

return {"output": output, ...}

where output is the network's output with 2 elements (2 output features).

So, unless I am missing something, this is working as intended. Tried reverting to older lightning versions, and the same error appears.

I have created #6645 to provide a better error message.

[carmocca]

After further investigation, you are correct that it should work for val/test just as for training. Will take a look at the cause.