Meta tensors have side effects

[carmocca]

🐛 Bug

To Reproduce

Remove the standalone decorator as in this commit: 25935fe

Run the Meta tests together with the rest

lightning/tests/tests_pytorch$ pytest utilities/test_meta.py utilities/test_model_summary.py utilities/test_parameter_tying.py

There are side effects that cause the following error. The Meta test must be leaving global state modified.

E       NotImplementedError: Could not run 'aten::_cat' with arguments from the 'Meta' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::_cat' is only available for these backends: [CPU, CUDA, QuantizedCPU, BackendSelect, Python, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradLazy, AutogradXPU, AutogradMLC, AutogradHPU, AutogradNestedTensor, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, AutocastCPU, Autocast, Batched, VmapMode, Functionalize].

Expected behavior

This should not happen. We want to remove the standalone mark because it's very expensive in our CI

Environment

PyTorch 1.10+

cc @Borda @akihironitta