Out of scope for this PR but maybe we should remove `Trainer(benchmark)` and set `torch.backends.cudnn.benchmark = not deterministic` _Originally posted by @carmocca in https://github.com/PyTorchLightning/pytorch-lightning/pull/9121#discussion_r696202228_ cc @borda @justusschock @awaelchli @akihironitta @rohitgr7 @tchaton