For multiple optimizers, model.toggle_optimizer() is setting the requires_grad=True for all the params. #5292
Description
Setup :
pytorch-lightning 1.1.2
pytorch 1.7.1
When using multiple optimizers, in the toggle_optimizers(..) function, the requires_grad property is set to true for all the params belonging to the param_groups of the optimizer. This is incorrect as in case the user has explicitly disabled the requires_grad property for some parameters permanently, the function would enable requires_grad for those parameters also.
Proposed fix:
The requires_grad value for all the parameters should be stored beforehand in an object variable self.params_dict of lightning object. In the toggle_optimizer() function, instead of setting params.requires_grad=True, set params.requires_grad = self.params_dict[params].This would set the correct value for the params of the concerned optimizer.