Training Type Plugin environment related setting from Trainer

[shuyingsunshine21]

🚀 Feature

Motivation

When user provide a specified training type plugin, user has to pass in num_nodes and sync_batchnorm explicitly. For example DDPPlugin. These parameters are set from Trainer, probably it is better to reuse the setting from Trainer instead of specifying again.

Now we have to specify like:

trainer = Trainer(
    num_nodes = 2,
    gpus = 8, 
    sync_batchnorm = True,
    plugins = [
       DDPPlugin(num_nodes=2, sync_batchnorm=True,  .... # other parameters)
   ]

Ideally, we could

trainer = Trainer(
    num_nodes = 2,
    gpus = 8, 
    sync_batchnorm = True,
    plugins = [
       DDPPlugin(.... # other parameters)
   ]

relying on accelerator_connector to connect num_nodes and sync_batchnorm to training type plugin instance.

Pitch

https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/connectors/accelerator_connector.py#L439-L445

        if hasattr(training_type, 'num_nodes') and getattr(training_type, 'num_nodes') is None:
            training_type.num_nodes = self.num_nodes

        # Automatically set sync_batchnorm if None.
        # Useful for custom plugins.
        if hasattr(training_type, 'sync_batchnorm') and getattr(training_type, 'sync_batchnorm') is None:
            training_type.sync_batchnorm = self.sync_batchnorm

here, instead of setting only when getattr(training_type, 'num_nodes') is None, we override as long as training_type has this attribute.

[shuyingsunshine21]

cc @ananthsub

[ananthsub]

@justusschock @awaelchli should these plugins expose setters for these properties? Are there any instances where we wouldn't want to override these properties within the trainer?

[justusschock]

@ananthsub Actually, I don't think that they shouldn't be overwritten. And in the connector you then want to check, if this attribute/property is present and if it is, we set it?

So similar to what we already do with DDPs parallel_devices?

I guess we just missed those when introducing the same behavior for parallel_devices.
@shuyingsunshine21 want to send a quick PR? Otherwise I can do that as well :) Probably also remove them from the plugin's init signature completely then :)

[shuyingsunshine21]

I will go ahead to send a PR