ReduceLROnPlateau Scheduler documentation problem

[KevinMathewT]

In the documentation it's given that to use ReduceLROnPlateau Scheduler we should do it as:

# The ReduceLROnPlateau scheduler requires a monitor
def configure_optimizers(self):
   return {
       'optimizer': Adam(...),
       'scheduler': ReduceLROnPlateau(optimizer, ...),
       'monitor': 'metric_to_track'
   }

But there is no variable initialized as optimizer for 'scheduler': ReduceLROnPlateau(optimizer, ...),, so I did:

def configure_optimizers(self):
        optimizer = torch.optim.Adam(
            params=self.parameters(), 
            lr=LEARNING_RATE,
            weight_decay=WEIGHT_DECAY
        )
        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer,
            patience=1,
            verbose=True
        )
        return {
           'optimizer': optimizer,
           'scheduler': scheduler,
           'monitor': 'val_loss'
       }

But for some reason the Scheduler was not working. I created a Callback to check, but trainer.lr_schedulers gives an empty list.
So I tried:

def configure_optimizers(self):
        optimizer = torch.optim.Adam(
            params=self.parameters(), 
            lr=LEARNING_RATE,
            weight_decay=WEIGHT_DECAY
        )
        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer,
            patience=1,
            verbose=True
        )
        return {
           'optimizer': optimizer,
           'lr_scheduler': scheduler, # Changed scheduler to lr_scheduler
           'monitor': 'val_loss'
       }

And now it's working.

[github-actions]

Hi! thanks for your contribution!, great first issue!

[rohitgr7]

it's a minor doc typo. Mind send a fix?

[KevinMathewT]

Sure

[KevinMathewT]

Here is the pull request.

[TheStoneMX]

same problem with ver 1.3.2

    def configure_optimizers(self):
        if self.cfg.use_swa:
            self.optimizer = SWA(torch.optim.Adam(self.model.parameters(), lr=self.cfg.lr))
        else:
            self.optimizer = torch.optim.Adam(self.model.parameters(), lr=self.cfg.lr)
            
        if self.cfg.scheduler == 'plateau':
            self.scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(self.optimizer, 
                                                                           mode='min', 
                                                                           factor=0.2, 
                                                                           threshold=0.01, 
                                                                           patience=5)
        else:
            self.scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(self.optimizer,
                                                                    T_max=self.cfg.t_max,
                                                                    eta_min=0,
                                                                    last_epoch=-1, #sets initial lr as lr.
#                                                                   eta_min=self.cfg.min_lr,
                                                                    verbose=True)

MisconfigurationException: configure_optimizers must include a monitor when a ReduceLROnPlateau scheduler is used. For example: {"optimizer": optimizer, "lr_scheduler": scheduler, "monitor": "metric_to_track"}

[rohitgr7]

@TheStoneMX if I understand correctly you are attaching the scheduler to self instead you need to return your optimizers and schedulers.

[TheStoneMX]

Hi there and thanks for the response, I been trying to learn Lighting, can you show me how to do it? thanks

[rohitgr7]

here are some docs: https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_module.html?highlight=configure_optimizers#configure-optimizers

[TheStoneMX]

thanks!

[TheStoneMX]

thanks!

[Kaviarasan-R]

Solution : return {'optimizer': optimizer,
'scheduler':scheduler,'monitor':"metric_to_track",}