Epoch level scheduler is not well called except check_val_every_n_epoch == 1

[seungwooYoo]

🐛 Bug

Please reproduce using the BoringModel

Run the following colab link.
You can see multi-step lr scheduler is only called at validation epoch (i.e. 4 - check_val_every_epoch is set as 4) even I set milestones as [1,2,3,4]. I set scheduler attribute as verbose=True to show changed lr_rates.
If check_val_every_epoch is set as 1, scheduler "setup()" after train epoch seems well called.

https://colab.research.google.com/drive/1IqgyIU6EsCTk1y1_xH6VIEdpDsoMyPyt?usp=sharing

To Reproduce

Use following BoringModel and post here

Result :

GPU available: True, used: False
TPU available: None, using: 0 TPU cores
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/utilities/distributed.py:51: UserWarning: GPU available but not used. Set the --gpus flag when calling the script.
warnings.warn(*args, **kwargs)
Adjusting learning rate of group 0 to 1.0000e-01. --> Initial lr set verbose

| Name | Type | Params

0 | layer | Linear | 66

66 Trainable params
0 Non-trainable params
66 Total params
0.000 Total estimated model params size (MB)
Validation sanity check: 0%
0/2 [00:00<?, ?it/s]
Epoch 4: 48%
300/626 [00:00<00:00, 374.57it/s, loss=3.06e-14, v_num=0]
Validating:
0/0 [00:00<?, ?it/s]
Adjusting learning rate of group 0 to 1.0000e-02. --> Incorrect outcome - should be called at every epoch since milestones are set as [1,2,3,4]

Testing: 100%
313/313 [00:00<00:00, 777.17it/s]

---

DATALOADER:0 TEST RESULTS
{'fake_test_acc': 3.062137267926049e-14}

Expected behavior

Should lr scheulder step is called after train epoch end if interval is set as epoch

Environment

Note: Bugs with code are solved faster ! Colab Notebook should be made public !

• IDE: Please, use our python bug_report_model.py template.

• Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

• CUDA:
  • GPU:
    • Tesla K80
  • available: True
  • version: 10.1
• Packages:
  • numpy: 1.19.5
  • pyTorch_debug: False
  • pyTorch_version: 1.8.0+cu101
  • pytorch-lightning: 1.2.4
  • tqdm: 4.41.1
• System:
  • OS: Linux
  • architecture:
    • 64bit
  • processor: x86_64
  • python: 3.7.10
  • version: Proposal for help #1 SMP Thu Jul 23 08:00:38 PDT 2020

Additional context

[invisprints]

Same issue, here is another simpler Colab version modified on RAPID PROTOTYPING TEMPLATES

[RaivoKoot]

I can confirm that I am running into the same issue no matter what scheduler I use (LambdaLR, MultiStepLR, etc.)

[RaivoKoot]

If you could propose a temporary workaround until this is fixed, that would be great!

[awaelchli]

I believe the PR that I just linked will solve your issue.