Skip to content

Add momentum to the LearningRateMonitor #4354

@jeremyjordan

Description

@jeremyjordan

🚀 Feature

We should add a flag to optionally log the momentum value (for optimizers that have the attribute) in our LearningRateMonitor.

Motivation

Some learning rate schedulers (eg. OneCycle) also adjust the momentum value when updating the learning rate.

Pitch

For each scheduler's optimizer, we can check to see if it has a momentum attribute. If so, we log the value in addition to the learning rate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureIs an improvement or enhancementhelp wantedOpen to be worked on

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions