Skip to content

Revisit usage of _MockOptimizer #11178

@ananthsub

Description

@ananthsub

In #1279, support for returning None from configure_optimizers was added to Lightning. The use case was training without an optimizer. This preceded support for manual optimization, in which the user controls the backward & optimizer step directly inside of their training step.

The _MockOptimizer leaks out like this which could be very confusing for developers.

class MyLightningModule(LightningModule):
    def configure_optimizers():
        return None

    def training_step(batch, batch_idx):
        opt = self.optimizers()
        # opt is not None! what?!
  • Is training with no optimizer a valid use case Lightning supports? Are there examples/references one could share to learn more about these use cases?
  • If the Trainer creates a mock optimizer for users, should the mock optimizer ever be exposed back to the user?
  • If training with no optimizer is a valid use case, should we require users to use manual optimization for this, so we don't configure a mock optimizer instance for them?

Originally posted by @ananthsub in #11155 (comment)

cc @tchaton @justusschock @awaelchli @Borda @rohitgr7

Metadata

Metadata

Assignees

No one assigned

    Labels

    designIncludes a design discussionoptimizerwon't fixThis will not be worked on

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions