Skip to content

truncated_bptt_steps should be a property of the LightningModule, not the Trainer #7322

@ananthsub

Description

@ananthsub

🚀 Feature

Very similar to #4317

https://pytorch-lightning.readthedocs.io/en/latest/common/trainer.html#truncated-bptt-steps
It's highly architecture dependent, meaning it's closer to the research code than the engineering code. Accordingly, this should live on the LightningModule, not on the Trainer.

Pitch

Define a property method for truncated_bptt_steps inside the LightningModule

Changes to the LightningModule

self._truncated_bptt_steps: Optional[int] = None

@property
def truncated_bptt_steps(self) -> Optional[int]:
    return _truncated_bptt_steps

@truncated_bptt_steps.setter
def truncated_bptt_steps(self, tbptt_steps) -> None:
    self._truncated_bptt_steps = tbptt_steps

Then the training loop can read these settings off the lightning module and adjust accordingly, similar to what's done today for automatic/manual optimization. We can then deprecate the trainer flag for this setting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureIs an improvement or enhancementhelp wantedOpen to be worked onrefactor

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions