-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Closed
Labels
featureIs an improvement or enhancementIs an improvement or enhancementhelp wantedOpen to be worked onOpen to be worked onrefactor
Description
🚀 Feature
Very similar to #4317
https://pytorch-lightning.readthedocs.io/en/latest/common/trainer.html#truncated-bptt-steps
It's highly architecture dependent, meaning it's closer to the research code than the engineering code. Accordingly, this should live on the LightningModule, not on the Trainer.
Pitch
Define a property method for truncated_bptt_steps inside the LightningModule
Changes to the LightningModule
self._truncated_bptt_steps: Optional[int] = None
@property
def truncated_bptt_steps(self) -> Optional[int]:
return _truncated_bptt_steps
@truncated_bptt_steps.setter
def truncated_bptt_steps(self, tbptt_steps) -> None:
self._truncated_bptt_steps = tbptt_steps
Then the training loop can read these settings off the lightning module and adjust accordingly, similar to what's done today for automatic/manual optimization. We can then deprecate the trainer flag for this setting.
Metadata
Metadata
Assignees
Labels
featureIs an improvement or enhancementIs an improvement or enhancementhelp wantedOpen to be worked onOpen to be worked onrefactor