Skip to content

Elapsed time not shown correctly when max_time is reached #7778

@manuel-munoz-aguirre

Description

@manuel-munoz-aguirre

🐛 Bug

When setting the max_time flag in the Trainer and the time is reached, the elapsed time is not displayed correctly in the trainer stopping message. Happens both when running the script through the terminal or in a jupyter notebook.

To Reproduce

Reproduced with the BoringModel:

def test_x():
    # init model
    model = BoringModel()

    # Initialize a trainer
    trainer = pl.Trainer(
        max_epochs=1000,
        max_time="00:00:00:15"
    )

    # Train the model ⚡
    trainer.fit(model, train, val)

Output:
Time limit reached. Elapsed time is <bound method Timer.time_elapsed of <pytorch_lightning.callbacks.timer.Timer object at 0x2b3101e19320>>. Signaling Trainer to stop.

Expected behavior

Elapsed time should be displayed.

Environment

* CUDA:
        - GPU:
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
                - GeForce RTX 2080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.19.0
        - pyTorch_debug:     False
        - pyTorch_version:   1.8.1+cu102
        - pytorch-lightning: 1.3.3
        - tqdm:              4.61.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                -
        - processor:         x86_64
        - python:            3.6.9
        - version:           #1 SMP Wed May 13 10:45:47 CDT 2020

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedOpen to be worked on

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions