Improve tqdm progress bar

[hadim]

At the moment the progress bar is initialized with the arg leave=False: https://github.com/PyTorchLightning/pytorch-lightning/blob/deffbaba7ffb16ff57b56fe65f62df761f25fbd6/pytorch_lightning/trainer/trainer.py#L861

Sometimes, it's nice to be able to see the previous progress bar to look at the evolution of the loss and metrics.

Would that be possible to add an arg to the trainer to be able to override default tqdm parameters?

Also, another point: tqdm progress bars can be nested (https://github.com/tqdm/tqdm#nested-progress-bars). Could we imagine having a global progress bar and then a nested progress bar for each epoch loop?

[hadim]

Another nice addition would be a global progress bar to have an ETA for the end of the whole training. Maybe a more general way to address this issue is to abstract the use of the progress bar in Trainer (with a callback system for example), so people can extend and tweak progress bar usage as they need.

[Borda]

@hadim sounds interesting, do you have any particular implementation in mind?
Would you mind to make a PR? =)

[hadim]

I think the progress bar should not be hardcoded in the trainer but abstracted in a callback. Once #776 is merged I could have a look if it's possible with the current API.

More generally the loggers should also be callbacks IMO. That being said it's easy to propose when you're not in charge :-)

I'll try to make a PR once #776 is merged.

[Borda]

@hadim are still interested in implementing this progress bar?

[hadim]

I've made a custom progress bar as a callback and it works well for my needs. Not sure it will fit everyone's needs.

from tqdm.auto import tqdm

import torch
from pytorch_lightning.callbacks import Callback


class ProgressBar(Callback):
    """Global progress bar.
    TODO: add progress bar for training, validation and testing loop.
    """

    def __init__(self, global_progress: bool = True, leave_global_progress: bool = True):
        super().__init__()

        self.global_progress = global_progress
        self.global_desc = "Epoch: {epoch}/{max_epoch}"
        self.leave_global_progress = leave_global_progress
        self.global_pb = None

    def on_fit_start(self, trainer, pl_module):
        desc = self.global_desc.format(epoch=trainer.current_epoch + 1, max_epoch=trainer.max_epochs)

        self.global_pb = tqdm(
            desc=desc,
            total=trainer.max_epochs,
            initial=trainer.current_epoch,
            leave=self.leave_global_progress,
            disable=not self.global_progress,
        )

    def on_fit_end(self, trainer, pl_module):
        self.global_pb.close()
        self.global_pb = None

    def on_epoch_end(self, trainer, pl_module):

        # Set description
        desc = self.global_desc.format(epoch=trainer.current_epoch + 1, max_epoch=trainer.max_epochs)
        self.global_pb.set_description(desc)

        # Set logs and metrics
        logs = pl_module.logs
        for k, v in logs.items():
            if isinstance(v, torch.Tensor):
                logs[k] = v.squeeze().item()
        self.global_pb.set_postfix(logs)

        # Update progress
        self.global_pb.update(1)

Only a global progress bar is implemented at the moment.

I could make a PR but some people might prefer the original one so I don't know if it's worth it.

[Borda]

yeah it looks the much cleaner way that using the callback driven progress bar then checking the for loop wrapped by tqdm

[danieltudosiu]

May I also add that I find the tqdm progress bar starting weirdly, with a percentage equal with 6% just after a single batch. And the progress bar shows final value of 790, but if I am to calculate it by hand an epoch either has 528 or 1056 (either one pass or one forward, one backward).

[williamFalcon]

the bar shows the sum of train + val