Feature/5275 clean progress bar print #5470

asnorkin · 2021-01-11T20:56:32Z

What does this PR do?

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified
Check that target branch and milestone match!

Did you have fun?

Make sure you had fun coding 🙃

pytorch_lightning/core/lightning.py

awaelchli

Looking very good, could you also add a stub to the base class (ProgressBarBase)? Thanks

codecov · 2021-01-12T01:56:13Z

Codecov Report

Merging #5470 (715a96d) into master (115e58a) will decrease coverage by 0%.
The diff coverage is 95%.

@@          Coverage Diff           @@
##           master   #5470   +/-   ##
======================================
- Coverage      91%     91%   -0%     
======================================
  Files         160     160           
  Lines       11407   11421   +14     
======================================
- Hits        10408   10394   -14     
- Misses        999    1027   +28

rohitgr7

LGTM apart from Adrian's comment.

carmocca

Cool! Can we add tests?

docs/source/multi_gpu.rst

tests/metrics/test_metric.py

awaelchli

I took the liberty and added some tests to keep coverage up :) Have a look.

pytorch_lightning/callbacks/progress.py

Borda

@asnorkin mind check comments from @awaelchli then I think we are ready to go... 🐰

pytorch_lightning/callbacks/progress.py

* resolve bug * merge tests

* Fix metric state reset * Fix test * Improve formatting Co-authored-by: Ananya Harsh Jha <[email protected]>

pep8speaks · 2021-02-18T13:03:03Z

Hello @asnorkin! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-02-18 15:32:17 UTC

CHANGELOG.md

tchaton

LGTM !

…ter) to github/third-party/PyTorchLightning/pytorch-lightning Summary: ### New commit log messages ## [UnReleased] - 2021-MM-DD ### Added - Added more explicit exception message when trying to execute `trainer.test()` or `trainer.validate()` with `fast_dev_run=True` ([#6667](Lightning-AI/pytorch-lightning#6667)) - Added `LightningCLI` class to provide simple reproducibility with minimum boilerplate training cli. ([#4492](Lightning-AI/pytorch-lightning#4492)) - Trigger warning when non-metric logged value with multi processes hasn't been reduced ([#6417](Lightning-AI/pytorch-lightning#6417)) - Added `gradient_clip_algorithm` argument to Trainer for gradient clipping by value ([#6123](Lightning-AI/pytorch-lightning#6123)). - Added a way to print to terminal without breaking up the progress bar ([#5470](Lightning-AI/pytorch-lightning#5470)) - Added support to checkpoint after training steps in `ModelCheckpoint` callback ([#6146](Lightning-AI/pytorch-lightning#6146)) - Added `checkpoint` parameter to callback's `on_save_checkpoint` hook ([#6072](Lightning-AI/pytorch-lightning#6072)) - Added `RunningStage.SANITY_CHECKING` ([#4945](Lightning-AI/pytorch-lightning#4945)) - Added `TrainerState.{FITTING,VALIDATING,TESTING,PREDICTING,TUNING}` ([#4945](Lightning-AI/pytorch-lightning#4945)) - Added `Trainer.validate()` method to perform one evaluation epoch over the validation set ([#4948](Lightning-AI/pytorch-lightning#4948)) - Added `LightningEnvironment` for Lightning-specific DDP ([#5915](Lightning-AI/pytorch-lightning#5915)) - Added `teardown()` hook to LightningDataModule ([#4673](Lightning-AI/pytorch-lightning#4673)) - Added `auto_insert_metric_name` parameter to `ModelCheckpoint` ([#6277](Lightning-AI/pytorch-lightning#6277)) - Added arg to `self.log` that enables users to give custom names when dealing with multiple dataloaders ([#6274](Lightning-AI/pytorch-lightning#6274)) - Added `teardown` method to `BaseProfiler` to enable subclasses defining post-profiling steps outside of `__del__` ([#6370](Lightning-AI/pytorch-lightning#6370)) - Added `setup` method to `BaseProfiler` to enable subclasses defining pre-profiling steps for every process ([#6633](Lightning-AI/pytorch-lightning#6633)) - Added no return warning to predict ([#6139](Lightning-AI/pytorch-lightning#6139)) - Added `Trainer.predict` config validation ([#6543](Lightning-AI/pytorch-lightning#6543)) - Added `AbstractProfiler` interface ([#6621](Lightning-AI/pytorch-lightning#6621)) - Added support for including module names for forward in the autograd trace of `PyTorchProfiler` ([#6349](Lightning-AI/pytorch-lightning#6349)) - Added support for the PyTorch 1.8.1 autograd profiler ([#6618](Lightning-AI/pytorch-lightning#6618)) - Added `outputs` parameter to callback's `on_validation_epoch_end` & `on_test_epoch_end` hooks ([#6120](Lightning-AI/pytorch-lightning#6120)) - Added `configure_sharded_model` hook ([#6679](Lightning-AI/pytorch-lightning#6679)) - Added support for `precision=64`, enabling training with double precision ([#6595](Lightning-AI/pytorch-lightning#6595)) - Added support for DDP communication hooks ([#6736](Lightning-AI/pytorch-lightning#6736)) - Added `artifact_location` argument to `MLFlowLogger` which will be passed to the `MlflowClient.create_experiment` call ([#6677](Lightning-AI/pytorch-lightning#6677)) - Added `model` parameter to precision plugins' `clip_gradients` signature ([#6764](Lightning-AI/pytorch-lightning#6764)) ### Changed - Renamed `pytorch_lightning.callbacks.swa` to `pytorch_lightning.callbacks.stochastic_weight_avg` ([#6259](Lightning-AI/pytorch-lightning#6259)) - Refactor `RunningStage` and `TrainerState` usage ([#4945](Lightning-AI/pytorch-lightning#4945)) - Changed `trainer.evaluating` to return `True` if validating or testing ([#4945](Lightning-AI/pytorch-lightning#4945)) - Changed `setup()` and `teardown()` stage argument to take any of `{fit,validate,test,predict}` ([#6386](Lightning-AI/pytorch-lightning#6386)) - Changed profilers to save separate report files per state and rank ([#6621](Lightning-AI/pytorch-lightning#6621)) - Changed `PyTorchProfiler` to use `torch.autograd.profiler.record_function` to record functions ([#6349](Lightning-AI/pytorch-lightning#6349)) ### Deprecated - `period` has been deprecated in favor of `every_n_val_epochs` in the `ModelCheckpoint` callback ([#6146](Lightning-AI/pytorch-lightning#6146)) - Deprecated `trainer.running_sanity_check` in favor of `trainer.sanity_checking` ([#4945](Lightning-AI/pytorch-lightning#4945)) - Deprecated `Profiler(output_filename)` in favor of `dirpath` and `filename` ([#6621](Lightning-AI/pytorch-lightning#6621)) - Deprecated `PytorchProfiler(profiled_functions)` in favor of `record_functions` ([#6349](Lightning-AI/pytorch-lightning#6349)) - Deprecated metrics in favor of `torchmetrics` ([#6505](Lightning-AI/pytorch-lightning#6505), [#6530](Lightning-AI/pytorch-lightning#6530), [#6540](Lightning-AI/pytorch-lightning#6540), [#6547](Lightning-AI/pytorch-lightning#6547), [#6515](Lightning-AI/pytorch-lightning#6515), [#6572](Lightning-AI/pytorch-lightning#6572), [#6573](Lightning-AI/pytorch-lightning#6573), [#6584](Lightning-AI/pytorch-lightning#6584), [#6636](Lightning-AI/pytorch-lightning#6636), [#6637](Lightning-AI/pytorch-lightning#6637), [#6649](Lightning-AI/pytorch-lightning#6649), [#6659](Lightning-AI/pytorch-lightning#6659), ) ### Removed - Removed support for passing a bool value to `profiler` argument of Trainer ([#6164](Lightning-AI/pytorch-lightning#6164)) - Removed no return warning from val/test step ([#6139](Lightning-AI/pytorch-lightning#6139)) - Removed passing a `ModelCheckpoint` instance to `Trainer(checkpoint_callback)` ([#6166](Lightning-AI/pytorch-lightning#6166)) - Removed deprecated Trainer argument `enable_pl_optimizer` and `automatic_optimization` ([#6163](Lightning-AI/pytorch-lightning#6163)) - Removed deprecated metrics ([#6161](Lightning-AI/pytorch-lightning#6161)) * from `pytorch_lightning.metrics.functional.classification` removed `to_onehot`, `to_categorical`, `get_num_classes`, `roc`, `multiclass_roc`, `average_precision`, `precision_recall_curve`, `multiclass_precision_recall_curve` * from `pytorch_lightning.metrics.functional.reduction` removed `reduce`, `class_reduce` - Removed deprecated `ModelCheckpoint` arguments `prefix`, `mode="auto"` ([#6162](Lightning-AI/pytorch-lightning#6162)) - Removed `mode='auto'` from `EarlyStopping` ([#6167](Lightning-AI/pytorch-lightning#6167)) - Removed legacy references for magic keys in the `Result` object ([#6016](Lightning-AI/pytorch-lightning#6016)) - Removed deprecated `LightningModule` `hparams` setter ([#6207](Lightning-AI/pytorch-lightning#6207)) - Removed legacy code to log or include metrics in the progress bar by returning them in a dict with the `"log"/"progress_bar"` magic keys. Use `self.log` instead ([#6734](Lightning-AI/pytorch-lightning#6734)) - Removed `optimizer_idx` argument from `training_step` in manual optimization ([#6093](Lightning-AI/pytorch-lightning#6093)) ### Fixed - Set better defaults for `rank_zero_only.rank` when training is launched with SLURM and torchelastic ([#6802](Lightning-AI/pytorch-lightning#6802)) - Made the `Plugin.reduce` method more consistent across all Plugins to reflect a mean-reduction by default ([#6011](Lightning-AI/pytorch-lightning#6011)) - Move lightning module to correct device type when using LightningDistributedWrapper ([#6070](Lightning-AI/pytorch-lightning#6070)) - Do not print top-k verbose log with `ModelCheckpoint(monitor=None)` ([#6109](Lightning-AI/pytorch-lightning#6109)) - Fixed csv extension check ([#6436](Lightning-AI/pytorch-lightning#6436)) - Fixed `ModelCheckpoint(monitor=None, save_last=True)` not saving checkpoints ([#6136](Lightning-AI/pytorch-lightning#6136)) - Fixed `ModelCheckpoint(save_top_k=0, save_last=True)` not saving the `last` checkpoint ([#6136](Lightning-AI/pytorch-lightning#6136)) - Fixed `.teardown(stage='fit')` getting called during `trainer.test` ([#6386](Lightning-AI/pytorch-lightning#6386)) - Fixed `.on_fit_{start,end}()` getting called during `trainer.test` ([#6386](Lightning-AI/pytorch-lightning#6386)) - Fixed LightningModule `all_gather` on cpu tensors ([#6416](Lightning-AI/pytorch-lightning#6416)) - Fixed torch distributed not available in setup hook for DDP ([#6506](Lightning-AI/pytorch-lightning#6506)) - Fixed `EarlyStopping` logic when `min_epochs` or `min_steps` requirement is not met ([#6705](Lightning-AI/pytorch-lightning#6705)) ## [1.2.7] - 2021-04-06 ### Fixed - Fixed resolve a bug with omegaconf and xm.save ([#6741](Lightning-AI/pytorch-lightning#6741)) - Fixed an issue with IterableDataset when __len__ is not defined ([#6828](Lightning-AI/pytorch-lightning#6828)) - Sanitize None params during pruning ([#6836](Lightning-AI/pytorch-lightning#6836)) - Enforce an epoch scheduler interval when using SWA ([#6588](Lightning-AI/pytorch-lightning#6588)) - Fixed TPU Colab hang issue, post training ([#6816](Lightning-AI/pytorch-lightning#6816)) - Fixed a bug where `TensorBoardLogger` would give a warning and not log correctly to a symbolic link `save_dir` ([#6730](Lightning-AI/pytorch-lightning#6730)) ## [1.2.6] - 2021-03-30 ### Changed - Changed the behavior of `on_epoch_start` to run at the beginning of validation & test epoch ([#6498](Lightning-AI/pytorch-lightning#6498)) ### Removed - Removed legacy code to include `step` dictionary returns in `callback_metrics`. Use `self.log_dict` instead. ([#6682](Lightning-AI/pytorch-lightning#6682)) ### Fixed - Fixed `DummyLogger.log_hyperparams` raising a `TypeError` when running with `fast_dev_run=True` ([#6398](Lightning-AI/pytorch-lightning#6398)) - Fixed error on TPUs when there was no `ModelCheckpoint` ([#6654](Lightning-AI/pytorch-lightning#6654)) - Fixed `trainer.test` freeze on TPUs ([#6654](Lightning-AI/pytorch-lightning#6654)) - Fixed a bug where gradients were disabled after calling `Trainer.predict` ([#6657](Lightning-AI/pytorch-lightning#6657)) - Fixed bug where no TPUs were detected in a TPU pod env ([#6719](Lightning-AI/pytorch-lightning#6719)) ## [1.2.5] - 2021-03-23 ### Changed - Update Gradient Clipping for the TPU Accelerator ([#6576](Lightning-AI/pytorch-lightning#6576)) - Refactored setup for typing friendly ([#6590](Lightning-AI/pytorch-lightning#6590)) ### Fixed - Fixed a bug where `all_gather` would not work correctly with `tpu_cores=8` ([#6587](Lightning-AI/pytorch-lightning#6587)) - Fixed comparing required versions ([#6434](Lightning-AI/pytorch-lightning#6434)) - Fixed duplicate logs appearing in console when using the python logging module ([#6275](Lightning-AI/pytorch-lightning#6275)) - Added Autocast in validation, test and predict modes for Native AMP ([#6565](Lightning-AI/pytorch-lightning#6565)) Reviewed By: shuyingsunshine21 Differential Revision: D27528929 fbshipit-source-id: 311c88f71461c2c79bbf185e28d7a6d683ccc26f

asnorkin requested review from Borda, SeanNaren, awaelchli, justusschock, tchaton and williamFalcon as code owners January 11, 2021 20:56

rohitgr7 reviewed Jan 11, 2021

View reviewed changes

pytorch_lightning/core/lightning.py Show resolved Hide resolved

Borda changed the base branch from master to release/1.2-dev January 11, 2021 21:17

Borda requested review from ananyahjha93, edenlightning and teddykoker as code owners January 11, 2021 21:17

Borda added the feature Is an improvement or enhancement label Jan 11, 2021

Borda added this to the 1.2 milestone Jan 11, 2021

Borda force-pushed the feature/5275_clean_progress_bar_print branch from 83cf8f7 to 1a62855 Compare January 11, 2021 21:21

awaelchli reviewed Jan 12, 2021

View reviewed changes

rohitgr7 approved these changes Jan 15, 2021

View reviewed changes

carmocca reviewed Jan 19, 2021

View reviewed changes

docs/source/multi_gpu.rst Show resolved Hide resolved

tests/metrics/test_metric.py Outdated Show resolved Hide resolved

rohitgr7 force-pushed the feature/5275_clean_progress_bar_print branch from 3ac7c60 to c1190fb Compare January 19, 2021 17:30

rohitgr7 requested review from awaelchli and carmocca January 19, 2021 17:31

awaelchli approved these changes Jan 20, 2021

View reviewed changes

pytorch_lightning/callbacks/progress.py Outdated Show resolved Hide resolved

pytorch_lightning/callbacks/progress.py Outdated Show resolved Hide resolved

Borda reviewed Jan 24, 2021

View reviewed changes

pytorch_lightning/callbacks/progress.py Outdated Show resolved Hide resolved

mergify bot added the has conflicts label Feb 3, 2021

asnorkin force-pushed the feature/5275_clean_progress_bar_print branch from 7a1f1c8 to 997062f Compare February 8, 2021 14:47

mergify bot removed the has conflicts label Feb 8, 2021

edenlightning modified the milestones: 1.2, 1.3 Feb 8, 2021

asnorkin force-pushed the feature/5275_clean_progress_bar_print branch from d1c5bb2 to f7f41b0 Compare February 9, 2021 18:34

asnorkin requested a review from Borda February 10, 2021 15:20

mergify bot added has conflicts and removed has conflicts labels Feb 16, 2021

tchaton and others added 14 commits February 17, 2021 10:42

Trainer.test should return only test metrics (Lightning-AI#5214)

85ea319

* resolve bug * merge tests

Fix metric state reset (Lightning-AI#5273)

5c112f0

* Fix metric state reset * Fix test * Improve formatting Co-authored-by: Ananya Harsh Jha <[email protected]>

print() method added to ProgressBar

2072b61

printing alongside progress bar added to LightningModule.print()

539e2e6

LightningModule.print() method documentation updated

023993b

ProgressBarBase.print() stub added

dbba2d7

stub

1bfc866

add progress bar tests

0ee4abc

fix isort

a70b7c0

Progress Callback fixes

a13a218

test_metric.py duplicate DummyList removed

19411f7

PEP and isort fixes

68be2b8

CHANGELOG updated

ab908cf

test_progress_bar_print win linesep fix

599aa6a

asnorkin force-pushed the feature/5275_clean_progress_bar_print branch from 78aa464 to 599aa6a Compare February 17, 2021 07:47

mergify bot added has conflicts and removed has conflicts labels Feb 17, 2021

Merge branch 'master' into feature/5275_clean_progress_bar_print

b9d61c6

mergify bot removed the has conflicts label Feb 18, 2021

test_progress_bar.py remove whitespaces

331aacc

SeanNaren approved these changes Feb 18, 2021

View reviewed changes

carmocca approved these changes Feb 18, 2021

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Update CHANGELOG.md

c152e0c

rohitgr7 enabled auto-merge (squash) February 18, 2021 16:54

tchaton approved these changes Feb 22, 2021

View reviewed changes

rohitgr7 merged commit 423ecf9 into Lightning-AI:master Feb 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/5275 clean progress bar print #5470

Feature/5275 clean progress bar print #5470

Uh oh!

asnorkin commented Jan 11, 2021 •

edited

Loading

Uh oh!

Uh oh!

awaelchli left a comment

Uh oh!

codecov bot commented Jan 12, 2021 •

edited

Loading

Uh oh!

rohitgr7 left a comment •

edited

Loading

Uh oh!

carmocca left a comment

Uh oh!

Uh oh!

Uh oh!

awaelchli left a comment

Uh oh!

Uh oh!

Uh oh!

Borda left a comment

Uh oh!

Uh oh!

pep8speaks commented Feb 18, 2021 •

edited

Loading

Uh oh!

Uh oh!

tchaton left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Feature/5275 clean progress bar print #5470

Feature/5275 clean progress bar print #5470

Uh oh!

Conversation

asnorkin commented Jan 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

PR review

Did you have fun?

Uh oh!

Uh oh!

awaelchli left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jan 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rohitgr7 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

carmocca left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

awaelchli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Borda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pep8speaks commented Feb 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2021-02-18 15:32:17 UTC

Uh oh!

Uh oh!

tchaton left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

asnorkin commented Jan 11, 2021 •

edited

Loading

codecov bot commented Jan 12, 2021 •

edited

Loading

rohitgr7 left a comment •

edited

Loading

pep8speaks commented Feb 18, 2021 •

edited

Loading