Skip to content

Commit 37b16e9

Browse files
authored
Merge branch 'master' into ci/black
2 parents 1bcc820 + aadd2a9 commit 37b16e9

36 files changed

+410
-201
lines changed

CHANGELOG.md

Lines changed: 88 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,91 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
66

77

8-
## [1.4.0] - 2021-MM-DD
8+
## [unReleased] - 2021-MM-DD
9+
10+
### Added
11+
12+
-
13+
14+
15+
-
16+
17+
18+
-
19+
20+
21+
-
22+
23+
24+
-
25+
26+
### Changed
27+
28+
- Replace `iteration_count` and other index attributes in the loops with progress dataclasses ([#8477](https://github.com/PyTorchLightning/pytorch-lightning/pull/8477))
29+
30+
31+
- Load ckpt path when model provided in validate/test/predict ([#8352](https://github.com/PyTorchLightning/pytorch-lightning/pull/8352)))
32+
33+
34+
35+
-
36+
37+
38+
-
39+
40+
41+
-
42+
43+
### Deprecated
44+
45+
-
46+
47+
48+
-
49+
50+
51+
-
52+
53+
54+
-
55+
56+
57+
-
58+
59+
### Removed
60+
61+
-
62+
63+
64+
-
65+
66+
67+
-
68+
69+
70+
-
71+
72+
73+
-
74+
75+
### Fixed
76+
77+
-
78+
79+
80+
-
81+
82+
83+
-
84+
85+
86+
-
87+
88+
89+
-
90+
91+
92+
## [1.4.0] - 2021-07-27
993

1094
### Added
1195

@@ -153,7 +237,6 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
153237
- Moved `DeviceDtypeModuleMixin` and `HyperparametersMixin` mixin to `core` ([#8396](https://github.com/PyTorchLightning/pytorch-lightning/pull/8396))
154238
- Return the `default_root_dir` as the `log_dir` when the logger is a `LoggerCollection` ([#8187](https://github.com/PyTorchLightning/pytorch-lightning/pull/8187))
155239

156-
157240
### Deprecated
158241

159242
- Deprecated `LightningModule.loaded_optimizer_states_dict` ([#8229](https://github.com/PyTorchLightning/pytorch-lightning/pull/8229))
@@ -173,7 +256,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
173256
- Deprecated the `Trainer.disable_validation` property in favor of `not Trainer.enable_validation` ([#8291](https://github.com/PyTorchLightning/pytorch-lightning/pull/8291))
174257
- Deprecated `mode` parameter in `ModelSummary` in favor of `max_depth` ([#8062](https://github.com/PyTorchLightning/pytorch-lightning/pull/8062))
175258
- Deprecated `reload_dataloaders_every_epoch` argument of `Trainer` in favor of `reload_dataloaders_every_n_epochs` ([#5043](https://github.com/PyTorchLightning/pytorch-lightning/pull/5043))
176-
259+
- Deprecated `distributed_backend` argument for `Trainer` ([#8575](https://github.com/PyTorchLightning/pytorch-lightning/pull/8575))
177260

178261
### Removed
179262

@@ -191,7 +274,6 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
191274
- Removed DeepSpeed FP16 Exception as FP32 is now supported ([#8462](https://github.com/PyTorchLightning/pytorch-lightning/pull/8462))
192275
- Removed environment variable `PL_EXP_VERSION` from DDP subprocesses ([7403](https://github.com/PyTorchLightning/pytorch-lightning/pull/7403))
193276

194-
195277
### Fixed
196278

197279
- Fixed the `GPUStatsMonitor` callbacks to use the correct GPU IDs if `CUDA_VISIBLE_DEVICES` set ([#8260](https://github.com/PyTorchLightning/pytorch-lightning/pull/8260))
@@ -233,6 +315,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
233315
- Fixed `accumulate_grad_batches` not been recomputed during model reload ([#5334](https://github.com/PyTorchLightning/pytorch-lightning/pull/5334))
234316
- Fixed a `TypeError` when wrapping optimizers in the `HorovodPlugin` and running `Trainer.test` ([#7840](https://github.com/PyTorchLightning/pytorch-lightning/pull/7840))
235317
- Fixed `BackboneFinetuning` restoration ([#8501](https://github.com/PyTorchLightning/pytorch-lightning/pull/8501))
318+
- Fixed `lr_scheduler` with metric (e.g. `torch.optim.lr_scheduler.ReduceLROnPlateau`) when using `automatic_optimization = False` ([#7643](https://github.com/PyTorchLightning/pytorch-lightning/pull/7643))
319+
- Fixed `DeepSpeed` breaking with no schedulers ([#8580](https://github.com/PyTorchLightning/pytorch-lightning/pull/8580))
236320

237321

238322
## [1.3.8] - 2021-07-01

README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -118,22 +118,22 @@ pip install pytorch-lightning
118118
conda install pytorch-lightning -c conda-forge
119119
```
120120

121-
#### Install stable 1.3.x
121+
#### Install stable 1.4.x
122122

123-
the actual status of 1.3 [stable] is following:
123+
the actual status of 1.4 [stable] is following:
124124

125-
![CI base testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20base%20testing/badge.svg?branch=release%2F1.3.x&event=push)
126-
![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=release%2F1.3.x&event=push)
127-
![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=release%2F1.3.x&event=push)
128-
![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=release%2F1.3.x&event=push)
129-
![Docs check](https://github.com/PyTorchLightning/pytorch-lightning/workflows/Docs%20check/badge.svg?branch=release%2F1.3.x&event=push)
125+
![CI base testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20base%20testing/badge.svg?branch=release%2F1.4.x&event=push)
126+
![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=release%2F1.4.x&event=push)
127+
![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=release%2F1.4.x&event=push)
128+
![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=release%2F1.4.x&event=push)
129+
![Docs check](https://github.com/PyTorchLightning/pytorch-lightning/workflows/Docs%20check/badge.svg?branch=release%2F1.4.x&event=push)
130130

131131
Install future release from the source
132132
```bash
133-
pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@release/1.3.x --upgrade
133+
pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@release/1.4.x --upgrade
134134
```
135135

136-
#### Install bleeding-edge - future 1.4
136+
#### Install bleeding-edge - future 1.5
137137

138138
Install nightly from the source (no guarantees)
139139
```bash

_notebooks

docs/source/clouds/cloud_training.rst

Lines changed: 30 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,39 +4,45 @@
44
Cloud Training
55
##############
66

7-
Lightning has a native solution for training on AWS/GCP at scale.
8-
Go to `grid.ai <https://www.grid.ai/>`_ to create an account.
7+
Lightning makes it easy to scale your training, without the boilerplate.
8+
If you want to train your models on the cloud, without dealing with engineering infrastructure and servers, you can try `Grid.ai <https://www.grid.ai/>`_.
99

10-
We've designed Grid to work seamlessly with Lightning, without needing to make ANY code changes.
10+
Developed by the creators of `PyTorch Lightning <https://www.pytorchlightning.ai/>`_, Grid is a platform that allows you to:
1111

12-
To use Grid, replace ``python`` in your regular command:
1312

14-
.. code-block:: bash
13+
- **Scale your models to multi-GPU and multiple nodes** instantly with interactive sessions
14+
- **Run Hyperparameter Sweeps on 100s of GPUs** in one command
15+
- **Upload huge datasets** for availability at scale
16+
- **Iterate faster and cheaper**, you only pay for what you need
17+
18+
19+
****************
20+
Training on Grid
21+
****************
22+
23+
.. raw:: html
1524

16-
python my_model.py --learning_rate 1e-6 --layers 2 --gpus 4
25+
<video width="50%" max-width="400px" controls
26+
poster="https://grid-docs.s3.us-east-2.amazonaws.com/grid.png"
27+
src="https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/pl_docs/grid.mp4"></video>
1728

18-
To use the ``grid run`` command:
29+
|
30+
31+
You can launch any Lightning model on Grid using the Grid `CLI <https://pypi.org/project/lightning-grid/>`_:
1932

2033
.. code-block:: bash
2134
22-
grid run --gpus 4 my_model.py --learning_rate 'uniform(1e-6, 1e-1, 20)' --layers '[2, 4, 8, 16]'
35+
grid run --instance_type v100 --gpus 4 my_model.py --gpus 4 --learning_rate 'uniform(1e-6, 1e-1, 20)' --layers '[2, 4, 8, 16]'
36+
37+
You can also start runs or interactive sessions from the `Grid platform <https://platform.grid.ai>`_, where you can upload datasets, view artifacts, view the logs, the cost, log into tensorboard, and so much more.
38+
2339

24-
The above command will launch (20 * 4) experiments, each running on 4 GPUs (320 GPUs!) - by making ZERO changes to
25-
your code.
40+
**********
41+
Learn More
42+
**********
2643

27-
The ``uniform`` command is part of our new expressive syntax which lets you construct hyperparameter combinations
28-
using over 20+ distributions, lists, etc. Of course, you can also configure all of this using yamls which
29-
can be dynamically assembled at runtime.
44+
`Sign up for Grid <http://platform.grid.ai>`_ and receive free credits to get you started!
3045

31-
***************
32-
Grid Highlights
33-
***************
46+
`Grid in 3 minutes <https://docs.grid.ai/#introduction>`_
3447

35-
* Run any public or private repository with Grid, or use an interactive session.
36-
* Grid allocates all the machines and GPUs you need on demand, so you only pay for what you need when you need it.
37-
* Grid handles all the other parts of developing and training at scale: artifacts, logs, metrics, etc.
38-
* Grid works with the experiment manager of your choice, no code changes needed.
39-
* Use Grid Datastores- high-performance, low-latency, versioned datasets.
40-
* Attach Datastores to a Run so you don't have to keep downloading datasets
41-
* Use Grid Sessions for fast prototyping on a cloud machine of your choice
42-
* For more information check the `grid documentation <https://docs.grid.ai/>`_
48+
`Grid.ai Terms of Service <https://www.grid.ai/terms-of-service/>`_

docs/source/common/optimizers.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,21 @@ If you want to call ``lr_scheduler.step()`` every ``n`` steps/epochs, do the fol
234234
if self.trainer.is_last_batch and (self.trainer.current_epoch + 1) % n == 0:
235235
sch.step()
236236

237+
If you want to call schedulers that require a metric value after each epoch, consider doing the following:
238+
239+
.. testcode::
240+
241+
def __init__(self):
242+
super().__init__()
243+
self.automatic_optimization = False
244+
245+
def training_epoch_end(self, outputs):
246+
sch = self.lr_schedulers()
247+
248+
# If the selected scheduler is a ReduceLROnPlateau scheduler.
249+
if isinstance(sch, torch.optim.lr_scheduler.ReduceLROnPlateau):
250+
sch.step(self.trainer.callback_metrics["loss"])
251+
237252
-----
238253

239254
Use closure for LBFGS-like optimizers

docs/source/common/test_set.rst

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,12 @@ To run the test set after training completes, use this method.
2020
trainer.fit(model)
2121
2222
# (1) load the best checkpoint automatically (lightning tracks this for you)
23-
trainer.test()
23+
trainer.test(ckpt_path='best')
2424
25-
# (2) don't load a checkpoint, instead use the model with the latest weights
26-
trainer.test(ckpt_path=None)
27-
28-
# (3) test using a specific checkpoint
25+
# (2) test using a specific checkpoint
2926
trainer.test(ckpt_path="/path/to/my_checkpoint.ckpt")
3027
31-
# (4) test with an explicit model (will use this model and not load a checkpoint)
28+
# (3) test with an explicit model (will use this model and not load a checkpoint)
3229
trainer.test(model)
3330
3431
----------

docs/source/common/trainer.rst

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ auto_scale_batch_size
335335
Automatically tries to find the largest batch size that fits into memory,
336336
before any training.
337337

338-
.. code-block::
338+
.. code-block:: python
339339
340340
# default used by the Trainer (no scaling of batch size)
341341
trainer = Trainer(auto_scale_batch_size=None)
@@ -1353,7 +1353,6 @@ By setting to False, you have to add your own distributed sampler:
13531353

13541354
.. code-block:: python
13551355
1356-
13571356
# in your LightningModule or LightningDataModule
13581357
def train_dataloader(self):
13591358
# default used by the Trainer
@@ -1575,7 +1574,7 @@ Can specify as float or int.
15751574
trainer = Trainer(val_check_interval=1000)
15761575

15771576

1578-
.. code-block::
1577+
.. code-block:: python
15791578
15801579
# Here is the computation to estimate the total number of batches seen within an epoch.
15811580

docs/source/conf.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,11 @@ def _transform_changelog(path_in: str, path_out: str) -> None:
142142
# List of patterns, relative to source directory, that match files and
143143
# directories to ignore when looking for source files.
144144
# This pattern also affects html_static_path and html_extra_path.
145-
exclude_patterns = [f"{FOLDER_GENERATED}/PULL_REQUEST_TEMPLATE.md", "notebooks/course_UvA-DL/*", "notebooks/template*"]
145+
exclude_patterns = [
146+
f"{FOLDER_GENERATED}/PULL_REQUEST_TEMPLATE.md",
147+
"notebooks/course_UvA-DL/*",
148+
"notebooks/sample-template*",
149+
]
146150

147151
# The name of the Pygments (syntax highlighting) style to use.
148152
pygments_style = None

docs/source/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ PyTorch Lightning Documentation
6262
notebooks/lightning_examples/datamodules.ipynb
6363
notebooks/lightning_examples/cifar10-baseline.ipynb
6464
notebooks/lightning_examples/basic-gan.ipynb
65+
notebooks/lightning_examples/mnist-tpu-training.ipynb
6566
notebooks/lightning_examples/text-transformers.ipynb
6667
notebooks/lightning_examples/reinforce-learning-DQN.ipynb
6768
notebooks/lightning_examples/augmentation_kornia.ipynb

pytorch_lightning/__about__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import time
22

33
_this_year = time.strftime("%Y")
4-
__version__ = "1.4.0rc2"
4+
__version__ = "1.5.0dev"
55
__author__ = "William Falcon et al."
66
__author_email__ = "[email protected]"
77
__license__ = "Apache-2.0"

0 commit comments

Comments
 (0)