Merge branch 'master' into feature/migration-2

Author: awaelchli

.azure-pipelines/gpu-tests.yml

@@ -51,7 +51,7 @@ jobs:
     - bash: |
         python -c "fname = 'requirements/extra.txt' ; lines = [line for line in open(fname).readlines() if 'horovod' not in line] ; open(fname, 'w').writelines(lines)"
         pip install fairscale>=0.3.4
-        pip install deepspeed>=0.4.0 -U
+        pip install "deepspeed==0.4.3" # FIXME: bug with >= 0.4.4
         pip install . --requirement requirements/devel.txt
         pip list
       displayName: 'Install dependencies'

.azure-pipelines/ipu-tests.yml

@@ -72,26 +72,11 @@ jobs:
         python -c "import poptorch; print(poptorch.__version__)"
       displayName: "Check poptorch installation"
 
-    - bash: |
-        wget https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip -P legacy/
-        unzip -o legacy/checkpoints.zip -d legacy/
-        ls -l legacy/checkpoints/
-      displayName: 'Get legacy checkpoints'
-
     - bash: |
         source ${{ variables.poplar_sdk }}/poplar-ubuntu*/enable.sh
         source ${{ variables.poplar_sdk }}/popart-ubuntu*/enable.sh
         export POPTORCH_WAIT_FOR_IPU=1
-        python -m coverage run --source pytorch_lightning -m pytest pytorch_lightning tests -v --junitxml=$(Build.StagingDirectory)/test-results.xml --durations=50
+        python -m coverage run --source pytorch_lightning -m pytest tests/accelerators/test_ipu.py -v --junitxml=$(Build.StagingDirectory)/test-results.xml --durations=50
       env:
         MKL_THREADING_LAYER: "GNU"
       displayName: 'Testing: standard'
-
-    - bash: |
-        source ${{ variables.poplar_sdk }}/poplar-ubuntu*/enable.sh
-        source ${{ variables.poplar_sdk }}/popart-ubuntu*/enable.sh
-        export POPTORCH_WAIT_FOR_IPU=1
-        bash tests/special_tests.sh
-      env:
-        MKL_THREADING_LAYER: "GNU"
-      displayName: 'Testing: special'

.codecov.yml

@@ -36,7 +36,6 @@ coverage:
     # https://codecov.readme.io/v1.0/docs/commit-status
     project:
       default:
-        against: auto
         target: 99% # specify the target coverage for each commit status
         threshold: 30% # allow this little decrease on project
         # https://github.com/codecov/support/wiki/Filtering-Branches
@@ -45,11 +44,14 @@ coverage:
     # https://github.com/codecov/support/wiki/Patch-Status
     patch:
       default:
-        against: auto
         target: 50% # specify the target "X%" coverage to hit
         # threshold: 50% # allow this much decrease on patch
     changes: false
 
+# https://docs.codecov.com/docs/github-checks#disabling-github-checks-patch-annotations
+github_checks:
+    annotations: false
+
 parsers:
   gcov:
     branch_detection:

.github/BECOMING_A_CORE_CONTRIBUTOR.md

@@ -6,22 +6,26 @@ We're currently recruiting for a team of 5 core maintainers.
 As a core maintainer you will have a strong say in the direction of the project. Big changes will require a majority of maintainers to agree.
 
 ## Code of conduct
+
 First and foremost, you'll be evaluated against [these core values](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md). Any code we commit or feature we add needs to align with those core values.
 
 ## The bar for joining the team
+
 Lightning is being used to solve really hard problems at the top AI labs in the world. As such, the bar for adding team members is extremely high. Candidates must have solid engineering skills, have a good eye for user experience, and must be a power user of Lightning and PyTorch.
 
 With that said, the Lightning team will be diverse and a reflection of an inclusive AI community. You don't have to be an engineer to contribute! Scientists with great usability intuition and PyTorch ninja skills are welcomed!
 
 ## Responsibilities:
+
 The responsibilities mainly revolve around 3 things.
 
 ### Github issues
+
 - Here we want to help users have an amazing experience. These range from questions from new people getting into DL to questions from researchers about doing something esoteric with Lightning
-Often, these issues require some sort of bug fix, document clarification or new functionality to be scoped out.
+  Often, these issues require some sort of bug fix, document clarification or new functionality to be scoped out.
 
 - To become a core member you must resolve at least 10 Github issues which align with the API design goals for Lightning. By the end of these 10 issues I should feel comfortable in the way you answer user questions
-Pleasant/helpful tone.
+  Pleasant/helpful tone.
 
 - Can abstract from that issue or bug into functionality that might solve other related issues or makes the platform more flexible.
 
@@ -37,22 +41,25 @@ Pleasant/helpful tone.
 - Ask yourself, could a non-engineer understand what’s happening here?
 - Make sure new tests are written
 - Is this NECESSARY for Lightning? There are some PRs which are just purely about adding engineering complexity which have no place in Lightning.
-Guidance
+  Guidance
 - Some other PRs are for people who are wanting to get involved and add something unnecessary. We do want their help though! So don’t approve the PR, but direct them to a Github issue that they might be interested in helping with instead!
 - To be considered for core contributor, please review 10 PRs and help the authors land it on master. Once you've finished the review, ping me
-for a sanity check. At the end of 10 PRs if your PR reviews are inline with expectations described above, then you can merge PRs on your own going forward,
-otherwise we'll do a few more until we're both comfortable :)
+  for a sanity check. At the end of 10 PRs if your PR reviews are inline with expectations described above, then you can merge PRs on your own going forward,
+  otherwise we'll do a few more until we're both comfortable :)
 
 ### Project directions
+
 There are some big decisions which the project must make. For these I expect core contributors to have something meaningful to add if it’s their area of expertise.
 
 ### Diversity
+
 Lightning should reflect the broader community it serves. As such we should have scientists/researchers from
 different fields contributing!
 
 The first 5 core contributors will fit this profile. Thus if you overlap strongly with experiences and expertise as someone else on the team, you might have to wait until the next set of contributors are added.
 
 ### Summary: Requirements to apply
+
 The goal is to be inline with expectations for solving issues by the last one so you can do them on your own. If not, I might ask you to solve a few more specific ones.
 
 - Solve 10+ Github issues.

.github/CODE_OF_CONDUCT.md

@@ -14,22 +14,22 @@ appearance, race, religion, or sexual identity and orientation.
 Examples of behavior that contributes to creating a positive environment
 include:
 
-* Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
-* Focusing on what is best for the community
-* Showing empathy towards other community members
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
 
 Examples of unacceptable behavior by participants include:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or
- advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or electronic
- address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a
- professional setting
+- The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+- Trolling, insulting/derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
 
 ## Our Responsibilities
 
@@ -70,7 +70,7 @@ members of the project's leadership.
 This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
 available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
 
-[homepage]: https://www.contributor-covenant.org
-
 For answers to common questions about this code of conduct, see
 https://www.contributor-covenant.org/faq
+
+[homepage]: https://www.contributor-covenant.org

.github/CONTRIBUTING.md

@@ -56,7 +56,7 @@ As a researcher, you can't have any part of your code going wrong. So, make thor
 
 Have a favorite feature from other libraries like fast.ai or transformers? Those should just work with lightning as well. Grab your favorite model or learning rate scheduler from your favorite library and run it in Lightning.
 
----
+______________________________________________________________________
 
 ## Contribution Types
 
@@ -73,26 +73,28 @@ A lot of good work has already been done in project mechanics (requirements.txt,
    - Add details on how to reproduce the issue - a minimal test case is always best, colab is also great.
      Note, that the sample code shall be minimal and if needed with publicly available data.
 
-2. Try to fix it or recommend a solution. We highly recommend to use test-driven approach:
+1. Try to fix it or recommend a solution. We highly recommend to use test-driven approach:
 
    - Convert your minimal code example to a unit/integration test with assert on expected results.
    - Start by debugging the issue... You can run just this particular test in your IDE and draft a fix.
    - Verify that your test case fails on the master branch and only passes with the fix applied.
 
-3. Submit a PR!
+1. Submit a PR!
 
-_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution, and we can help you or finish it with you :]_
+_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution, and we can help you or finish it with you :\]_
 
 ### New Features:
 
 1. Submit a GitHub issue - describe what is the motivation of such feature (adding the use case, or an example is helpful).
-2. Determine the feature scope with us.
-3. Submit a PR! We recommend test driven approach to adding new features as well:
+
+1. Determine the feature scope with us.
+
+1. Submit a PR! We recommend test driven approach to adding new features as well:
 
    - Write a test for the functionality you want to add.
    - Write the functional code until the test passes.
 
-4. Add/update the relevant tests!
+1. Add/update the relevant tests!
 
 - [This PR](https://github.com/PyTorchLightning/pytorch-lightning/pull/2671) is a good example for adding a new metric, and [this one for a new logger](https://github.com/PyTorchLightning/pytorch-lightning/pull/2721).
 
@@ -102,12 +104,14 @@ Want to keep Lightning healthy? Love seeing those green tests? So do we! How to
 
 Most of the tests in PyTorch Lightning train a trial MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). The tests expect the model to perform to a reasonable degree of testing accuracy to pass. Want to add a new test case and not sure how? [Talk to us!](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ)
 
----
+______________________________________________________________________
 
 ## Guidelines
 
 ### Developments scripts
+
 To build the documentation locally, simply execute the following commands from project root (only for Unix):
+
 - `make clean` cleans repo from temp/generated files
 - `make docs` builds documentation under _docs/build/html_
 - `make test` runs all project's tests with coverage
@@ -121,7 +125,7 @@ In case you adding new dependencies, make sure that they are compatible with the
 ### Coding Style
 
 1. Use f-strings for output formation (except logging when we stay with lazy `logging.info("Hello %s!", name)`.
-2. You can use `pre-commit` to make sure your code style is correct.
+1. You can use `pre-commit` to make sure your code style is correct.
 
 ### Documentation
 
@@ -191,6 +195,15 @@ python -m pip install ".[dev, examples]"
 python -m pip install pre-commit
 ```
 
+Additionally, for testing backward compatibility with older versions of PyTorch Lightning, you also need to download all saved version-checkpoints from the public AWS storage. Run the following script to get all saved version-checkpoints:
+
+```bash
+wget https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip -P legacy/
+unzip -o legacy/checkpoints.zip -d legacy/
+```
+
+Note: These checkpoints are generated to set baselines for maintaining backward compatibility with legacy versions of PyTorch Lightning. Details of checkpoints for back-compatibility can be found [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/legacy/README.md).
+
 You can run the full test-case in your terminal via this make script:
 
 ```bash
@@ -223,18 +236,18 @@ We welcome any useful contribution! For your convenience here's a recommended wo
    - Create a branch and prepare your changes.
    - Tip: do not work with your master directly, it may become complicated when you need to rebase.
    - Tip: give your PR a good name! It will be useful later when you may work on multiple tasks/PRs.
-2. Test your code!
+1. Test your code!
    - It is always good practice to start coding by creating a test case, verifying it breaks with current behaviour, and passes with your new changes.
    - Make sure your new tests cover all different edge cases.
    - Make sure all exceptions are handled.
-3. Create a "Draft PR" which is clearly marked, to let us know you don't need feedback yet.
-4. When you feel ready for integrating your work, mark your PR "Ready for review".
+1. Create a "Draft PR" which is clearly marked, to let us know you don't need feedback yet.
+1. When you feel ready for integrating your work, mark your PR "Ready for review".
    - Your code should be readable and follow the project's design principles.
    - Make sure all tests are passing.
    - Make sure you add a GitHub issue to your PR.
-5. Use tags in PR name for following cases:
-   - **[blocked by #<number>]** if your work is dependent on other PRs.
-   - **[wip]** when you start to re-edit your work, mark it so no one will accidentally merge it in meantime.
+1. Use tags in PR name for following cases:
+   - **\[blocked by #<number>\]** if your work is dependent on other PRs.
+   - **\[wip\]** when you start to re-edit your work, mark it so no one will accidentally merge it in meantime.
 
 ### Question & Answer
 
@@ -275,20 +288,21 @@ git rebase upstream/master
 git push -f
 ```
 
-#### How to add new tests?**
+#### How to add new tests?
 
 We are using [pytest](https://docs.pytest.org/en/stable/) in Pytorch Lightning.
 
 Here are tutorials:
-* (recommended) [Visual Testing with pytest](https://www.youtube.com/playlist?list=PLCTHcU1KoD99Rim2tzg-IhYY2iu9FFvNo) from JetBrains on YouTube
-* [Effective Python Testing With Pytest](https://realpython.com/pytest-python-testing/) article on realpython.com
+
+- (recommended) [Visual Testing with pytest](https://www.youtube.com/playlist?list=PLCTHcU1KoD99Rim2tzg-IhYY2iu9FFvNo) from JetBrains on YouTube
+- [Effective Python Testing With Pytest](https://realpython.com/pytest-python-testing/) article on realpython.com
 
 Here is the process to create a new test
 
-* 0. Optional: Follow tutorials !
-* 1. Find a file in tests/ which match what you want to test. If none, create one.
-* 2. Use this template to get started !
-* 3. Use `BoringModel and derivates to test out your code`.
+- 0. Optional: Follow tutorials !
+- 1. Find a file in tests/ which match what you want to test. If none, create one.
+- 2. Use this template to get started !
+- 3. Use `BoringModel and derivates to test out your code`.
 
 ```python
 # TEST SHOULD BE IN YOUR FILE: tests/..../...py
@@ -318,47 +332,49 @@ def test_explain_what_is_being_tested(tmpdir):
     # assert the behaviour is correct.
     assert ...
 ```
+
 run our/your test with
+
 ```bash
 python -m pytest tests/..../...py::test_explain_what_is_being_tested --verbose --capture=no
 ```
 
-
 #### How to fix PR with mixed base and target branches?
 
 Sometimes you start your PR as a bug-fix but it turns out to be more of a feature (or the other way around).
 Do not panic, the solution is very straightforward and quite simple.
 All you need to do are these two steps in arbitrary order:
-   - Ask someone from Core to change the base/target branch to the correct one
-   - Rebase or cherry-pick your commits onto the correct base branch...
+
+- Ask someone from Core to change the base/target branch to the correct one
+- Rebase or cherry-pick your commits onto the correct base branch...
 
 Let's show how to deal with the git...
 the sample case is moving a PR from `master` to `release/1.2-dev` assuming my branch name is `my-branch`
 and the last true master commit is `ccc111` and your first commit is `mmm222`.
-   * **Cherry-picking** way
-     ```bash
-     git checkout my-branch
-     # create a local backup of your branch
-     git checkout -b my-branch-backup
-     # reset your branch to the correct base
-     git reset release/1.2-dev --hard
-     # ACTION: this step is much easier to do with IDE
-     #  so open one and cherry-pick your last commits from `my-branch-backup`
-     #  resolve all eventual conflict as the new base may contain different code
-     # when all done, push back to the open PR
-     git push -f
-     ```
-   * **Rebasing way**, see more about [rebase onto usage](https://womanonrails.com/git-rebase-onto)
-     ```bash
-     git checkout my-branch
-     # rebase your commits on the correct branch
-     git rebase --onto release/1.2-dev ccc111
-     # if there is no collision you shall see just success
-     #  eventually you would need to resolve collision and in such case follow the instruction in terminal
-     # when all done, push back to the open PR
-     git push -f
-     ```
 
+- **Cherry-picking** way
+  ```bash
+  git checkout my-branch
+  # create a local backup of your branch
+  git checkout -b my-branch-backup
+  # reset your branch to the correct base
+  git reset release/1.2-dev --hard
+  # ACTION: this step is much easier to do with IDE
+  #  so open one and cherry-pick your last commits from `my-branch-backup`
+  #  resolve all eventual conflict as the new base may contain different code
+  # when all done, push back to the open PR
+  git push -f
+  ```
+- **Rebasing way**, see more about [rebase onto usage](https://womanonrails.com/git-rebase-onto)
+  ```bash
+  git checkout my-branch
+  # rebase your commits on the correct branch
+  git rebase --onto release/1.2-dev ccc111
+  # if there is no collision you shall see just success
+  #  eventually you would need to resolve collision and in such case follow the instruction in terminal
+  # when all done, push back to the open PR
+  git push -f
+  ```
 
 ### Bonus Workflow Tip