Weekly Patch Release v1.5.8

[awaelchli]

What does this PR do?

Contains all bugfixes to be released next week. As bugfixes get merged to master, they will get cherry-picked into this branch by the individual author (core) or on-caller.

For staff/core: The new release guide is here.

gh pr list -s merged -S 'merged:2021-12-21T18:30:00.000Z..2022-01-06T18:30:00.000Z' --json mergedAt,milestone,url,mergeCommit,title --jq 'sort_by(.mergedAt) | reverse | .[] | select((.milestone.title == 
                                                              "1.5.x") or (.milestone.title == null)) | [.mergedAt, .url, .mergeCommit.oid, .title] | join(" ")' --limit 100

2022-01-03T22:17:51Z https://github.com/PyTorchLightning/pytorch-lightning/pull/11288 cf32127e7ed75da2308e4841dc816d0379ff3fd0 Avoid non-blocking GPU->CPU copies.
2022-01-03T12:49:17Z https://github.com/PyTorchLightning/pytorch-lightning/pull/11294 17cb3c70f7893f318ed9344af8c8f9c4e5d128da Fix data fetcher selection
2022-01-03T11:45:42Z https://github.com/PyTorchLightning/pytorch-lightning/pull/10580 7637550ab513cf2880d6b4a1b18fc006172800bc Revert "[CI] Comment flaky tests (#10084)"
2021-12-28T01:20:57Z https://github.com/PyTorchLightning/pytorch-lightning/pull/11036 1fc046cde28684316323693dfca227dbd270663f Fix `_should_reload_dl_epoch` causing inconsistent validation dataloader reloading
2021-12-23T16:45:07Z https://github.com/PyTorchLightning/pytorch-lightning/pull/11199 f44b209e7273a229af9979b9b391f66114b89cad Fix CLI race condition saving the config
2021-12-22T16:05:44Z https://github.com/PyTorchLightning/pytorch-lightning/pull/11223 52f308eb89bcd517d1d8ac5b1c10b7076b1ea4a5 include Lezwon in alumni
2021-12-22T03:47:14Z https://github.com/PyTorchLightning/pytorch-lightning/pull/10990 ba8e7cd787815d8bd3a0d0c5b859dcde301c6ccd Fix BF16 teardown for TPU precision plugin

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

Part #1

cc @Borda

[tchaton]

LGTM !

[awaelchli]

I will drop #10990 to prevent delaying the release further. The TPU tests are not passing and the strategy/accelerator code has diverged a lot from master. I will try to include it in the next release.