Improve SWA docs

[carmocca]

What does this PR do?

Part of #8654

Does your PR introduce any breaking changes? If yes, please list them.

None

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• [n/a] Did you write any new necessary tests? (not for typos and docs)
• [n/a] Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• [n/a] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

[codecov]

Codecov Report

Merging #8717 (d3eb981) into master (299e289) will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master   #8717   +/-   ##
======================================
  Coverage      93%     93%           
======================================
  Files         169     169           
  Lines       14073   14073           
======================================
  Hits        13042   13042           
  Misses       1031    1031           

[chanshing]

@carmocca Thank you for improving the doc!

Could you also clarify what are the defaults when using Trainer(stochastic_weight_avg=True)? Are they those of StochasticWeightAveraging? And in that case, what is the default value for swa_lrs?

If I understood the code correctly, it seems to be the last learning rate(s) used -- is this correct?
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/callbacks/stochastic_weight_avg.py#L162-L189

[carmocca]

Could you also clarify what are the defaults when using Trainer(stochastic_weight_avg=True)? Are they those of StochasticWeightAveraging?

Yes

If I understood the code correctly, it seems to be the last learning rate(s) used -- is this correct?

The current learning rate for the optimizer at the time the SWA procedure starts.

I'll update the docs with these