Pid port + duplicate rank_zero logging

[williamFalcon]

Implements #2140 in a way that works.

For local ddp, we now use the process id as a numpy sub-seed to generate a random port. This way since the user sets the full seed we can use the parent process id as a seed before spawning off children.

Also solves an issue where on init when ranks are know, we shouldn't see duplicate messages

[williamFalcon]

@ZhaofengWu mind putting your Full name + email here for the commit?

[ZhaofengWu]

Zhaofeng Wu; [email protected] :)

[ZhaofengWu]

This comment should also be changed then
https://github.com/PyTorchLightning/pytorch-lightning/blob/bd49b07fbba09b1e7d8851ee5a1ffce3d5925e9e/pytorch_lightning/trainer/distrib_data_parallel.py#L426

[codecov]

Codecov Report

Merging #2231 into master will decrease coverage by 0%.
The diff coverage is 22%.

@@          Coverage Diff           @@
##           master   #2231   +/-   ##
======================================
- Coverage      88%     88%   -0%     
======================================
  Files          70      70           
  Lines        5415    5421    +6     
======================================
+ Hits         4759    4761    +2     
- Misses        656     660    +4     

[Borda]

@ZhaofengWu mind putting your Full name + email here for the commit?

unfortunately, this way of co-authoring is pure git thing and dos do not appear in any GitHub stats, see my comment here #1920 (comment)

[ExpectationMax]

I think this line hear breaks logging in my setup. rank_zero_only.rank is set to a non-zero value (the slurm job id) and thus non of the logging functions are ever executed. Did you mean SLURM_PROCID which is the MPI rank?

[ExpectationMax]

I think this pull request breaks logging in some setups (when running on slurm clusters). See https://github.com/PyTorchLightning/pytorch-lightning/pull/2231/files#r443131338

(sorry for the double post, I did not realize that the review also shows up in this thread)