-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Description
Proposed refactoring or deprecation
init_ddp_connection is not only used by DDP, but also used by FSDP and other plugins inherit DDP. The log info and rank_zero_info saying ""initializing ddp" or "DDP processes", which could be confusing for user
Motivation
Debuggability and readability
Pitch
Update info message to be generic message
Update function name to initialize_distributed_connction as it's used by fsdp and sdp
Additional context
If you enjoy Lightning, check out our other projects! ⚡
-
Metrics: Machine learning metrics for distributed, scalable PyTorch applications.
-
Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning
-
Bolts: Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch
-
Lightning Transformers: Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.