Skip to content

[CLI] Can not instantiate DDPStrategy using lazy_instance() with find_unused_parameters=False #12697

@semaphore-egg

Description

@semaphore-egg

🐛 Bug

Instantiate DDPStrategy with find_unused_parameters=False raise exceptions when using lazy_instance().
The non-lazy mode and no-parameter mode work as expected.

To Reproduce

import os

import torch
from torch.utils.data import DataLoader, Dataset

import pytorch_lightning as pl
from pytorch_lightning import LightningModule, Trainer
from pytorch_lightning.utilities import cli
from pytorch_lightning.strategies import DDPStrategy
from jsonargparse import lazy_instance


class RandomDataset(Dataset):
    def __init__(self, size, length):
        self.len = length
        self.data = torch.randn(length, size)

    def __getitem__(self, index):
        return self.data[index]

    def __len__(self):
        return self.len


class RandomDataModule(pl.LightningDataModule):
    def train_dataloader(self):
        return DataLoader(RandomDataset(32, 64), batch_size=2)


class BoringModel(LightningModule):
    def __init__(self):
        super().__init__()
        self.layer = torch.nn.Linear(32, 2)

    def forward(self, x):
        return self.layer(x)

    def training_step(self, batch, batch_idx):
        loss = self(batch).sum()
        self.log("train_loss", loss)
        return {"loss": loss}

    def configure_optimizers(self):
        return torch.optim.SGD(self.layer.parameters(), lr=0.1)


if __name__ == "__main__":
    c = cli.LightningCLI(
        BoringModel,
        RandomDataModule,
        trainer_defaults={
            # Case 1:
            # ValueError: 'Configuration check failed :: No action for
            # destination key "find_unused_parameters" to check its value.'
            'strategy': lazy_instance(DDPStrategy, find_unused_parameters=False),

            # Case 2: (lazy mode without find_unused_parameters)  work as expected
            'strategy': lazy_instance(DDPStrategy),

            # Case 3: (non-lazy mode) work as expected
            'strategy': DDPStrategy(find_unused_parameters=False),
        }
    )

Expected behavior

All three cases are supposed to work.

Environment

  • CUDA:
    • GPU:
      • NVIDIA GeForce RTX 3090
    • available: True
    • version: 11.3
  • Packages:
    • numpy: 1.22.3
    • pyTorch_debug: False
    • pyTorch_version: 1.11.0+cu113
    • pytorch-lightning: 1.6.0
    • tqdm: 4.64.0
  • System:
    • OS: Linux
    • architecture:
      • 64bit
      • ELF
    • processor: x86_64
    • python: 3.9.11
    • version: Extend CI #44~20.04.1-Ubuntu SMP Thu Mar 24 16:43:35 UTC 2022

Additional context

cc @carmocca @mauvilsa

Metadata

Metadata

Assignees

No one assigned

    Labels

    3rd partyRelated to a 3rd-partybugSomething isn't workinglightningclipl.cli.LightningCLI

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions