InnerEyeDDPPlugin - unknown issue comment

[awaelchli]

Hi

Was snooping around in the codebase and found you have a custom DDPPlugin with a comment at the top that there is apparently a bug you have discovered. It is unclear what exactly is causing issues, could you let me know and I will try to help. I see no substantial differences in this custom plugin compared to upstream, except a few lines of hydra specifc code missing. Is it possible the original problem you encountered is already solved in Lighting 1.3+?

AB#4155

[ant0nsc]

Hey @awaelchli, thanks for checking! Looking that latest code, it appears indeed that the problem is solved in 1.3+. FYI, the original issue is described in #437 , it broke multi-node training. We'll check if we can update easily.

[ant0nsc]

done