nvidia dali support

[smallzzy]

🚀 Feature

Re-open the case for #513. i.e. Support nvidia dali iterator as a possible data loader.

Pitch

I have submitted a wip pull request to the master. #789
I would like to know if it is ok for the test to be depending on nvidia-dali in addition to apex.

[smallzzy]

The dali iterator does not support resetting while epoch is not finished. I suppose I shall provide a warning for that.

[williamFalcon]

@smallzzy this is sick. super excited by this feature

[Borda]

@smallzzy we have cleared the API and now it should be stable... Mind resume your addition?

[brunoalano]

Some news about DALI support?

[Borda]

@brunoalano interested in implementing DALI?

[brunoalano]

@Borda Do you have a pipeline what should have be done to support it? But I'm available to do that with a minimal guidance (started using PyTorch Lightning and DALI these days)