Skip to content

enable ffcv with PL #11538

@williamFalcon

Description

@williamFalcon

A team introduced ffcv which does optimizations in the dataloader level.
As long as it can be a drop in replacement for dataloader it should add the benefits to any PyTorch and Lightning script.

ie:

# current
data = Dataloader(...)

trainer.fit(..., data)

# 10x faster with ffcv
data = FFCVDataloader

trainer.fit(..., data)

Let's make sure when users want to get the benefits of both FFCV and PL they can do that!
cc @Borda @rohitgr7 @Felonious-Spellfire @justusschock @awaelchli @ninginthecloud @otaj @tchaton @carmocca

Metadata

Metadata

Assignees

Labels

3rd partyRelated to a 3rd-partydata handlingGeneric data-related topicdocsDocumentation related

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions