Skip to content

Commit 6c01608

Browse files
ozencarmocca
andauthored
Updated iterable dataset with len warning message (#6972)
Co-authored-by: Carlos Mocholi <[email protected]>
1 parent 1b3e4f9 commit 6c01608

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

pytorch_lightning/utilities/data.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,9 @@ def has_len(dataloader: DataLoader) -> bool:
4040
if has_len and has_iterable_dataset(dataloader):
4141
rank_zero_warn(
4242
'Your `IterableDataset` has `__len__` defined.'
43-
' In combination with multi-processing data loading (e.g. batch size > 1),'
44-
' this can lead to unintended side effects since the samples will be duplicated.'
43+
' In combination with multi-process data loading (when num_workers > 1),'
44+
' `__len__` could be inaccurate if each worker is not configured independently'
45+
' to avoid having duplicate data.'
4546
)
4647
return has_len
4748

0 commit comments

Comments
 (0)