Skip to content

Conversation

@ninginthecloud
Copy link
Contributor

@ninginthecloud ninginthecloud commented Aug 17, 2021

What does this PR do?

Fixes #8733
This PR has completed the following:

  1. add DataHooks with prepare_data_per_node property
  2. set prepare_data_per_node in trainer as optional
  3. update on_trainer_init function in data_connector class
  4. update tests: test_datamodules.py and test_hooks.py
  5. update prepare_data() implementation and remove can_prepare_data() in data_connector class
  6. user can handle prepare_data in lightningmodule and lightningdatamodule independently.

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Aug 17, 2021

Codecov Report

Merging #8958 (702cdc8) into master (8a93173) will decrease coverage by 4%.
The diff coverage is 96%.

@@           Coverage Diff           @@
##           master   #8958    +/-   ##
=======================================
- Coverage      93%     89%    -4%     
=======================================
  Files         175     175            
  Lines       14379   14391    +12     
=======================================
- Hits        13322   12749   -573     
- Misses       1057    1642   +585     

@ninginthecloud
Copy link
Contributor Author

This PR needs rebase, I will update soon.

@tchaton
Copy link
Contributor

tchaton commented Aug 17, 2021

Dear @ninginthecloud,

Let's use human readable title for PR :)

Best,
T.C

@ninginthecloud ninginthecloud changed the title Refactor issue8733 [Refactor] Move prepare_data_per_node to the DataHooks class Aug 17, 2021
@ninginthecloud ninginthecloud changed the title [Refactor] Move prepare_data_per_node to the DataHooks class Deprecate prepare_data_per_node flag on Trainer and set it as a property for DataHooks Aug 18, 2021
Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@ananthsub ananthsub added the data handling Generic data-related topic label Aug 18, 2021
@ananthsub ananthsub added this to the v1.5 milestone Aug 18, 2021
ninginthecloud and others added 25 commits August 22, 2021 21:08
Co-authored-by: Adrian Wälchli <[email protected]>
@carmocca carmocca enabled auto-merge (squash) August 23, 2021 12:33
@carmocca carmocca merged commit 2481816 into Lightning-AI:master Aug 23, 2021
@ninginthecloud ninginthecloud deleted the refactor_issue8733 branch August 23, 2021 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data handling Generic data-related topic deprecation Includes a deprecation ready PRs ready to be merged refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move prepare_data_per_node to the DataHooks class

7 participants