Add before_batch_transfer and after_batch_transfer hooks

[rohitgr7]

What does this PR do?

Fixes #3399
Fixes #3087
Fixes #3461

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[mergify]

This pull request is now in conflict... :(

[edenlightning]

Hi @rohitgr7! Mind rebasing your PR?

[rohitgr7]

Hi @rohitgr7! Mind rebasing your PR?

will complete this PR this week probably.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need further help see our docs: https://pytorch-lightning.readthedocs.io/en/latest/CONTRIBUTING.html#pull-request or ask the assistance of a core contributor here or on Slack. Thank you for your contributions.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need further help see our docs: https://pytorch-lightning.readthedocs.io/en/latest/CONTRIBUTING.html#pull-request or ask the assistance of a core contributor here or on Slack. Thank you for your contributions.

[Borda]

LGTM, is it ready to review (regarding the very last reverting actions needed)

[rohitgr7]

LGTM, is it ready to review (regarding the very last reverting actions needed)

not yet @Borda.. little busy this week. Will try to complete this ASAP.

[codecov]

Codecov Report

Merging #3671 (6ba5dc3) into master (5157ba5) will decrease coverage by 2%.
The diff coverage is 85%.

@@           Coverage Diff           @@
##           master   #3671    +/-   ##
=======================================
- Coverage      93%     91%    -2%     
=======================================
  Files         160     160            
  Lines       11340   11341     +1     
=======================================
- Hits        10550   10270   -280     
- Misses        790    1071   +281     

[leifdenby]

Thanks for adding these hooks! I would like to use these to apply transforms across my entire batch when training. How would I find out if a batch is part of the training, validation or test set? I want to make sure I'm only applying augmentations during training. I had a look to see I could get that from a state variable within the trainer, but I don't think I can access that within methods called by these hooks. Thank you again, it's really a joy porting my code to pytorch-lightning

[SeanNaren]

Thanks for adding these hooks! I would like to use these to apply transforms across my entire batch when training. How would I find out if a batch is part of the training, validation or test set? I want to make sure I'm only applying augmentations during training. I had a look to see I could get that from a state variable within the trainer, but I don't think I can access that within methods called by these hooks. Thank you again, it's really a joy porting my code to pytorch-lightning

Hey @leifdenby! Do you mind making a new question issue or asking the question in the PyTorch Lightning Slack channel? primarily I don't want this getting lost as the issue is closed already :)

[rohitgr7]

hey @leifdenby you can use self.trainer.training/validating/testing (bool) for this purpose. In case you are using .tune or .predict you can use self.trainer.tuning/predicting respectively.

def transfer_batch_to_device(self, batch, ...):
    if self.trainer.testing:
        return test_transforms(batch)
    if self.trainer.training:
        return train_transforms(batch)
   ...

will add this to docs for convenience.