-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[feat] Support custom filesystems in LightningModule.to_torchscript #7617
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feat] Support custom filesystems in LightningModule.to_torchscript #7617
Conversation
Codecov Report
@@ Coverage Diff @@
## master #7617 +/- ##
=======================================
- Coverage 92% 88% -4%
=======================================
Files 197 198 +1
Lines 12871 12937 +66
=======================================
- Hits 11871 11375 -496
- Misses 1000 1562 +562 |
tests/models/test_torchscript.py
Outdated
| _DUMMY_PRFEIX = "dummy" | ||
| _PREFIX_SEPARATOR = "://" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these set as variables (instead of hardcoded) for a reason? Do you expect that other tests would need to modify them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think a dummy custom filesystem could be helpful to ensure fsspec works as expected, but this is not the right spot to land this. for now, i will move it into the test. i initially avoided putting this into the test to avoid registering the filesysem 3 times
tchaton
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM !
What does this PR do?
This code wasn't using
fsspecmeaning remote filepaths (azure, s3, gcs) weren't supported out of the box. This PR fixes that by using fsspec, similar to how we use this in other parts of the codebase..Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃