Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Change root directory for datasets #1740

Merged
merged 1 commit into from
May 25, 2022
Merged

Change root directory for datasets #1740

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion torchtext/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from torchtext import _extension # noqa: F401

_TEXT_BUCKET = "https://download.pytorch.org/models/text/"
_CACHE_DIR = os.path.expanduser("~/.torchtext/cache")
_CACHE_DIR = os.path.expanduser("~/.cache/torch/text")
Copy link
Contributor

@mthrok mthrok May 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~/.cache/torch is just a default path of TORCH_HOME, which is governed by PyTorch core.
If users set TORCH_HOME to different place, this torchtext cache directory won't be affected. There is a pros and cons on either behavior. If If domain libraries are piggy-backing on it, and the change is being made for the consistency, shouldn't that be accounted? Meaning that did you settle on how this should interact with TORCH_HOME?

Copy link
Contributor

@mthrok mthrok May 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://pytorch.org/docs/stable/hub.html#where-are-my-downloaded-models-saved

Copy link
Contributor Author

@parmeet parmeet May 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mthrok for the catch. I missed to take TORCH_HOME into account in this PR :(. Let me make a quick fix.


from . import data, datasets, experimental, functional, models, nn, transforms, utils, vocab

Expand Down
2 changes: 1 addition & 1 deletion torchtext/data/datasets_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def decorator(fn):

@functools.wraps(fn)
def wrapper(root=_CACHE_DIR, *args, **kwargs):
new_root = os.path.join(root, dataset_name)
new_root = os.path.join(root, "datasets", dataset_name)
if not os.path.exists(new_root):
os.makedirs(new_root, exist_ok=True)
return fn(root=new_root, *args, **kwargs)
Expand Down