Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Make TorchText installable from direct reference #1154

Closed
wants to merge 1 commit into from
Closed

Make TorchText installable from direct reference #1154

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 23 additions & 5 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
#!/usr/bin/env python
from build_tools import setup_helpers
import distutils.command.clean
import io
import os
import shutil
import subprocess
from pathlib import Path
import distutils.command.clean
from setuptools import setup, find_packages

from build_tools import setup_helpers
import shutil
import subprocess
import sys
import time

ROOT_DIR = Path(__file__).parent.resolve()

Expand Down Expand Up @@ -44,6 +45,22 @@ def _export_version(version, sha):
fileobj.write("git_version = {}\n".format(repr(sha)))


def check_submodules():
third_party_dir = ROOT_DIR / 'third_party'
if not (third_party_dir / 're2' / 'CMakeFiles.txt').exists():
print(' --- third_party/re2/CMakeFiles.txt do not exist, trying to init submodules')
try:
Copy link
Contributor

@cpuhrsch cpuhrsch Feb 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyTorch fails and asks the user to do it manually. I worry that this might cause issues when someone wants to debug / modify the submodules and then subsequently runs setup.py. Are there similar plans for PyTorch as well?

Copy link
Contributor Author

@malfet malfet Feb 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpuhrsch yes, I plan to make similar change for pytorch as well, although Pytorch is less likely to be built directly from source (building TorchText using direct link takes less than 15 min, whereas for PyTorch, especially with GPU support will likely to take more than an hour)

Copy link
Contributor

@mthrok mthrok Feb 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the context of this work? Are we going to do the same for torchaudio (which I support)?

start = time.time()
subprocess.check_call(['git', 'submodule', 'init'])
subprocess.check_call(['git', 'submodule', 'update'])
end = time.time()
print(f' --- Submodule initialization took {end-start:.2f} sec')
except Exception:
print(' --- Submodule initalization failed')
print('Please run:\n\tgit submodule update --init --recursive')
sys.exit(1)


VERSION, SHA = _get_version()
_export_version(VERSION, SHA)

Expand Down Expand Up @@ -112,4 +129,5 @@ def run(self):
},
)

check_submodules()
setup(**setup_info)