Rewrite our packaging from scratch #217
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 0e64ab6.
This reverts commit 5ad6d2e.
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
soumith
approved these changes
Aug 7, 2019
| setup_python() { | ||
| if [[ "$(uname)" == Darwin ]]; then | ||
| eval "$(conda shell.bash hook)" | ||
| conda create -yn "env$PYTHON_VERSION" python="$PYTHON_VERSION" |
Member
There was a problem hiding this comment.
what if such an env is already created, and potentially corrupted?
Contributor
Author
There was a problem hiding this comment.
conda env remove -n "env$PYTHON_VERSION" || true
:)
| build: | ||
| number: {{ environ.get('TORCHAUDIO_BUILD_NUMBER') }} | ||
| string: py{{py}}_{{environ.get('TORCHAUDIO_BUILD_NUMBER')}} | ||
| string: py{{py}} |
Member
There was a problem hiding this comment.
you dont have a BUILD_NUMBER, are you sure you will never botch a binary build ever?
Contributor
Author
There was a problem hiding this comment.
My thinking is, if I botch build of 1.2.0, then I will next build 1.2.0.post2 (because I don't get build number in pip wheels). Now the version has changed and I don't need it. But I could be wrong!
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
ezyang
added a commit
to pytorch/vision
that referenced
this pull request
Aug 7, 2019
Following a similar line of inquiry to pytorch/audio#217
ezyang
added a commit
to pytorch/vision
that referenced
this pull request
Aug 7, 2019
Following a similar line of inquiry to pytorch/audio#217
ezyang
added a commit
that referenced
this pull request
Aug 8, 2019
* New entry points are packaging/build_wheel.sh and packaging/build_conda.sh. The only mandatory environment variable you have to set is PYTHON_VERSION * CircleCI configuration uses 2.1-style parametrized builds to let you toggle python version, etc. as you do builds. We create a separate job per build configuration for maximum parallelism * build_tools/packaging got moved to packaging, to be in-line with directory structure in torchvision * The build_conda.sh and build_wheel.sh delegate most of the heavy lifting to pkg_helpers.bash, which defines a number of bash functions for performing common operations. The intent is that I'll copy-paste this file between other domain API projects. * TORCHAUDIO_ prefix removed from envvars, so that I can more easily share packaging scripts between projects * BUILD_VERSION is completely gone; just change the version number if you need to rebuild * No more logic for cloning and checking out a fresh copy of torchaudio
ezyang
added a commit
to pytorch/vision
that referenced
this pull request
Aug 8, 2019
* Rewrite torchvision packaging (#1209) Following a similar line of inquiry to pytorch/audio#217 * Packaging fixes (#1214) Add uploading support, make CUDA builds actually work. * 0.4.0 parameters Signed-off-by: Edward Z. Yang <[email protected]> * Actually upload wheels (please port to master) Signed-off-by: Edward Z. Yang <[email protected]> * Put macos binaries in the right place Signed-off-by: Edward Z. Yang <[email protected]> * Propagate more environment variables. Signed-off-by: Edward Z. Yang <[email protected]> * Change the version number Signed-off-by: Edward Z. Yang <[email protected]> * Go time Signed-off-by: Edward Z. Yang <[email protected]>
ezyang
added a commit
that referenced
this pull request
Aug 8, 2019
* Rewrite our packaging from scratch (#217) * New entry points are packaging/build_wheel.sh and packaging/build_conda.sh. The only mandatory environment variable you have to set is PYTHON_VERSION * CircleCI configuration uses 2.1-style parametrized builds to let you toggle python version, etc. as you do builds. We create a separate job per build configuration for maximum parallelism * build_tools/packaging got moved to packaging, to be in-line with directory structure in torchvision * The build_conda.sh and build_wheel.sh delegate most of the heavy lifting to pkg_helpers.bash, which defines a number of bash functions for performing common operations. The intent is that I'll copy-paste this file between other domain API projects. * TORCHAUDIO_ prefix removed from envvars, so that I can more easily share packaging scripts between projects * BUILD_VERSION is completely gone; just change the version number if you need to rebuild * No more logic for cloning and checking out a fresh copy of torchaudio * Setup cuda_suffix (#218) Signed-off-by: Edward Z. Yang <[email protected]> * Add uploading support (#220) Also bugfixes and updates to the packaging scripts. * Finish up and set parameters Signed-off-by: Edward Z. Yang <[email protected]> * Updates from vision Signed-off-by: Edward Z. Yang <[email protected]> * Go time Signed-off-by: Edward Z. Yang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Structure of the PR:
packaging/build_wheel.shandpackaging/build_conda.sh. The only mandatory environment variable you have to set isPYTHON_VERSIONbuild_tools/packaginggot moved topackaging, to be in-line with directory structure in torchvisionbuild_conda.shandbuild_wheel.shdelegate most of the heavy lifting topkg_helpers.bash, which defines a number of bash functions for performing common operations. The intent is that I'll copy-paste this file between other domain API projects.TORCHAUDIO_prefix removed from envvars, so that I can more easily share packaging scripts between projectsBUILD_VERSIONis completely gone; just change the version number if you need to rebuild