From f15d5a1a32e63bc2af6779137f50d3029d71d2e7 Mon Sep 17 00:00:00 2001 From: Yao-Yuan Yang Date: Thu, 17 Jun 2021 17:23:56 -0700 Subject: [PATCH 1/4] Update document for unittest and CONTRIBUTING.md --- CONTRIBUTING.md | 13 +++++++++++-- test/torchaudio_unittest/README.md | 5 ++++- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 94526661e0..af124995f4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -55,9 +55,12 @@ conda install pytorch -c pytorch-nightly pip install cmake ninja ``` +### Install other requirements + ```bash git clone https://github.com/pytorch/audio.git cd audio +git submodule sync git submodule update --init --recursive BUILD_SOX=1 python setup.py develop # or, for OSX @@ -66,7 +69,7 @@ BUILD_SOX=1 python setup.py develop # DEBUG=1 python setup.py develop ``` -If you built sox, set the `PATH` variable so that the tests properly use the newly built `sox` binary: +If you built sox, set the `PATH` variable so that the tests properly use the newly built `SoX` binary: ```bash export PATH="/third_party/install/bin:${PATH}" @@ -75,7 +78,13 @@ export PATH="/third_party/install/bin:${PATH}" The following dependencies are also needed for testing: ```bash -pip install typing pytest scipy numpy parameterized +pip install typing pytest scipy numpy parameterized pyyaml +``` + +Optional packages to install if you want to run related tests: + +```bash +pip install librosa requests soundfile kaldi_io transformers fariseq ``` ## Development Process diff --git a/test/torchaudio_unittest/README.md b/test/torchaudio_unittest/README.md index 69f8498f05..98dd4f6016 100644 --- a/test/torchaudio_unittest/README.md +++ b/test/torchaudio_unittest/README.md @@ -9,6 +9,7 @@ For testing, the following packages are needed: ```bash pip install typing pytest scipy numpy parameterized +pip install -r requirements.txt ``` Make sure to follow the installation instruction in the [contributing @@ -73,7 +74,9 @@ The following test modules are defined for corresponding `torchaudio` module/fun - [`torchaudio.compliance.kaldi`](./compliance_kaldi_test.py) - [`torchaudio.kaldi_io`](./kaldi_io_test.py) - [`torchaudio.sox_effects`](./sox_effect) -- [`torchaudio.save`, `torchaudio.load`, `torchaudio.info`](./io_test.py) +- [`torchaudio.save`, `torchaudio.load`, `torchaudio.info`] + - soundfile backend: [[save](./backend/soundfile/save_test.py), [load](./backend/soundfile/load_test.py), [info](./backend/soundfile/info_test.py)] + - SoX backend: [[save](./backend/sox_io/save_test.py), [load](./backend/sox_io/load_test.py), [info](./backend/sox_io/info_test.py)] ### Test modules that do not fall into the above categories - [test_dataloader.py](./dataloader_test.py) From e2eeb8c8ec3e6fc942587e032f3485025fc7e056 Mon Sep 17 00:00:00 2001 From: Yao-Yuan Yang Date: Fri, 18 Jun 2021 15:12:33 -0700 Subject: [PATCH 2/4] Direct people to CONTRIBUTION.md for installing related package in the unittest README.md --- CONTRIBUTING.md | 8 +++----- test/torchaudio_unittest/README.md | 15 ++++----------- 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index af124995f4..987f91087b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -55,8 +55,6 @@ conda install pytorch -c pytorch-nightly pip install cmake ninja ``` -### Install other requirements - ```bash git clone https://github.com/pytorch/audio.git cd audio @@ -69,7 +67,7 @@ BUILD_SOX=1 python setup.py develop # DEBUG=1 python setup.py develop ``` -If you built sox, set the `PATH` variable so that the tests properly use the newly built `SoX` binary: +If you built sox, set the `PATH` variable so that the tests properly use the newly built `sox` binary: ```bash export PATH="/third_party/install/bin:${PATH}" @@ -78,13 +76,13 @@ export PATH="/third_party/install/bin:${PATH}" The following dependencies are also needed for testing: ```bash -pip install typing pytest scipy numpy parameterized pyyaml +pip install typing pytest scipy numpy parameterized ``` Optional packages to install if you want to run related tests: ```bash -pip install librosa requests soundfile kaldi_io transformers fariseq +pip install librosa requests soundfile kaldi_io transformers ``` ## Development Process diff --git a/test/torchaudio_unittest/README.md b/test/torchaudio_unittest/README.md index 98dd4f6016..dd6249fb4a 100644 --- a/test/torchaudio_unittest/README.md +++ b/test/torchaudio_unittest/README.md @@ -5,15 +5,10 @@ You can use `pytest` to run `torchaudio`'s test suites. See https://docs.pytest.org/ for the detail of how to use `pytest` command. -For testing, the following packages are needed: +For testing, please refer to [contributing guide](../../CONTRIBUTING.md) for +the installation of the required and optional packages. -```bash -pip install typing pytest scipy numpy parameterized -pip install -r requirements.txt -``` - -Make sure to follow the installation instruction in the [contributing -guide](../../CONTRIBUTING.md) first. For running `kaldi`-related tests: +For running `kaldi`-related tests: ```bash export PATH="${PATH}:/src/featbin/" @@ -74,9 +69,7 @@ The following test modules are defined for corresponding `torchaudio` module/fun - [`torchaudio.compliance.kaldi`](./compliance_kaldi_test.py) - [`torchaudio.kaldi_io`](./kaldi_io_test.py) - [`torchaudio.sox_effects`](./sox_effect) -- [`torchaudio.save`, `torchaudio.load`, `torchaudio.info`] - - soundfile backend: [[save](./backend/soundfile/save_test.py), [load](./backend/soundfile/load_test.py), [info](./backend/soundfile/info_test.py)] - - SoX backend: [[save](./backend/sox_io/save_test.py), [load](./backend/sox_io/load_test.py), [info](./backend/sox_io/info_test.py)] +- [`torchaudio.backend`](./backend) ### Test modules that do not fall into the above categories - [test_dataloader.py](./dataloader_test.py) From b84f48cb039b9ff2fa468e16097189c7094a8cfa Mon Sep 17 00:00:00 2001 From: Yao-Yuan Yang Date: Mon, 21 Jun 2021 13:25:23 -0700 Subject: [PATCH 3/4] remove 'git submodule sync' from CONTRIBUTING.md --- CONTRIBUTING.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 987f91087b..6f77aa58e2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -58,7 +58,6 @@ pip install cmake ninja ```bash git clone https://github.com/pytorch/audio.git cd audio -git submodule sync git submodule update --init --recursive BUILD_SOX=1 python setup.py develop # or, for OSX From f6bdca724f2ae91b22f7482179585a1250edfe31 Mon Sep 17 00:00:00 2001 From: Yao-Yuan Yang Date: Mon, 21 Jun 2021 14:55:59 -0700 Subject: [PATCH 4/4] list optional packages instead of a pip install command --- CONTRIBUTING.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6f77aa58e2..67ea6caed2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -80,9 +80,13 @@ pip install typing pytest scipy numpy parameterized Optional packages to install if you want to run related tests: -```bash -pip install librosa requests soundfile kaldi_io transformers -``` +- `librosa` +- `requests` +- `soundfile` +- `kaldi_io` +- `transformers` +- `fairseq` (it has to be newer than `0.10.2`, so you will need to install from + source. Commit `e6eddd80` is known to work.) ## Development Process