torchaudio Installation issue

[bhargavkathivarapu]

🐛 Bug

Unable to install torchaudio from git clone . Getting below error .

 $python setup.py develop
DEBUG: False IS_WHEEL: False IS_CONDA: False
-- Building version 0.6.0a0+313f4f5
running develop
running egg_info
writing torchaudio.egg-info/PKG-INFO
writing dependency_links to torchaudio.egg-info/dependency_links.txt
writing requirements to torchaudio.egg-info/requires.txt
writing top-level names to torchaudio.egg-info/top_level.txt
reading manifest file 'torchaudio.egg-info/SOURCES.txt'
writing manifest file 'torchaudio.egg-info/SOURCES.txt'
running build_ext
building 'torchaudio._torchaudio' extension
gcc -pthread -B /miniconda/envs/python36/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/root/audio -I/miniconda/envs/python36/lib/python3.6/site-packages/torch/include -I/miniconda/envs/python36/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/miniconda/envs/python36/lib/python3.6/site-packages/torch/include/TH -I/miniconda/envs/python36/lib/python3.6/site-packages/torch/include/THC -I/miniconda/envs/python36/include/python3.6m -c torchaudio/csrc/sox.cpp -o build/temp.linux-x86_64-3.6/torchaudio/csrc/sox.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_torchaudio -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from torchaudio/csrc/sox.cpp:1:0:
/root/audio/torchaudio/csrc/sox.h:1:10: fatal error: sox.h: No such file or directory
 #include <sox.h>
          ^~~~~~~
compilation terminated.

To Reproduce

Steps to reproduce the behavior:

1. git clone https://github.com/pytorch/audio.git
2. python setup.py develop

Expected behavior

Torchaudio module should be installed

Environment

Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: Could not collect

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.17.2
[pip] torch==1.4.0
[pip] torchvision==0.5.0
[conda] blas 1.0 mkl
[conda] cudatoolkit 10.0.130 0
[conda] mkl 2019.4 243
[conda] mkl-service 2.3.0 py36he904b0f_0
[conda] mkl_fft 1.0.14 py36ha843d7b_0
[conda] mkl_random 1.1.0 py36hd6b4f25_0
[conda] numpy 1.17.2 py36haad9e8e_0
[conda] numpy-base 1.17.2 py36hde5b4d6_0
[conda] pytorch 1.4.0 py3.6_cuda10.0.130_cudnn7.6.3_0 pytorch
[conda] torchvision 0.5.0 py36_cu100 pytorch

Additional context

I tried above in 2 different environments , But it resulted in same " #include <sox.h> " issue .
But some how my already existing clone of torchaudio master in develop mode with pytorch also in develop mode in different env , is able to compile .

I think keeping include <sox.h> inside sox.h is not needed ? Am i correct ?

[mthrok]

Hi @bhargavkathivarapu

The sox.h referred in torch audio/csrc/sox.h is one from official SoX torchaudio depends on. I am on the same page when it comes to dependency resolution that it should be automatically handled and the current one is confusing for no benefit. I am resolving this in #625, but before this lands you can do the following on Linux-based environment

git clone ...
cd audio
./packaging/build_from_source.sh $PWD
IS_WHEEL=1 python setup.py develop

This way, the third-party packages (such as SoX and other codecs) are present in third_party and they are included properly.

[ark626]

` /tmp/torchaudio-deps/lame-3.99.5 /tmp/torchaudio-deps ~/MelGAN/audio

• ./configure --disable-shared --enable-static --prefix=/home/ark626/MelGAN/audio/third_party/lame CFLAGS=-fPIC CXXFLAGS=-fPIC --with-pic --disable-debug --disable-dependency-tracking --enable-nasm
  checking build system type... ./config.guess: unable to guess system type

This script, last modified 2003-07-02, has failed to recognize
the operating system you are using. It is advised that you
download the most up to date version of the config scripts from

ftp://ftp.gnu.org/pub/gnu/config/

If the version you run (./config.guess) is already up to date, please
send the following data and any information you think might be
pertinent to config-patches@gnu.org in order to provide the needed
information to handle your system.

config.guess timestamp = 2003-07-02

uname -m = aarch64
uname -r = 4.9.140-tegra
uname -s = Linux
uname -v = #1 SMP PREEMPT Wed Apr 8 18:15:20 PDT 2020

/usr/bin/uname -p =
/bin/uname -X =

hostinfo =
/bin/universe =
/usr/bin/arch -k =
/bin/arch =
/usr/bin/oslevel =
/usr/convex/getsysinfo =

UNAME_MACHINE = aarch64
UNAME_RELEASE = 4.9.140-tegra
UNAME_SYSTEM = Linux
UNAME_VERSION = #1 SMP PREEMPT Wed Apr 8 18:15:20 PDT 2020
`

Seems like i cannot install it that way

Happens when running ./packaging/build_from_source.sh $PWD

Currently running on the Jetson Xavier AGX, any help would be appreciatet for this since i cannot use Torchaudio otherwise.

Issue seems to be libmad and lame.

[mthrok]

Hi @ark626

Your issue is different from what this issue is tracking, so can you open a new issue?
The ./packaging/build_from_source.sh does not support your environment and unfortunately we do not have resource for actively expanding support for aarch64 at the moment. The best way to bring aarch64 support to torchaudio is external contribution.

The starting point is to install just SoX. If your environment has a package manager that installs SoX development package (libsox and sox.h), then you can start trying to build torchaudio on top of it by adding the package location to proper environment variables that your compile recognizes. Once it succeeded, you can try adding other codecs like mad or lame.

Also when you open an issue, please paste the result of the command python -m torch.utils.collect_env
And also note that, master branch of torchaudio expects nightly build of PyTorch.

[bhargavkathivarapu]

git clone ...
cd audio
./packaging/build_from_source.sh $PWD
IS_WHEEL=1 python setup.py develop

I am able to install torchaudio successfully after following the above steps .

[bhargavkathivarapu]

@mthrok , I think we should keep this issue open until the fix is merged into master , so that others can the find the temporary fix . Otherwise we can close it .

[mthrok]

@mthrok , I think we should keep this issue open until the fix is merged into master , so that others can the find the temporary fix . Otherwise we can close it .

Sounds good to me. We can keep this open.

[mthrok]

#625 is merged. Just python setup.py develop should work now.

[mthrok]

cc @bhargavkathivarapu I am reverting the behavior of codecs build since it caused issues elsewhere
#669
The new behavior to build SoX/codecs will be BUILD_SOX=1 python setup.py develop.

[cpuhrsch]

Are there docs we might need to update for this new flag?

[mthrok]

Are there docs we might need to update for this new flag?

README has been updated.