Skip to content

Conversation

@salilsdesai
Copy link
Contributor

@salilsdesai salilsdesai commented Jun 7, 2021

In response to #1552

  • Update the skipIfNoCuda decorator to raise an error if the TORCHAUDIO_TEST_FORCE_CUDA variable is set but CUDA is not available
  • Update CI to use TORCHAUDIO_TEST_FORCE_CUDA in gpu unittests

@mthrok mthrok marked this pull request as ready for review June 7, 2021 14:25
@salilsdesai salilsdesai marked this pull request as draft June 7, 2021 14:26
@mthrok
Copy link
Contributor

mthrok commented Jun 7, 2021

Hi @salilsdesai

Thanks for working on this. Instead of changing config.yml directly, you need to apply the change to config.yml.in and use regenerate.py to update config.yml. We do this because we have to define the same CI jobs to different Python / CUDA version.

- run:
name: Run tests
command: docker run -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" .circleci/unittest/linux/scripts/run_test.sh
command: docker run -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" -e "TORCHAUDIO_TEST_FORCE_CUDA=1" .circleci/unittest/linux/scripts/run_test.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-e has to be before "${image_name}"

@mthrok
Copy link
Contributor

mthrok commented Jun 8, 2021

Looks like our CI does have GPU but the firmware is too old. I see a warning

  c:\users\circleci\project\env\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 10010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:115.)

https://app.circleci.com/pipelines/github/pytorch/audio/6271/workflows/66eeb5f8-1841-4b97-a0b2-9c7a36adf4a5/jobs/232928/steps

Let me see what's the alternative way to install PyTorch.

@mthrok
Copy link
Contributor

mthrok commented Jun 9, 2021

@salilsdesai Can you leave the Windows as-is for now? The Windows CI situation is somewhat complicated to resolve, so we will tackle later. The you are adding is variable so let's merge it first.

@salilsdesai salilsdesai marked this pull request as ready for review June 9, 2021 14:00
@salilsdesai
Copy link
Contributor Author

@salilsdesai Can you leave the Windows as-is for now? The Windows CI situation is somewhat complicated to resolve, so we will tackle later. The you are adding is variable so let's merge it first.

Oh, you mean I should revert the windows gpu test, right? I'll do that now then.

@mthrok
Copy link
Contributor

mthrok commented Jun 9, 2021

@salilsdesai Can you leave the Windows as-is for now? The Windows CI situation is somewhat complicated to resolve, so we will tackle later. The you are adding is variable so let's merge it first.

Oh, you mean I should revert the windows gpu test, right? I'll do that now then.

Yes, exactly. Thanks for updating it.

@mthrok mthrok merged commit e9415df into pytorch:master Jun 9, 2021
mthrok pushed a commit to mthrok/audio that referenced this pull request Dec 13, 2022
* Recipe for skipping parameter init

Co-authored-by: Joel Benjamin Schlosser <[email protected]>
Co-authored-by: Brian Johnson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants