Add smoke test for binary package #442

guyang3532 · 2020-05-11T03:18:38Z

No description provided.

peterjc123

Generally lgtm, just some nits.

peterjc123 · 2020-05-12T03:37:24Z

windows/internal/smoke_test.bat

+set LIB=%LIB%;%install_root%/lib
+set PATH=%PATH%;%install_root%/lib
+
+>example-app.cpp (


nit: Consider creating a new file?

And the name of the executable is not clear.

peterjc123 · 2020-05-12T03:38:59Z

windows/internal/smoke_test.bat

+.\example-app.exe
+if ERRORLEVEL 1 exit /b 1
+
+echo Checking that MKL is available


peterjc123 · 2020-05-12T03:39:22Z

windows/internal/smoke_test.bat

+
+del example-app.cpp
+>example-app.cpp (
+echo #include ^<torch/torch.h^>


peterjc123 · 2020-05-12T03:41:42Z

windows/internal/smoke_test.bat

+conda create -qyn testenv python=%DESIRED_PYTHON%
+source activate testenv >/dev/null
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *.tar.bz2') do conda install -y "%%i" --offline
+conda install -yq future numpy protobuf six


nit: Check the exitcode here?

peterjc123 · 2020-05-12T03:41:48Z

windows/internal/smoke_test.bat

+set CUDA_VER_MAJOR=%CUDA_VERSION:~0,-1%
+set CUDA_VER_MINOR=%CUDA_VERSION:~-1,1%
+set CUDA_VERSION_STR=%CUDA_VER_MAJOR%.%CUDA_VER_MINOR%
+conda install -yq -c pytorch "cudatoolkit=%CUDA_VERSION_STR%"


nit: Check the exitcode here?

peterjc123 · 2020-05-12T03:44:26Z

windows/internal/smoke_test.bat

+powershell internal/vs_install.ps1
+if ERRORLEVEL 1 exit /b 1
+
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *-latest.zip') do unzip "%%i" -d tmp


nit: Consider using 7z instead. unzip is only available in Bash. The equivalent line using 7z is 7z x "%%i" -otmp.

peterjc123 · 2020-05-12T03:46:34Z

windows/internal/smoke_test.bat

+set "PATH=%CD%\Python%PYTHON_VERSION%\Scripts;%CD%\Python%PYTHON_VERSION%;%PATH%"
+
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *.whl') do pip install "%%i"
+pip install -q future numpy protobuf six "mkl>=2019"


nit: Check exitcode here？

peterjc123 · 2020-05-12T03:48:27Z

windows/internal/smoke_test.bat

+set "PATH=%CONDA_HOME%;%CONDA_HOME%\scripts;%CONDA_HOME%\Library\bin;%PATH%"
+
+conda create -qyn testenv python=%DESIRED_PYTHON%
+source activate testenv >/dev/null


Again this is Bash only, please use call %CONDA_HOME%\condabin\activate.bat testenv > NUL.

peterjc123 · 2020-05-15T05:47:12Z

windows/internal/update_driver.bat

@@ -0,0 +1,4 @@
+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe


nit:

check error code

add --retry option

peterjc123 · 2020-05-15T05:47:27Z

windows/internal/update_driver.bat

+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
+
+start /wait 442.50-tesla-desktop-winserver-2019-2016-international.exe -s -noreboot


nit: check exit code

peterjc123 · 2020-05-15T05:50:11Z

windows/internal/update_driver.bat

@@ -0,0 +1,4 @@
+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe


Suggested change

curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

curl --retry 3 -kL "%DRIVER_DOWNLOAD_LINK%" --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

if errorlevel 1 exit /b 1

peterjc123 · 2020-05-16T15:12:28Z

check_binary.sh

@@ -256,8 +256,8 @@ build_and_run_example_cpp () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
-  ./example-app
+  g++ ${BUILDER_ROOT}/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


Why is $1 used here?

Why is $1 used here?
The code file name is passed as argument from outside.

peterjc123 · 2020-05-16T15:13:37Z

windows/internal/driver_update.bat

+if errorlevel 1 exit /b 1
+
+del 442.50-tesla-desktop-winserver-2019-2016-international.exe || ver > NUL
+


nit: empty line?

peterjc123 · 2020-05-16T15:15:01Z

Could you please take some time to review this PR, @seemethere?

peterjc123 · 2020-05-18T03:10:14Z

check_binary.sh

@@ -256,8 +256,8 @@ build_and_run_example_cpp () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
-  ./example-app
+  g++ /builder/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


Wow, just noticed that this is an absolute path.

peterjc123 · 2020-05-18T03:10:24Z

check_binary.sh

@@ -287,7 +287,7 @@ build_example_cpp_with_incorrect_abi () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
+  g++ /builder/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


guyang3532 force-pushed the binary_test branch 4 times, most recently from 687b13a to 68407ea Compare May 12, 2020 02:55

peterjc123 reviewed May 12, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch 3 times, most recently from cb42869 to 7318044 Compare May 15, 2020 02:39

peterjc123 reviewed May 15, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch 6 times, most recently from 6492ccc to e4a05e3 Compare May 16, 2020 12:18

peterjc123 reviewed May 16, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch from e4a05e3 to 6c371b8 Compare May 17, 2020 09:10

Add smoke test for binary package

3187392

guyang3532 force-pushed the binary_test branch from 6c371b8 to 3187392 Compare May 17, 2020 13:49

peterjc123 mentioned this pull request May 17, 2020

Add ci binary test for windows pytorch/pytorch#38297

Closed

peterjc123 approved these changes May 17, 2020

View reviewed changes

soumith merged commit 8e5b65f into pytorch:master May 18, 2020

peterjc123 reviewed May 18, 2020

View reviewed changes

peterjc123 mentioned this pull request May 18, 2020

Using relative path to test example code #443

Merged

guyang3532 mentioned this pull request Jul 28, 2020

Add Windows tests for multiple configurations pytorch/pytorch#18319

Closed

malfet mentioned this pull request Jan 18, 2022

locally installed PyTorch upgraded or superseded on windows/macos conda tests pytorch/pytorch#71340

Open

		@@ -0,0 +1,4 @@
		set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
		curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

	curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
	curl --retry 3 -kL "%DRIVER_DOWNLOAD_LINK%" --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
	if errorlevel 1 exit /b 1

		if errorlevel 1 exit /b 1

		del 442.50-tesla-desktop-winserver-2019-2016-international.exe \|\| ver > NUL

Add smoke test for binary package #442

Add smoke test for binary package #442

Uh oh!

Conversation

guyang3532 commented May 11, 2020

Uh oh!

peterjc123 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peterjc123 commented May 16, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!