image.so on Apple Silicon can't load libpng16.16.dylib and libjpeg.9.dylib

[alanyoungCN]

🐛 Describe the bug

code:

import torchvision

UserWarning:

/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: dlopen(/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Library not loaded: @rpath/libpng16.16.dylib
  Referenced from: /Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so
  Reason: tried: '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/lib-dynload/../../libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/bin/../lib/libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file)
  warn(f"Failed to load image Python extension: {e}")

on M1 Max.
In #5137 (comment), it is said to be fixed but I don't think so.

Therefore you can't use torchvision.io.read_img().

import torchvision
img = torchvision.io.read_img('./1.jpg')

RuntimeError: No such operator image::read_file

In https://discuss.pytorch.org/t/failed-to-load-image-python-extension-could-not-find-module/140278/20, Andrade has the same problem.

Versions

Collecting environment information...
PyTorch version: 1.10.2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.1 (arm64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.29.30)
CMake version: version 3.22.0
Libc version: N/A

Python version: 3.9.7 (default, Sep 16 2021, 23:53:23) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-12.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.2
[pip3] torch==1.10.2
[pip3] torchaudio==0.10.2
[pip3] torchvision==0.11.3
[conda] Could not collect

cc @seemethere @malfet

[alanyoungCN]

BTW, I use pip to install the packages above. I didn't try conda.

[mnslarcher]

Same here, I also used pip, I cannot use conda at work

[zhiqwang]

I've tried Miniconda on M1, but failed with UnsatisfiableError

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

[alanyoungCN]

@zhiqwang
You can try reinstalling pillow==6.1(6.x could be okay), which is mentioned at python-pillow/Pillow#4130 (comment). Installing torchvision by conda will automatically use the latest pillow(7.0). I met this problem when I used conda to install torch. But UnsatisfiableError is weird.

[NicolasHug]

Pinging @jdsgomes: we should try to triple check with the release engineers that M1 will be properly supported in the upcoming release.

[jdsgomes]

@alanyoungCN would you be able to try the latest RC build (torchvision==0.12 and pytorch=1.11) for the upcoming release?

conda install -c pytorch-test torchvision pytorch

or

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html

[alanyoungCN]

@jdsgomes @malfet cc plz
After thinking it for a long while, I find the key is libpng and libjpeg.

The image.so is indeed in torchvision/ even when torchvision=0.11.3, I only checked the torchvision/io/ folder before. So sorry for the previous title.

Then I take a deep look at the warning(same as the top):

/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: dlopen(/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Library not loaded: @rpath/libpng16.16.dylib
  Referenced from: /Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so
  Reason: tried: '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/lib-dynload/../../libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/bin/../lib/libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file)
  warn(f"Failed to load image Python extension: {e}")

I think the missing of @rpath/libpng16.16.dylib is the key. And the warning on my Mac even has a malfet name, which is so surprising.

Following the Reason: tried: ..... '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' ....., I try conda install libpng in my base conda env. Then I activate the torchvision env and import torchvision, the warning becomes Library not loaded: @rpath/libjpeg.9.dylib. This is very confusing.

What I think for now is that dlopen image.so needs both libpng and libjpeg and both of them can't be found in ARM Mac, but I'm not sure.

[NicolasHug]

/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: dlopen(/Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so, 0x0006): Library not loaded: @rpath/libpng16.16.dylib
  Referenced from: /Users/alanyoung/Documents/Codes/Kaggle/ClassifyLeaves/venv/lib/python3.9/site-packages/torchvision/image.so
  Reason: tried: '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/Users/malfet/miniforge3/envs/py_39_torch-1.10.2/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/lib-dynload/../../libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/lib/libpng16.16.dylib' (no such file), '/opt/homebrew/Caskroom/miniforge/base/bin/../lib/libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file)
  warn(f"Failed to load image Python extension: {e}")

I think the missing of @rpath/libpng16.16.dylib is the key. And the warning on my Mac even has a malfet name, which is so surprising.

@malfet looks like the linker is trying to link against paths on your laptop :p

[seemethere]

I think this would probably be resolved if we were building these in CI rather than on a local laptop. Maybe related to #5171 ?