pywavelets fails to install on Alpine Linux v3.15 ARM

[andrey18106]

Hi! Trying to install pywavelets on Nextcloud 23 Docker Alpine image with ARM and facing next issue:

Using pip 22.0.2 from /tmp/pip-standalone-pip-iygumj5_/__env_pip__.zip/pip (python 3.9)
  Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "aarch64"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.9" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64" and platform_python_implementation != "PyPy"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.8" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64" and platform_python_implementation != "PyPy"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.10" and platform_python_implementation != "PyPy"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.11"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_python_implementation == "PyPy"' don't match your environment

numpy already installed:

python3 -m pip show numpy
Name: numpy
Version: 1.21.4

[rgommers]

That's a partial log of a build from source. We do have 64-bit ARM wheels, so are you on 32-bit ARM?

Can you please provide a full build log, here or as a gist?

[andrey18106]

I didn't found 64bit pywavelets wheels for Alpine (musllinux_1_1_).

Full build log available here.

[rgommers]

Ugh, we have aarch64 wheels and musllinux_1_1 wheels, but not musl-on-aarch64 wheels.

So yes you have to build from source. The relevant error is:

RuntimeError: Broken toolchain: cannot link a simple C program

So you don't have a compiler installed or it's not working correctly for some reason.

[rom1504]

I'm getting something very similar:

pip install PyWavelets
Collecting PyWavelets
  Using cached PyWavelets-1.2.0.tar.gz (4.6 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      setup.py:366: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command, proceeding with "
      Traceback (most recent call last):
        File "/home/ec2-user/.env/lib64/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/home/ec2-user/.env/lib64/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/ec2-user/.env/lib64/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-16nqypti/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 174, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-16nqypti/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 268, in run_setup
          self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-16nqypti/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 478, in <module>
          setup_package()
        File "setup.py", line 468, in setup_package
          ext_modules = get_ext_modules(USE_CYTHON)
        File "setup.py", line 182, in get_ext_modules
          from numpy import get_include as get_numpy_include
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  

on a c6g.4xlarge aws instance (https://aws.amazon.com/ec2/instance-types/ c6g)

Custom built AWS Graviton2 Processor with 64-bit Arm Neoverse cores

what would you advise regarding compiling ?

[grlee77]

@rom1504, that looks like pip did not automatically install NumPy as specified in the pyproject.toml file's build requirements.

We have not tested on the AWS Graviton2 platform, so it is possible that we haven't set the numpy version selection logic in pyproject.toml properly for that case. If you manually install the dependencies via python -m pip install numpy cython setuptools wheel does that complete successfully?

I may also be worth first upgrading pip via: python -m pip install -–upgrade pip. If you are still having problems after this can you post a more verbose output using pip install pywavelets -vvv

[rom1504]

hi @grlee77 , I tested again on this platform and if I'm using ubuntu 20.04 instead of aws centos based distribution, it seems to be working.
I guess the difference might come from using python 3.8 in ubuntu 20.04 instead of 3.7 in aws distribution
so this works for me.
I could try again on the aws distribution if that's useful for you

[letmaik]

Same problem here. Using GitHub Actions and building within the quay.io/pypa/manylinux2014_aarch64 docker image. I note that 1.1.1 has manylinux2014_aarch64 wheels, whereas 1.2.0 does not. Previously my build wouldn't fail as 1.1.1 was used, but now this bar got raised by some transitive dependency (probably scikit-image) so that it tries to build 1.2.0 from source (given the lack of wheels) and fails. So IMO there are two issues here: failing build, missing wheel.

 Collecting PyWavelets>=1.1.1
  Downloading PyWavelets-1.2.0.tar.gz (4.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 8.8 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      setup.py:366: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command, proceeding with "
      Traceback (most recent call last):
        File "/opt/_internal/cpython-3.7.12/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/_internal/cpython-3.7.12/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/_internal/cpython-3.7.12/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-zfcvmo8d/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-zfcvmo8d/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 282, in run_setup
          self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-zfcvmo8d/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 478, in <module>
          setup_package()
        File "setup.py", line 468, in setup_package
          ext_modules = get_ext_modules(USE_CYTHON)
        File "setup.py", line 182, in get_ext_modules
          from numpy import get_include as get_numpy_include
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

[letmaik]

Looking at the pyproject.toml there is no variant that applies to Python 3.7 and aarch64:

"numpy==1.17.3; python_version=='3.7' and (platform_machine!='arm64' or platform_system!='Darwin') and platform_machine!='aarch64' and platform_python_implementation != 'PyPy'",

[rgommers]

Looking at the pyproject.toml there is no variant that applies to Python 3.7 and aarch64:

That does sound right. NumPy 1.20.0 does have an aarch64 wheel for Python 3.7: numpy-1.20.0-cp37-cp37m-manylinux2014_aarch64.whl

Adding this to pyproject.toml will probably fix that one:

"numpy==1.20.0; python_version=='3.7' and platform_machine=='arm64' and platform_system=='Darwin'",

That's different from the original report on Python 3.9 though, which was due to musllinux.

[luckydonald]

Yes, also stumbled into that second bug of the ticket, on a regular python docker image on a regular ubuntu host.

[luckydonald]

In fact, downgrade to installing PyWavelets==1.1.1 did work with the docker image pypy:3.7-buster

Here's some install logs:

PyWavelets==1.1.1 ✅

root@ef1afc23bfa5:/app# pip install --force-reinstall  PyWavelets==1.1.1
Collecting PyWavelets==1.1.1
  Downloading PyWavelets-1.1.1.tar.gz (4.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 43.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting numpy>=1.13.3
  Downloading numpy-1.21.5-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.2/15.2 MB 51.7 MB/s eta 0:00:00
Building wheels for collected packages: PyWavelets
  Building wheel for PyWavelets (setup.py) ... done
  Created wheel for PyWavelets: filename=PyWavelets-1.1.1-pp37-pypy37_pp73-linux_x86_64.whl size=4365536 sha256=45ca48848873de2381e4bffe7b0fae306849e0555cba826fffdcc0d6e5de57c3
  Stored in directory: /tmp/pip-ephem-wheel-cache-13r9xh5k/wheels/1e/9f/59/0c5dcd724cfc80b3cc7162beff7af6c52dae351afd991b1058
Successfully built PyWavelets
Installing collected packages: numpy, PyWavelets
  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.2
    Uninstalling numpy-1.21.2:
      Successfully uninstalled numpy-1.21.2
  Attempting uninstall: PyWavelets
    Found existing installation: PyWavelets 1.1.1
    Uninstalling PyWavelets-1.1.1:
      Successfully uninstalled PyWavelets-1.1.1
Successfully installed PyWavelets-1.1.1 numpy-1.21.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@ef1afc23bfa5:/app# 

PyWavelets==1.2.0 ❌

root@ef1afc23bfa5:/app# pip install --force-reinstall  PyWavelets==1.2.0
Collecting PyWavelets==1.2.0
  Downloading PyWavelets-1.2.0.tar.gz (4.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 43.8 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      setup.py:366: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command, proceeding with "
      Traceback (most recent call last):
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-rnifkqqy/overlay/site-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-rnifkqqy/overlay/site-packages/setuptools/build_meta.py", line 282, in run_setup
          self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-rnifkqqy/overlay/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 478, in <module>
          setup_package()
        File "setup.py", line 468, in setup_package
          ext_modules = get_ext_modules(USE_CYTHON)
        File "setup.py", line 182, in get_ext_modules
          from numpy import get_include as get_numpy_include
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
root@ef1afc23bfa5:/app# 

PyWavelets==1.3.0 ❌

root@ef1afc23bfa5:/app# pip install --force-reinstall  PyWavelets==1.3.0
Collecting PyWavelets==1.3.0
  Downloading PyWavelets-1.3.0.tar.gz (4.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 54.3 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      setup.py:366: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command, proceeding with "
      Traceback (most recent call last):
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/pypy/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-yguhu8o7/overlay/site-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-yguhu8o7/overlay/site-packages/setuptools/build_meta.py", line 282, in run_setup
          self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-yguhu8o7/overlay/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 478, in <module>
          setup_package()
        File "setup.py", line 468, in setup_package
          ext_modules = get_ext_modules(USE_CYTHON)
        File "setup.py", line 182, in get_ext_modules
          from numpy import get_include as get_numpy_include
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
root@ef1afc23bfa5:/app# 

[rgommers]

The 1.2.0 release (https://pypi.org/project/PyWavelets/1.2.0/#files) has

PyWavelets-1.2.0-cp310-cp310-musllinux_1_1_x86_64.whl 
PyWavelets-1.2.0-cp310-cp310-musllinux_1_1_i686.whl 

and the same for Python 3.7, 3.8 and 3.9. No aarch64 musllinux wheels though - the reason for that is (from gh-610): I had to disable some musl wheels (see PEP656) on aarch64 or those jobs are right around the timeout threshold. That timeout is why there is a CIBW_SKIP: "*-musllinux_*" set for this case.

The 1.1.1, 1.3.0 and 1.4.0/1 releases all have no musllinux wheels at all (this was skipped on purpose since commit 731e093). So the 1.1.1 successful install reported in the comment above is for building from source. The skipping of all musllinux wheels was noted in the PR description of gh-626, but it looks like there was no reason given - probably because numpy et al. also don't provide such wheels. Those are coming though, for numpy 1.25.0 I believe (June'23 probably).

We've dropped Python 3.7 support already (pyproject.toml contains requires-python = ">=3.8), so I think that is what it is. The minimum numpy version for the next release is 1.20.3, which AFAIK builds on Alpine Linux just fine (numpy has no musllinux wheels).

The pyproject.toml content has been updated, so I'm not sure if there's anything to do. Perhaps starting to build musllinux wheels again, at least for x86_64?