Kiwisolver build fails, Python.h not found

[RobertFlatt]

System:

Ubuntu 24.04
System Python 3.12.3

buildozer.spec is default except:

requirements = python3,kivy,kiwisolver
p4a.branch = develop

Log:

running build_ext
building 'kiwisolver._cext' extension
creating build/temp.linux-x86_64-cpython-311/py/src
/home/bobf/.buildozer/android/platform/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -target armv7a-linux-androideabi24 -fomit-frame-pointer -march=armv7-a -mfloat-abi=softfp -mfpu=vfp -mthumb -fPIC -target armv7a-linux-androideabi24 -fomit-frame-pointer -march=armv7-a -mfloat-abi=softfp -mfpu=vfp -mthumb -fPIC -DANDROID -I/home/bobf/.buildozer/android/platform/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include -I/home/bobf/ex/hello/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a/build/python-installs/myapp/armeabi-v7a/include/python3.1 -fPIC -I/tmp/build-env-xfj5oz7g/lib/python3.11/site-packages/cppy/include -I. -I/tmp/build-env-xfj5oz7g/include -I/usr/local/include/python3.11 -c py/src/constraint.cpp -o build/temp.linux-x86_64-cpython-311/py/src/constraint.o -std=c++11
In file included from py/src/constraint.cpp:10:
In file included from /tmp/build-env-xfj5oz7g/lib/python3.11/site-packages/cppy/include/cppy/cppy.h:16:
/tmp/build-env-xfj5oz7g/lib/python3.11/site-packages/cppy/include/cppy/defines.h:10:10: fatal error: 'Python.h' file not found
#include <Python.h>
         ^~~~~~~~~~
1 error generated.

Comment:

Does build with p4a.branch = master, so possibly this is related to PyProjectRecipe #3007

[RobertFlatt]

Oh wait, I already reported this one #3056 and it was tagged priority low :(
I suggest it is not low because kiwisolver is a dependency of matplotlib https://github.com/kivy/python-for-android/blob/develop/pythonforandroid/recipes/matplotlib/__init__.py#L12

[mriscoc]

there is a problem with cppy and the develop branch of p4a, Kiwisolver depends on cppy.

[RobertFlatt]

@mriscoc I'd say that is about 3/4 right!

cppy is just a set a header files, it is download to /tmp and referenced from there; but there is no cppy specific recipe that would actually be the problem. And kiwisolver is the only recipe that depends on cppy.

This is a develop issue because master uses a different recipe template for kivisolver. The numpy recipe requires develop on this platform so develop is almost the de-facto choice.

I just finished building all the other PyProjectRecipe, but none show the same issue.

I suggest the kiwisolver recipe is the only place to provide any missing include path.

[T-Dynamos]

I think adding python include dir to CFLAGS would fix the error.

Here is modified version:

from pythonforandroid.recipe import PyProjectRecipe


class KiwiSolverRecipe(PyProjectRecipe):
    site_packages_name = 'kiwisolver'
    version = '1.4.5'
    url = 'git+https://github.com/nucleic/kiwi'
    depends = ['cppy']
    need_stl_shared = True

    def get_recipe_env(self, arch, **kwargs):
        env = super().get_recipe_env(arch, **kwargs)
        flags = " -I" + self.ctx.python_recipe.include_root(arch.arch)
        env["CFLAGS"] = flags
        env["CPPFLAGS"] = flags
        return env


recipe = KiwiSolverRecipe()

(untested)

[RobertFlatt]

Tried to test your fork, but having unrelated download issues with sdl2, will try again later.

FYI

[DEBUG]:        Cloning into 'external/libpng'...
[DEBUG]:        remote: Enumerating objects: 109787, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (6/6), done.
error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
[DEBUG]:        error: 3705 bytes of body are still expected
[DEBUG]:        fetch-pack: unexpected disconnect while reading sideband packet
[DEBUG]:        fatal: early EOF
[DEBUG]:        fatal: fetch-pack: invalid index-pack output

The error is consistent, though the number of bytes in the message is not.

[mriscoc]

I think adding python include dir to CFLAGS would fix the error.

Here is modified version:

from pythonforandroid.recipe import PyProjectRecipe

 class KiwiSolverRecipe(PyProjectRecipe):
     site_packages_name = 'kiwisolver'
     version = '1.4.5'
     url = 'git+https://github.com/nucleic/kiwi'
     depends = ['cppy']
     need_stl_shared = True
 
     def get_recipe_env(self, arch, **kwargs):
         env = super().get_recipe_env(arch, **kwargs)
         flags = " -I" + self.ctx.python_recipe.include_root(arch.arch)
         env["CFLAGS"] = flags
         env["CPPFLAGS"] = flags
         return env
 
 
 recipe = KiwiSolverRecipe()

(untested)

This change, from @T-Dynamos, allowed me to compile kiwisolver with the current PythonforAndroid development branch.

[RobertFlatt]

Great, please generate PR.

[mriscoc]

Great, please generate PR.

We should wait for another confirmation, I saw that the compilation process in P4A is not always consistent, so I did a buildozer android clean before to test and all was ok but my project has another dependencies than could mask another problems.

[RobertFlatt]

OK, I'll try again later.

(This is not for me, I'm advocating for a clueless user whose ability requires a turnkey solution)

[RobertFlatt]

In retrospect I wonder if that issue is due to the cppy files being under /tmp/

[T-Dynamos]

In retrospect I wonder if that issue is due to the cppy files being under /tmp/

It's not an issue, python -m build creates venv at /tmp and installs cppy as listed in
https://github.com/nucleic/kiwi/blob/2d7e60552bc59a6743714c78313f8199b1e7b888/pyproject.toml#L41 . It's just that it can't find python header.

[RobertFlatt]

Unrelated to Kivisolver, but while trying to test the Kiwisolver fix using your repository, get the following error:

[INFO]:    Building util-linux for arm64-v8a
[INFO]:    -> directory context /home/bobf/ex/hello/.buildozer/android/platform/build-arm64-v8a/build/other_builds/util-linux/arm64-v8a__ndk_target_24/util-linux
[DEBUG]:   -> running configure --host=aarch64-linux-android --without-systemd

(many checks removed for clarity)

[DEBUG]:        checking for ncurses/term.h... no
[DEBUG]:        checking for ncurses.h... (cached) no
[DEBUG]:        checking for term.h... (cached) no
[DEBUG]:        checking for tinfo... yes
[DEBUG]:        checking for readline in -lreadline... no
[DEBUG]:        checking for sqlite3... no
[DEBUG]:        configure: error: liblastlog2 selected, but required sqlite3 library not available
Exception in thread background thread for pid 108203:
Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/bobf/venv/lib/python3.12/site-packages/sh.py", line 1642, in wrap
    fn(*rgs, **kwargs)
  File "/home/bobf/venv/lib/python3.12/site-packages/sh.py", line 2647, in background_thread
    handle_exit_code(exit_code)
  File "/home/bobf/venv/lib/python3.12/site-packages/sh.py", line 2338, in fn
    return self.command.handle_command_exit_code(exit_code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bobf/venv/lib/python3.12/site-packages/sh.py", line 823, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: