Make AppImageKit more cross-compile friendly

[nurupo]

I want to cross-compile my program to several architectures (e.g. i386, x86_64, armel, armhf, etc.) and publish each as an AppImage. Besides cross-compiling my own application, I found that AppImageKit itself needs to be cross-compiled for each architecture in order to create the AppImage bundles for them. However, it has proven to be rather challenging to cross-compile AppImageKit.

src/build-runtime.sh.in and cmake/dependencies.cmake are the biggest cross-compilation offenders here.

• src/build-runtime.sh.in assumes that the build is made using system's native toolchain, it uses gcc, strip, readelf, ld, etc. without any way to specify the triplet prefix.
• cmake/dependencies.cmake builds dependencies not found on the system without ability to set --host= configure's argument, which results in the build failure when cross-compiling.

Here are full examples of how I have patched AppImageKit when cross-compiling it to armel and i386 on an x86_64 machine, so that you could see what I'm talking about and be able to reproduce:

armel

sudo docker run --rm -it debian:stretch-slim /bin/bash

dpkg --add-architecture armel
apt-get update
apt-get install -y --no-install-recommends ca-certificates libgtest-dev:armel git ca-certificates make libarchive-dev:armel automake autoconf libtool make gcc g++ libfuse-dev:armel liblzma-dev:armel libglib2.0-dev:armel libssl-dev:armel libinotifytools0-dev:armel liblz4-dev:armel libcairo-dev:armel desktop-file-utils cmake patch wget xxd patchelf

apt-get install -y --no-install-recommends binutils-arm-linux-gnueabi gcc-arm-linux-gnueabi g++-arm-linux-gnueabi

echo "
    SET(CMAKE_SYSTEM_NAME Linux)

    SET(CMAKE_C_COMPILER   /usr/bin/arm-linux-gnueabi-gcc)
    SET(CMAKE_CXX_COMPILER /usr/bin/arm-linux-gnueabi-g++)
    SET(CMAKE_AR           /usr/bin/arm-linux-gnueabi-gcc-ar CACHE FILEPATH "Archiver")

    SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
    SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
    SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
" > /usr/local/share/toolchain.cmake

cd /usr/src/gtest
cmake -DCMAKE_TOOLCHAIN_FILE=/usr/local/share/toolchain.cmake .
make install
cd -

# Make AppImageKit pick up liblzma.a instead of liblzma.so by hiding liblzma.so
mv /usr/lib/arm-linux-gnueabi/liblzma.so /usr/lib/arm-linux-gnueabi/foo-liblzma.so

git clone --recursive https://github.com/AppImage/AppImageKit
cd AppImageKit
git checkout c0ff6f738a65bfa890d5286d81d97754167237ea
# The broken stuff
sed -i 's|<SOURCE_DIR>/configure |<SOURCE_DIR>/configure --host=arm-linux-gnueabi |' cmake/dependencies.cmake
sed -i 's|gcc|/usr/bin/arm-linux-gnueabi-gcc|' src/build-runtime.sh.in
sed -i 's|ld |/usr/bin/arm-linux-gnueabi-ld |' src/build-runtime.sh.in
sed -i 's|objcopy |/usr/bin/arm-linux-gnueabi-objcopy |' src/build-runtime.sh.in
sed -i 's|readelf |/usr/bin/arm-linux-gnueabi-readelf |' src/build-runtime.sh.in
sed -i 's|strip|/usr/bin/arm-linux-gnueabi-strip|' src/build-runtime.sh.in
sed -i 's|objdump |/usr/bin/arm-linux-gnueabi-objdump |' src/build-runtime.sh.in
mkdir build
cd build
# Don't mind the pkg-config command, this is not broken, that's the proper way to do it
export PKG_CONFIG_LIBDIR=/usr/lib/arm-linux-gnueabi/pkgconfig:/usr/share/pkgconfig
cmake -DCMAKE_BUILD_TYPE=RELEASE -DBUILD_TESTING=OFF -DCMAKE_TOOLCHAIN_FILE=/usr/local/share/toolchain.cmake -DUSE_SYSTEM_XZ=ON -DUSE_SYSTEM_INOTIFY_TOOLS=ON -DUSE_SYSTEM_LIBARCHIVE=ON -DUSE_SYSTEM_GTEST=ON ..
make VERBOSE=1
make install

# Restore it back
mv /usr/lib/arm-linux-gnueabi/foo-liblzma.so /usr/lib/arm-linux-gnueabi/liblzma.so

i386

sudo docker run --rm -it debian:stretch-slim /bin/bash

dpkg --add-architecture i386
apt-get update
apt-get install -y --no-install-recommends ca-certificates git lib32z1-dev libgtest-dev:i386 gcc-multilib g++-multilib libarchive-dev:i386 automake autoconf libtool make gcc g++ libfuse-dev:i386 liblzma-dev:i386 libglib2.0-dev:i386 libssl-dev:i386 libinotifytools0-dev:i386 liblz4-dev:i386 libcairo-dev:i386 desktop-file-utils cmake patch wget xxd patchelf

# There is no i386 cross-toolchain, you are supposed to use the multilib support for it, so let's create mock toolchain

echo '
#!/bin/sh
/usr/bin/x86_64-linux-gnu-gcc -m32 "$@"
' > /usr/bin/i386-linux-gnu-gcc
chmod +x /usr/bin/i386-linux-gnu-gcc

echo '
#!/bin/sh
/usr/bin/x86_64-linux-gnu-g++ -m32 "$@"
' > /usr/bin/i386-linux-gnu-g++
chmod +x /usr/bin/i386-linux-gnu-g++

echo '
#!/bin/sh
/usr/bin/x86_64-linux-gnu-ar "$@"
' > /usr/bin/i386-linux-gnu-ar
chmod +x /usr/bin/i386-linux-gnu-ar

echo "
    SET(CMAKE_SYSTEM_NAME Linux)

    SET(CMAKE_C_COMPILER   /usr/bin/i386-linux-gnu-gcc)
    SET(CMAKE_CXX_COMPILER /usr/bin/i386-linux-gnu-g++)
    SET(CMAKE_AR           /usr/bin/i386-linux-gnu-ar CACHE FILEPATH "Archiver")

    SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
    SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
    SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
" > /usr/local/share/toolchain.cmake

cd /usr/src/gtest
cmake -DCMAKE_TOOLCHAIN_FILE=/usr/local/share/toolchain.cmake .
make install
cd -

git clone --recursive https://github.com/AppImage/AppImageKit
cd AppImageKit
git checkout c0ff6f738a65bfa890d5286d81d97754167237ea
# The broken stuff
sed -i 's|gcc|/usr/bin/i386-linux-gnu-gcc|' src/build-runtime.sh.in
sed -i 's|ld |ld -m elf_i386 |' src/build-runtime.sh.in
mkdir build
cd build
# Don't mind the pkg-config command, this is not broken, that's the proper way to do it
export PKG_CONFIG_LIBDIR=/usr/lib/i386-linux-gnu/pkgconfig:/usr/share/pkgconfig
cmake -DCMAKE_BUILD_TYPE=RELEASE -DBUILD_TESTING=OFF -DCMAKE_TOOLCHAIN_FILE=/usr/local/share/toolchain.cmake -DUSE_SYSTEM_XZ=ON -DUSE_SYSTEM_INOTIFY_TOOLS=ON -DUSE_SYSTEM_LIBARCHIVE=ON -DUSE_SYSTEM_GTEST=ON ..
make VERBOSE=1
make install

[TheAssassin]

You don't need to cross-compile appimagetool. You just need the right runtime for your platform. You can e.g., download the i386 runtime, and use it with appimagetool by passing via a CLI parameter.

[nurupo]

download the i386 runtime

Oh, I forgot to mention that I want to build AppImage from source, without downloading pre-built executables.

[TheAssassin]

I'm just trying to say that it should suffice cross-compile the runtime. That could save a lot of time.

[nurupo]

That's great to know. How do I cross-compile just the runtime?

[TheAssassin]

Well, you could just call make runtime, couldn't you? You'd still need to compile the dependencies, though.

Actually, I have plans to extract the runtime from the AppImageKit project, and move it into a separate repo. That'd make building it easier.

[nurupo]

Also, is the runtime enough for linuxdeployqt? Because I also use linuxdeployqt to -bundle-non-qt-libs -appimage my Qt program.

[TheAssassin]

linuxdeployqt bundles appimagetool, and it is hardcoded how appimagetool is called from it. Better don't use -appimage but download and call appimagetool directly with the correct options.

[probonopd]

Better don't use -appimage but download and call appimagetool directly with the correct options.

What do you mean by this?

[TheAssassin]

@probonopd please read the rest of the thread, and you'll see what is meant by that. There is no way to tell linuxdeployqt to tell appimagetool use a custom runtime with -appimage.

[probonopd]

Ah right, to use advanced options of linuxdeployqt one needs to run it manually, not using the -appimage option of linuxdeployqt. This is intentional so that only people who know what they are doing get exposed to the linuxdeployqt command line options.

[TheAssassin]

Ah right, to use advanced options of linuxdeployqt one needs to run it manually, not using the -appimage option of linuxdeployqt. This is intentional so that only people who know what they are doing get exposed to the linuxdeployqt command line options.

JFYI linuxdeploy's AppImage plugin aims to provide more configurability how the AppImage is generated.

[TheAssassin]

@nurupo I will try to rewrite the runtime build script in CMake today, that'll help making AppImageKit more cross compile friendly.

Perhaps we can even get rid of the 32-bit version of your AppImageBuild containers, build a multilib version of gcc and build the 32-bit versions of the dependencies in there as well. Then, the 32-bit builds could be created through cross compiling in the 64-bit container, using a toolchain like https://github.com/TheAssassin/linuxdeploy/blob/master/cmake/toolchains/i386-linux-gnu.cmake.

[nurupo]

Perhaps we can even get rid of the 32-bit version of your AppImageBuild containers

Um, those are not my containers at https://github.com/AppImage/AppImageBuild. My container is provided in the opening post of this thread and it does use multilib gcc to build 32-bit versions of my software and the AppImage on a 64-bit machine, notice the -m32 in my toolchain wrapper scripts, and it does use a CMake toolchain file to make CMake use those wrappers. Also, Debian's gcc has multilib support, there is no need to build gcc from source, just install gcc-multilib and g++-multilib packages.

[TheAssassin]

@nurupo the point is that we can make use of cross compilation support ourselves, and that increases the priority of this issue, which is in your interest.

I know you don't use AppImageBuild, but we do. And if you look into the Dockerfiles, you'll see that we don't use Debian, we use a really old version of CentOS. We're building our own compiler because it won't work with the outdated one the CentOS packages allow us to use. Same goes for a couple of other dependencies, e.g., GLib.