Automated ARM builds

[coolaj86]

I would be happy to build arm6 and arm7 bins for both sqlite3 and sqlcipher.

v0.10 and v1.2 work

v0.11 and <= v1.1 fail

v0.12 I haven't tested.

I have internet-accessible PIs connected to a few routers that I can offer for such a task.

[springmeyer]

We build all binaries on travis (for linux and mac) and appveyor (for windows). What would be very useful is a travis job that used qemu or something like it to build rpi binaries. @bander9289 has been experimenting with this (and I've been meaning to learn but not had time yet): mapnik/mapnik-packaging@master...bander9289:master#diff-7142a27433410bb3f7e3ae34a1674807R16

@coolaj86 - interested in providing a pull request adding rpi build support to travis.yml?

/cc @mojodna who may also be interested in this effort.

[mojodna]

Hmm. Interested I am, yet not yet needing it. Thanks.

[coolaj86]

My work on embedded systems has lead me to believe that emulation is no substitution for compiling on the real deal, so I wait the 15+ minutes out when I need it.

I'm open to trying something like Qemu, but it would probably be easier for me to set up a Pi as a build server than to learn Qemu and what special compile options I'd need to pass and such.

What are the advantages of Qemu that make you prefer that solution?

[alexellis]

@coolaj86 does it not come down to automation, automation, automation and repeatability? If I were to take a binary for sqlite3 instead of source, then I'd definitely want it to have been built through the 'trusted' channels - not a third party. Having said that something really needs to be done #597

[marvinroger]

Scaleway baremetals ARMv7 servers, might help here. Each build, assuming it would last less than 59 minutes, would cost €0.006.

Moreover, with their CLI, create, start and SSH into a server is as easy as:

$ scw exec $(scw start --wait $(scw create ubuntu-trusty)) /bin/bash
[root@noname ~]#

Note I am not affiliated to them in any way, that's just how I handle my personal Docker builds. 😉

[kkaefer]

Note that we don't have build on ARMv7; instead we could also cross-compile.

[alexellis]

The build takes a ridiculous amount of time to complete especially on the Pi Zero - that I'd give up on sqllite or use python. Please provide a binary through npm to support continued use.

[skl]

I've automated an ARM build with Vagrant, Docker (as opposed to chroot in @bander9289's approach) and QEMU which is one option for an overall npm install use case for ARM targets - whilst being able to build on a more powerful machine. I haven't open sourced it yet but this is the general idea that I use for sqlite3 and bcrypt:

1. Spin up a single-core vagrant box with "debian/jessie64"
2. Install docker-engine and qemu-user-static in the vm
3. Spin up a docker container with docker run -it --volume /usr/bin/qemu-arm-static:/usr/bin/qemu-arm-static philipz/rpi-raspbian bash
4. Install build-essential and nodejs in the container
5. npm install inside the container

I was not able to get a full QEMU ARM VM running but I did have success with the QEMU User Mode emulation as above. So it runs only the docker process under ARM emulation rather than providing a full virtual machine.

One major drawback of either QEMU approach is that is only supports a single thread, so for example I could not install Git LFS within the ARM docker container as Go binaries cannot be run through QEMU due to the concurrency aspects of the language. So my advice is do as much as you can within the "host VM" then delegate the ARM portions to the container.

Maybe some of the above could help for the Travis build? Cross-compilation would be ideal but it didn't seem straight forward to me, that's why I went the QEMU route instead as it also covers other npm modules at the same time.

[StefanScherer]

@skl What about using Travis or at least using the steps described here: https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/

[skl]

Thanks for the article, uses the same principal and applies it to Travis. Just in my case I need private builds for work.