CI build platforms

[rvagg]

With both Node and libuv being very widely adopted across disparate platforms, it's time for a CI system to match that spread. We should be able to define a list of primary targets that are essential as part of the mix and secondary targets that add additional value but not a main focus of the core team.

Current Node.js core and libuv Jenkins build bot list: http://jenkins.nodejs.org/computer/

Let's try and limit this discussion to CI as much as possible and leave release build platforms for another discussion.

Likely using Jenkins with a very distributed collection of build bots. I've been in contact with DigitalOcean, IBM and @mmalecki so far on hardware provisioning, looking forward to Rackspace and any others that want to step up. NodeSource is happy to cop the maintenance burden and likely some of the cost and do the bidding of the core team(s).

Here's my straw-man, to start discussion off:

Primary

• Linux (64-bit with at least one 32-bit, maybe CentOS)
  • Ubuntu LTS versions still being supported
  • Ubuntu latest stable
  • EL last three versions (CentOS 5, 6 & 7 in lieu of RHEL 5, 6 & 7)
  • Debian stable
  • Something for ARMv6 (rpi) and ARMv7
• Windows (64-bit only)
  • Windows Server 2008 R2 (NT 6.1, same as Windows 7)
  • Windows Server 2012 (same as Windows 8)
  • Need variations for VS 2012 and VS 2013
• OSX (64-bit only)
  • 10.8 "Mountain Lion"
  • 10.9 "Mavericks"
• Solaris (64-bit only)
  • SmartOS 13.4.2
  • SmartOS 14.2.0

Secondary

• Linux
  • Debian unstable & testing
  • EL next (CentOS 7 beta)
• Windows
  • Windows 7 32-bit
  • MinGW
  • VS 2010 on something
• FreeBSD
• POWER

---

Looking for input from anyone but particularly the core team who need to be the ones deciding which are the primary platforms they actually care about, and we're considering both Node and libuv here. I'm happy to do a bunch of the legwork for you but I'll need your guidance because build targets is not my decision to make.

@tjfontaine @bnoordhuis @piscisaureus @trevnorris @TooTallNate @saghul @indutny

Others who have shown an interest in this discussion (or I just feel like pulling in!):

@ingsings @pquerna @voodootikigod @mmalecki @andrewlow @guille @othiym23 @dshaw @wblankenship @wolfeidau

Please subscribe to https://github.com/node-forward/build for further notifications from other issues if you're interested so we don't have to go and pull everyone in each time.

[bnoordhuis]

Debian unstable & testing

debian/testing seems fine (I think that's what most Debian desktop users run - I do, at least) but sid is frequently broken. It might cause a lot of false negatives.

SmartOS 13.4.2 / SmartOS 14.2.0

Is anyone outside of Joyent running SmartOS? The number of (non-SmartOS) Solaris users is a rounding error, I know both of them personally.

Need variations for VS 2012 and VS 2013

There was a post on the v8-users mailing list this morning that VS 2012 support is being phased out. Node.js is a few V8 releases behind, of course.

[andrewlow]

Please seriously consider supporting 32bit versions of the Node binaries. The build infrastructure can be 64bit versions of the OS, but only offering 64bit binaries means we're going to run V8 in 64bit mode.

For most Node applications this will be a waste of memory since they don't need to access more than 4G address space and V8 today doesn't do anything beyond a little bit of header compression in the 64bit environment.

Also for what it's worth - Node can probably be built once on a 64bit Linux machine, then packaged as a .dep, rpm, tgz for the various distros. I know from experience that building on RHEL 6.5 produces a binary that runs just fine on Ubuntu 10 and 12. Let's make it a goal to have as few unique binary builds as we can (while allowing for installer packaging wrappers to support the various OSes)

[rmg]

From a never-looked-at-v8-internals POV, I would expect the extra registers from 64-bit mode would be quite useful for a VM. @andrewlow does the memory bloat outweigh the benefits of doubling the number of registers available?

[andrewlow]

Intel benefits from having more registers in 64bit mode (and more instructions) but it's not enough to overcome the object size bloat. If you do the trick Java did with compressed references http://lowtek.ca/roo/2008/java-performance-in-64bit-land/ then you do get an overall win, but if you've done that you're really using 32bit objects.

PowerPC gets wider registers, but no new instructions or registers. It's more of a challenge on that platform.

I'm not sure where ARM and MIPS sit with 64bit, I suspect they don't get as big as win as Intel does.

In the end, the memory is so slow relative to CPU speed that having to move more is bad for performance.

[rmg]

@andrewlow thanks for the explanation :-)

[rvagg]

/cc @chrislea

[chrislea]

It's an interesting discussion to me. Before I continue, I will state for the record that I strongly suspect I have considerably less knowledge about the register state / pointer size / performance issues than @andrewlow does.

What I've seen from experience is that convenience and familiarity tend to trump, essentially, everything else. In this case specifically, the end user being able to type apt-get install nodejs or yum install nodejs and having the tooling of their Linux distro of choice install Node for them would, I am guessing, be more useful for overall adoption rates than putting effort into shipping 32bit builds into 64bit machines for some extra bit of runtime speed. I guess this because I also assume that if people were that concerned about runtime speed they'd be developing in Java (or something else that's not Node). I would also guess that most of these apps people are writing in Node are generally very I/O bound, so the runtime speed isn't the bottleneck.

This is sort of tossing $0.02 into the air I know. And I could be dead wrong if there are enterprise customers who really care about benchmarks. But without additional info, I'd continue to ship arch specific builds simply for the reason that it's easier to make everything "just work" in familiar ways to the end user when they are installing and updating.

[saghul]

MinGW

I would very much like to see MinGW (it's 3 variants) as a primary. Lots of hours have gone into making them actually work, and many people rely on them. Proof if that is that every time we break some MinGW build I get a pull request with a fix, which I need to manually try.

Supporting these doesn't require new gear, the 3 toolchains can be installed on a Windows 7 or 8 machine.

[saghul]

Obviously my previous comment was regarding libuv :-)

[rvagg]

@saghul from a previous thread you listed:

• MinGW32
• MinGW-w64 (32 bit)
• MinGW-w64 (64bit)

Which I'm guessing are the ones you are concerned about. Are the build instructions for these included with libuv somewhere or does gyp do a neat job of sorting things out?

[saghul]

@rvagg yes, those are the ones.

As for how to build them: I found that autoconf is the easiest way. GYP can also work, but it's not officially supported.

The MinGW32 installer comes with a graphical package manager allows you to install some packages, so once you have autoconf, automake and libtool you can just build an run with ./autogen.sh && ./configure && make && make check.

For MinGW-w64, I'd suggest to use MSYS2. There is an installer to get things started. Then in the MSYS2 shell you can install the 2 MinGW-w64 toolchains using pacman (they ported the Arch Linux package manager) and also autoconf, automake and libtool. Building is done the same (running these commands on the appropriate shell): ./autogen.sh && ./configure && make && make check.

I don't have much free time, but I can try to help if needed.

[rvagg]

FYI I've been messing with Buildbot and am noticing that CentOS 5 is going to require some work to make the tests a little less brittle across both Node and libuv. Mostly things work but there are a couple of tests in both that persistently fail and a few that occasionally fail.

I know this is a pain but I've run into two major US companies already that are deploying Node into RHEL5 still in production. Ultimately it's up to the TC to decide minimum supported operating systems but I'd like to encourage them to seriously consider supporting EL 5, within a reasonable timeframe (perhaps till the end of the year when 7 starts to gain acceptance, which means enterprise is more likely to be running on 6!).

I don't want to share a URL in here but if anyone's interested in taking a look at the failures then email me @ rvagg@nodesource.com and I'll hook you up and can even give Node & libuv core members access to a box to play around with if needed.

[rvagg]

Next question: llvm vs gcc, is there anything worth testing there or are we assured that compiling with both leads to a similar enough result? I've seen many opting for llvm compiles because of lldb goodness and quicker compiles I think.

[chrislea]

I've talked to a few people including TJ about this.

The answer is "use gcc when on Linux", even though you can build Node with
clang.

The reason is that C++ doesn't have a stable ABI (which I always manage to
forget because it's 2014 and really !!??!?), and so if people are
building binary modules and their system defaults to using gcc, which it
will, it's possible that things just won't work unless you use the same
compiler.

So, use gcc, and more specifically use the gcc that your target distro ships with.

Annoying, but no way around it. So there you go.

On Sun, Aug 10, 2014 at 9:18 PM, Rod Vagg notifications@github.com wrote:

Next question: llvm vs gcc, is there anything worth testing there or are
we assured that compiling with both leads to a similar enough result?
I've seen many opting for llvm compiles because of lldb goodness and
quicker compiles I think.

—
Reply to this email directly or view it on GitHub
#1 (comment).

__
https://chrislea.com
http://about.me/chrislea
310-709-4021