Added tests and 32-bit targets to haxm-windows Travis script

[AlexAltea]

Previously, we couldn't build haxm-tests.vcxproj on TravisCI as it depends on the Windows SDK and:

• Windows SDK 8/8.1 required the Universal CRT to be separately installed for which no Chocolatey package is available (as of today). This is solved in Windows SDK 10.x as it's already included.
• Windows SDK 10.0.17134.12 and earlier failed under Windows Server 2016 (v1803) which is used by TravisCI VMs. This is solved in v10.0.17763.1, which wasn't available at the time.

The bug was filed to the maintainers of the Windows SDK and WDK (since they need to be synchronized) packages, and both offer now the version 10.0.17763.1 (v1809). Further details are available at:

• windows-sdk-10.1: https://chocolatey.org/packages/windows-sdk-10.1#comment-4196624327
• windowsdriverkit10: https://chocolatey.org/packages/windowsdriverkit10#comment-4201628042

Now the complete solution is built, both for 32-bit and 64-bit targets (on Debug mode), and tests are run. This solves the request at #123 (comment). I'm afraid 32-bit Linux builds might be too complex for now (not sure how to cross-compile drivers). We can leave this and parallelizing builds for future PRs. Either way, it's safe too assume that any 32-bit specific bugs will be caught by the 32-bit Windows builds.

Additionally, I have pinned the versions of both packages to prevent breaking the builds if they get updated again, see #129 (comment) and #126 (comment) (among others).

Signed-off-by: Alexandro Sanchez Bach [email protected]

[AlexAltea]

I forgot to mention: there's an added property /p:SpectreMitigation=false. This is required since the VS2017 Build Tools included in the TravisCI VM do not include Spectre-mitigated libraries, and the latest Windows SDK looks for them by default.

[AlexAltea]

Another remark: The Windows SDK still takes 5 minutes to install (I'll try to reach TravisCI devs about this). Nonetheless the total build time right now is 10 minutes, which is still within acceptable ranges, in my opinion. TravisCI limit is 50 minutes.

[raphaelning]

Thanks! Since this PR fixes the CI build, I'll merge it right away.

[raphaelning]

there's an added property /p:SpectreMitigation=false. This is required since the VS2017 Build Tools included in the TravisCI VM do not include Spectre-mitigated libraries [...]

I see. I used to get missing Spectre libraries errors too, when I upgraded VS2017 last time. I forgot how exactly I fixed the problem, but it should be by installing either WDK v1809, or the Spectre libraries via the VS2017 installer, probably the latter. Are the Travis folks aware of this issue?

[AlexAltea]

Are the Travis folks aware of this issue?

They might be aware, but right now they are overwhelmed with Windows issues (it's been in early access for only a month). Plus, the root issue isn't caused by them, but actually by visualstudio2017buildtools. After they fix it, TravisCI will need to update their VMs.

I'll take care of filing issues to the corresponding teams. In the meantime, disabling Spectre mitigations seems alright since we are just testing whether the driver builds (and not deploying it to end-users).

[raphaelning]

Absolutely! Even Chocolatey seems to be overwhelmed--their website is down...

[maronz]

I found the question of how to build a 32-bit Linux driver on a 64-bit system somewhat interesting. So I've done a little "proof of concept" script: build-HAXM-x86.zip

Please note that this script was not tested in a Travis-CI environment, but rather on a (live) Ubuntu 14.04 x64 system. The script built a 32-bit driver in under 6 minutes, with most of the time spend in downloading packages for the chroot setup. Maybe this will be faster in a Travis-CI environment, and im sure a few tweaks will be required to get it running there. But the resulting 32-bit module, was at least successfully tested on a Ubuntu 14.04 x86 system.

[raphaelning]

@maronz Thanks, please feel free to adapt the script for Travis. Hopefully it won't incur too much extra build time. Note that Linux-specific PRs need to be reviewed by @AlexAltea.

[raphaelning]

While reviewing #134 I noticed a compiler warning in the Travis build log for Windows, and it appears in this PR's Travis build log, too:

ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX86\x86\CL.exe /c /I"C:\Users\travis\build\intel\haxm\platforms\windows\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.0\build\native\include" /I"C:\Users\travis\build\intel\haxm\platforms\windows\packages\keystoneengine.0.9.1\installed\x86-windows\include" /Zi /nologo /W3 /WX- /diagnostics:classic /Od /Oy- /D HAX_TESTS /D WIN32 /D _DEBUG /D _CONSOLE /D _SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /D _UNICODE /D UNICODE /Gm /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"C:\Users\travis\build\intel\haxm\platforms\windows\build\intermediates\haxm-tests\Win32\\" /Fd"C:\Users\travis\build\intel\haxm\platforms\windows\build\intermediates\haxm-tests\Win32\\vc141.pdb" /Gd /TP /analyze- /FC /errorReport:queue ..\..\tests\test_emulator.cpp
cl : Command line warning D9035: option 'Gm' has been deprecated and will be removed in a future release [C:\Users\travis\build\intel\haxm\platforms\windows\haxm-tests.vcxproj]

It shouldn't be difficult to get rid of the deprecated /Gm (Enable Minimal Rebuild) flag, but I can't reproduce the warning in my local EWDK environment, even though I'm already using the latest EWDK (v1809, EWDK_rs5_release_17763_180914-1434.iso). I'll try another build using the latest VS2017 later.

[raphaelning]

Ah, it seems EWDK v1809 uses MSVC 14.15.26726, whereas Travis uses 14.16.27023. Probably that's the reason.

[maronz]

The reason that I have not done a Travis script straight away is simply a lack of experience in that matter on my part. I'm happy to go away and try to set up my own fork, and come back with a "proper" PR. I just thought it might be faster for @AlexAltea to use the parts from my script that create the chroot etc. and work it into the existing one.

In any case, this raises the question whether to have a single job for Linux for both architectures (like it is for Windows) or two separate jobs. If the idea of keeping the build artifacts comes to pass, maybe one job per architecture will be better (ending up with 5 jobs across the 3 platforms).

Anyway, I've updated the script somewhat to now build the modules for both architectures in one run:
build-HAXM.zip Please note, that I'm using essentially two copies of the tree, as I'm not sure whether any out-of-tree building is possible with the current Makefile.

[AlexAltea]

@maronz Thank you! I'll integrate your script into the Travis configuration file this weekend.

[AlexAltea]

EDIT: This issue is fixed.

@maronz I've tried integrating your script into the current Travis configuration file, and nearly succeeded: There seems to be an issue with the step:

$ schroot -c ${CHROOT} -u root -- apt install -y linux-headers-$(uname -r)
Reading package lists... Done
Building dependency tree... Done
E: Unable to locate package linux-headers-4.4.0-101-generic
E: Couldn't find any package by regex 'linux-headers-4.4.0-101-generic'

You can see the full logs and the corresponding script at:
https://travis-ci.com/kryptoslogic/haxm/jobs/163873857

When querying $ schroot -c ${CHROOT} -u root -- apt-cache search linux-header, I get:

linux-headers-3.13.0-24 - Header files related to Linux kernel version 3.13.0
linux-headers-generic - Generic Linux kernel headers
linux-headers-generic-lts-quantal - Generic Linux kernel headers
linux-headers-generic-lts-raring - Generic Linux kernel headers
linux-headers-generic-lts-saucy - Generic Linux kernel headers
linux-headers-generic-lts-trusty - Generic Linux kernel headers
linux-headers-lowlatency - lowlatency Linux kernel headers
linux-headers-server - Transitional package.
linux-headers-virtual - Transitional package.
linux-libc-dev - Linux Kernel Headers for development
linux-libc-dev-arm64-cross - Linux Kernel Headers for development (for cross-compiling)
linux-libc-dev-armel-cross - Linux Kernel Headers for development (for cross-compiling)
linux-libc-dev-armhf-cross - Linux Kernel Headers for development (for cross-compiling)
linux-libc-dev-powerpc-cross - Linux Kernel Headers for development (for cross-compiling)
linux-libc-dev-ppc64el-cross - Linux Kernel Headers for development (for cross-compiling)
linux-source-3.13.0 - Linux kernel source for version 3.13.0 with Ubuntu patches
linux-virtual - Minimal Generic Linux kernel and headers
linux-headers-3.13.0-24-generic - Linux kernel headers for version 3.13.0 on 32 bit x86 SMP
linux-headers-3.13.0-24-lowlatency - Linux kernel headers for version 3.13.0 on 32 bit x86 SMP
linux-headers-generic-pae - Transitional package
linux-headers-lowlatency-pae - Transitional package

It's not entirely clear why does it default to 3.13.0-24 even though uname -r returns 4.4.0-101-generic.
Any clues? I'm not really familiar with schroot.

[AlexAltea]

Nevermind, /etc/apt/sources.list wasn't being updated correctly.
Pull request on the way!

EDIT: Done at #142.