Add necessary patches for LLVM 3.7(.1)

[Keno]

And make ORC JIT the default for those LLVM versions. This does
not activate the new LLVM versions by default yet, to give time
to update everything on CI, etc., but doing the actual activation
is a one line change after this.

@staticfloat @tkelman Once this is done, how do we go about
preparing our CI infrastructure for the switch?

Close #14466

[tkelman]

3.7.1 was already tagged but isn't up at http://llvm.org/releases/3.7.1 yet, so I think we should skip the 3.7.0 checksums. I have a test branch where now we can actually just build llvm from source and cache the result on Linux Docker Travis. Mac will need a homebrew bottle.

edit: and AppVeyor will need me to build win32 and win64 binaries and put them up on bintray.

[vtjnash]

why is this in the diff? we don't even use landingpads

[Keno]

Miscompiles Cxx otherwise.

[tkelman]

Need to ping llvm-dev about putting up source tarballs. I don't get what in their release process takes so long after tagging (http://lists.llvm.org/pipermail/llvm-dev/2015-December/093179.html was 10 days ago).

[Keno]

Not sure what's going on with 3.7.1. Let's just go with 3.7.0 for now and switch later.

[tkelman]

It's more of a time investment for me to rebuild binaries across 2-3 different platforms than it is to get the llvm release manager to finish the job here. I'm not subscribed to llvm-dev, can we ask what the holdup is here?

[hayd]

This thread was started yesterday: http://lists.llvm.org/pipermail/llvm-dev/2015-December/093572.html

[Keno]

It was suggested that we switch LLVM to git-external to be able to pull in the release branch even though the tarballs aren't up yet. What would we do about the patches? Those are based on the LLVM version which we wouldn't know any more.

[ViralBShah]

Can't we just create a source tarball of 3.7.1 on our own and host it on s3? If llvm.org releases a tarball, we can switch to it.

[Keno]

Yeah, I'm in favor of that solution as well. Let's just do the switch. It's up to @tkelman though. I'm happy to do whatever is required to finally get this done.

[tkelman]

With a git external we'd be using sha instead of version number, so the patch application should be tweaked to also check for that.

Using a different url for one specific version is pretty messy, but maybe the fastest way to get something temporary in place.

[ViralBShah]

Why is it so messy? It just changes a URL for download that can be reverted when the official tarball is made available. Seems like the smallest change to get this going.

[Keno]

Ok, S3 mirror is up. Should be good to go. @tkelman shall we merge this, so you can prepare the buildbots and then we'll do the actual switch in a separate PR?

[tkelman]

I think you need to fix the permissions so that bucket is readable. What are the chances your tarballs will end up identical to the real ones once they go up?

[Keno]

I thought the bucket was readable. I tested with this branch, but that was from within AWS, so maybe that was the difference.

[Keno]

Ok, try again. The files were public, but I hadn't made the bucket listable.

[Keno]

Oh, and I think the content is the same (this is the final tags), but I don't know how the release tarballs are built so I'm pretty sure they won't be bit-identical.

[tkelman]

we should make the directory name inside the tarball consistent with upstream releases, so llvm-3.7.1.src

[tkelman]

The release win32 build, but not the debug build, segfaults at the start of bootstrap on this branch with llvm 3.7.1.

edit: and the debug build segfaults during second stage inference.ji at range.jl, line 718

[Keno]

win64 is fine?

[tkelman]

will let you know when it finishes

[Keno]

Also, what gcc version? Anything > 4.9 miscompiles llvm.

[tkelman]

4.9.2. That's a bit of a problem, given clang as a mingw-w64 cross compiler isn't especially easy to set up or use. Can we patch llvm or add a gcc flag to work around it? Fedora and Arch are also on gcc 5 by now afaik.

[Keno]

Yes, you can use -fno-ipa-ra.

[tkelman]

What would be the downside of adding that flag anywhere we're using gcc? Is it needed just for llvm, or libjulia, or (somehow) all deps?

[Keno]

Probably fine to add it unconditionally when targeting mingw. Do we have a way to detect compiler version?

[tkelman]

We need > 4.7 due to c++11 in new llvm anyway. Looks like there's a proposed patch for the gcc issue at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66655, just need to build mingw to test it.

win64 actually looks okay, tests are passing and performance looks normal. getting <invalid> in backtraces if I try to start with --precompiled=yes though.

[Keno]

Ok, I'll take a look at the win32 failure.

[tkelman]

sorry wrong button. given a9b3015#diff-9, may want to manually define USE_ORCJIT if we know our homebrew llvm is patched

[Keno]

I'm suspecting the remaining win32 failure to be another miscompilation that's not fixed by the indicated workaround. I'm actually quite speechless as to how bad gcc has gotten in version 5. I think we should move ahead anyway, and just tell people not to use -O on win32 for now. Maybe we can switch our build system to clang later.

[tkelman]

4.9.2 from cygwin actually looks like it's working now. Which toolchain were you using for win32, opensuse or the one from msys2 pacman? What kind of failure are you seeing?

I used the opensuse build service to apply the suggested patch from the gcc bug thread and it fixed the test case, so hopefully that may get applied before gcc 6 and/or 5.4. Could also direct the get_toolchain.sh script at my repository to see if it helps. If it does, I could submit the patch to the opensuse cross-compiler maintainers.

[Keno]

msys2 pacman gcc 5.3.0. I'm seeing LLVM being miscompiled. Easily checked by running the llvm tests to see if they're failing.

[tkelman]

"our build system" does not use msys2 pacman's gcc. On win32 that's also dwarf exception handling, so I'm surprised it built at all. I do still see

     * subarray             LLVM ERROR: Unable to allocate section memory!
Makefile:12: recipe for target 'all' failed
make[1]: *** [all] Error 1
Makefile:573: recipe for target 'testall1' failed
make: *** [testall1] Error 2

which looks similar to #11083.

LLVM tests don't run particularly easily here, make -C deps check-llvm is a no-op in cross compiles.

[Keno]

I realized I was wrong about my gcc. It's the one we get using contrib/windows/get_toolchain.sh, i.e. the opensuse one. It seems not only does gcc 5.3 miscompile LLVM, but it's libstdc++ is also broken (hopefully to be fixed by 5.4 though), so I can't even compile with clang at the moment.

[Keno]

@tkelman I can't reproduce the subarray error.

[tkelman]

try running tests in a single process with make testall1

[ViralBShah]

@staticfloat @tkelman This seems to require diving into a fair amount of homebrew, which I am happy to do, but won't be able to for all of the coming week.

[Keno]

I can believe that make testall1 would run out of memory. However, I'm not entirely convinced there's too much to do about that. One thing we could do is leave pages as RWX such that we can avoid internal fragmentation, but having RWX pages is not really best practice. I am a little surprised that virtual memory usage is still so high though. Not entirely sure why that is.

[tkelman]

It isn't broken now and it is broken under llvm 3.7, so it's definitely a regression.

[Keno]

Ok, I can try to do to improve the situation, but I don't think we should hold up the transition on other platforms because of it.

[tkelman]

squash out commits that fail to build, fix the folder name in the tarballs (and adjust the checksums accordingly) then I think we can merge this and move to the next step.

[Keno]

Done

[tkelman]

aaand they finally put up the real release today at http://llvm.org/releases/3.7.1. Ping me when homebrew stuff is getting ready and I'll do a PR to switch the CI on all 5 main platforms.

edit: https://github.com/Homebrew/homebrew-versions/blob/master/llvm37.rb is probably the best place to start, then add our patches