Skip to content

Conversation

gottesmm
Copy link
Contributor

@gottesmm gottesmm commented Oct 5, 2016

This group of commits does two things:

  1. It responds to some post commit review from @llvm-beanz on e4d8cc7. Specifically, we only need to stop -g from being put on the command line by cmake when we compile with LTO+RelWithDebInfo.
  2. In a separate commit, it adds -DNDEBUG to the overridden flags line for RelWithDebInfo to ensure that asserts are properly enabled given how LLVM implements LLVM_ENABLE_ASSERTIONS (it just rewrites the cflags flag to change the -D in -DNDEBUG to -U).

I am going to file a radar for a separate piece of work to fix LLVM to do something sane, namely, adding -DNDEBUG or -UNDEBUG to the end of command lines and perhaps create some sort of verification on the swift side that we are getting asserts where we think they are supposed to be.

This patch resolves some post-commit feedback from @llvmbeanz. Specifically, the
only time that we need to hard code these flags are when compiling with
RelWithDebInfo since we need extra control in that case to ensure that CMake
does not sneak in a -g flag on the command line. This is important to ensure
that we are compiling with -gline-tables-only instead of -g.

Given that it is a performance fix specific to compiling with LTO with debug
info, we should no perform such a change when compiling with:

1. MINRELSIZE since we do not ever perform that build.
2. RELEASE since it does not have debug info.
3. DEBUG since why would one perform an LTO build in DEBUG mode.

In terms of 3, I can potentially imagine someone trying to compile Swift in
Debug with LTO to try to debug an LTO bug, but I imagine such a test case to be
such a rare need that there is no point in trying to make it faster.
…S_RELWITHDEBINFO for swift and llvm.

Turns out -DLLVM_ENABLE_ASSERTIONS does not actually enable assertions in LLVM!
What it actually does is rewrite the -DNDEBUG flag to be -UNDEBUG. This means
that we always want to pass in -DNDEBUG to swift. There are some complications
though. This means that we can not compile the C++ part of the runtime with a
different assertions setting than the swift compiler/llvm. This is a trade off
that for now we are ok with to get the no-asserts build working.

As part of a separate change, we are going to make -DLLVM_ENABLE_ASSERTIONS more
robust by doing what swift does now, putting -DNDEBUG or -UNDEBUG after CMake's
-DNDEBUG so we get the correct behavior of disabling or enabling assertions.

rdar://28485525
@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 5, 2016

@swift-ci Please clean test

@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 5, 2016

I am also going to try a benchmarking test since that test is broken without this change.

@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 5, 2016

@swift-ci Please benchmark

@swift-ci
Copy link
Contributor

swift-ci commented Oct 5, 2016

Build comment file:

Build failed before running benchmark.


@shahmishal
Copy link
Member

Benchmark failed due to

Assertion failed: (isEncoding() && hasEncodingData()), function getEncodingData, file /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/llvm/include/llvm/Bitcode/BitCodes.h, line 113.
Stack dump:
0.  Program arguments: /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/bin/swift -frontend -c /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/stdlib/public/SwiftOnoneSupport/SwiftOnoneSupport.swift -disable-objc-attr-requires-foundation-module -target x86_64-apple-macosx10.9 -enable-objc-interop -sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -I /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64 -F /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/../../../Developer/Library/Frameworks -module-cache-path /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./module-cache -module-link-name swiftSwiftOnoneSupport -parse-stdlib -resource-dir /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift -sil-serialize-all -emit-module-doc-path /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64/SwiftOnoneSupport.swiftdoc -O -parse-as-library -module-name SwiftOnoneSupport -emit-module-path /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64/SwiftOnoneSupport.swiftmodule -o /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/stdlib/public/SwiftOnoneSupport/macosx/x86_64/SwiftOnoneSupport.o 
1.  While type-checking '_Prespecialize' at /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/stdlib/public/SwiftOnoneSupport/SwiftOnoneSupport.swift:16:1
2.  While resolving type Int at [/Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/stdlib/public/SwiftOnoneSupport/SwiftOnoneSupport.swift:147:47 - line:147:47] RangeText="I"
3.  While reading from /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64/Swift.swiftmodule
4.  While deserializing 'Int' (StructDecl #2186)in 'Swift'
[640/1065] Generating ../../../lib/swift/macosx/libswiftCore.dylib
[641/1065] Compiling /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/stdlib/internal/SwiftExperimental/macosx/x86_64/SwiftExperimental.o
FAILED: stdlib/internal/SwiftExperimental/macosx/x86_64/SwiftExperimental.o lib/swift/macosx/x86_64/SwiftExperimental.swiftmodule lib/swift/macosx/x86_64/SwiftExperimental.swiftdoc 
cd /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/stdlib/internal/SwiftExperimental && /Applications/CMake.app/Contents/bin/cmake -E make_directory /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/stdlib/internal/SwiftExperimental/macosx/x86_64 && /Applications/CMake.app/Contents/bin/cmake -E make_directory /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64 && /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/utils/line-directive /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/stdlib/internal/SwiftExperimental/SwiftExperimental.swift -- /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./bin/swiftc -c -sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -target x86_64-apple-macosx10.9 -resource-dir /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift -F /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/../../../Developer/Library/Frameworks -O -I /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64 -module-cache-path /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./module-cache -no-link-objc-runtime -Xfrontend -sil-serialize-all -parse-stdlib -module-link-name swiftSwiftExperimental -force-single-frontend-invocation -parse-as-library -emit-module -emit-module-path /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/./lib/swift/macosx/x86_64/SwiftExperimental.swiftmodule -o /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/swift-macosx-x86_64/stdlib/internal/SwiftExperimental/macosx/x86_64/SwiftExperimental.o /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/swift/stdlib/internal/SwiftExperimental/SwiftExperimental.swift

@jrose-apple
Copy link
Contributor

Hm. That is literally what this change is supposed to fix.

@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 5, 2016

I do not think the benchmark test did a clean build. @shahmishal Can you cause it to always do a clean build?

@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 5, 2016

Or maybe it is using the wrong commits. This is the build-script line:

  • env /usr/local/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -DCMAKE_MAKE_PROGRAM=/Users/buildnode/jenkins/workspace/swift-PR-osx-perf/Ninja-Release/ninja-build/ninja -DLLVM_ENABLE_ASSERTIONS=FALSE '-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;PowerPC;SystemZ' '-DCMAKE_C_FLAGS= -fno-stack-protector' '-DCMAKE_CXX_FLAGS= -fno-stack-protector' -DCMAKE_C_FLAGS_DEBUG= -DCMAKE_CXX_FLAGS_DEBUG= -DCMAKE_C_FLAGS_RELEASE=-O3 -DCMAKE_CXX_FLAGS_RELEASE=-O3 -DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -DCMAKE_C_FLAGS_MINSIZEREL=-Os -DCMAKE_CXX_FLAGS_MINSIZEREL=-Os -DCMAKE_BUILD_TYPE:STRING=Release -DLLVM_TOOL_SWIFT_BUILD:BOOL=NO -DLLVM_INCLUDE_DOCS:BOOL=TRUE -DLLVM_ENABLE_LTO:STRING= -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.9 -DCMAKE_OSX_SYSROOT:PATH=/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -DLLVM_ENABLE_LIBCXX:BOOL=TRUE -DCOMPILER_RT_ENABLE_IOS:BOOL=FALSE -DCOMPILER_RT_ENABLE_WATCHOS:BOOL=FALSE -DCOMPILER_RT_ENABLE_TVOS:BOOL=FALSE -DSANITIZER_MIN_OSX_VERSION=10.9 -DLLVM_ENABLE_MODULES:BOOL=TRUE -DLLVM_TOOL_COMPILER_RT_BUILD:BOOL=TRUE -DLLVM_BUILD_EXTERNAL_COMPILER_RT:BOOL=TRUE -DLLVM_HOST_TRIPLE:STRING=x86_64-apple-macosx10.9 -DLLVM_LIT_ARGS=-sv -DCMAKE_INSTALL_PREFIX:PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/ -DINTERNAL_INSTALL_PREFIX=local /Users/buildnode/jenkins/workspace/swift-PR-osx-perf/llvm

When I look in the tree with my sources, I am literally seeing no -DCMAKE_C_FLAGS_RELEASE. I do not know how that line is getting there.

@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 6, 2016

@shahmishal ping

@shahmishal
Copy link
Member

@gottesmm Benchmark job builds master branch first and compares the results with the PR branch. We are seeing this failure because master does not have this change yet, so I would recommend we merge this fix and test it on https://ci.swift.org/job/oss-swift_tools-R_stdlib-RD_test-simulator/

@gottesmm gottesmm merged commit 27fc4b0 into swiftlang:master Oct 6, 2016
@gottesmm gottesmm deleted the fix-no-asserts-build branch October 6, 2016 20:02
@gottesmm
Copy link
Contributor Author

gottesmm commented Oct 6, 2016

SGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants