Segfaults in llvm::CallGraphSCC::ReplaceNode() while building llvm trunk

[LebedevRI]

Bugzilla Link	34480
Resolution	FIXED
Resolved on	Feb 13, 2020 11:19
Version	trunk
OS	Linux
Blocks	#43900
Attachments	[https://user-images.githubusercontent.com/88600/143756273-01d53b56-2d30-4c1e-b906-0d1a77703517.gz)
CC	@efriedma-quic,@zmodem,@JVApen,@Keno,@vchuravy,@wjristow

Extended Description

$ ninja -j1
[1/885] Building CXX object tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o
FAILED: tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o
/usr/local/bin/clang++ -DGTEST_HAS_RTTI=0 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/lib/ARCMigrate -I/build/clang/lib/ARCMigrate -I/build/clang/include -Itools/clang/include -I/usr/include/libxml2 -Iinclude -I/build/llvm/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fcolor-diagnostics -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -fno-exceptions -fno-rtti -MD -MT tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o -MF tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o.d -o tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o -c /build/clang/lib/ARCMigrate/TransBlockObjCVariable.cpp
#​0 0x00007fa2ed4cc7da llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x8997da)
#​1 0x00007fa2ed4caac6 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x897ac6)
#​2 0x00007fa2ed4cabe3 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x897be3)
#​3 0x00007fa2f03fc0c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x110c0)
#​4 0x00007fa2ee1a868c llvm::CallGraphSCC::ReplaceNode(llvm::CallGraphNode*, llvm::CallGraphNode*) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x157568c)
#​5 0x00007fa2ee006412 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x13d3412)
#​6 0x00007fa2ee1aabb1 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x1577bb1)
#​7 0x00007fa2ed5a176f llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x96e76f)
#​8 0x0000556c41bc4c51 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_deletellvm::raw_pwrite_stream >) (/usr/lib/llvm-6.0/bin/clang+0x48dc51)
#​9 0x0000556c4210f022 (/usr/lib/llvm-6.0/bin/clang+0x9d8022)
#​10 0x0000556c4234d54a clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/llvm-6.0/bin/clang+0xc1654a)
#​11 0x0000556c4210e261 clang::CodeGenAction::ExecuteAction() (/usr/lib/llvm-6.0/bin/clang+0x9d7261)
#​12 0x0000556c41f61996 clang::FrontendAction::Execute() (/usr/lib/llvm-6.0/bin/clang+0x82a996)
#​13 0x0000556c41f2dcfc clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-6.0/bin/clang+0x7f6cfc)
#​14 0x0000556c42001843 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-6.0/bin/clang+0x8ca843)
#​15 0x0000556c41b75b10 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/lib/llvm-6.0/bin/clang+0x43eb10)
#​16 0x0000556c41b64329 main (/usr/lib/llvm-6.0/bin/clang+0x42d329)
#​17 0x00007fa2ebddb2e1 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e1)
#​18 0x0000556c41b7391a _start (/usr/lib/llvm-6.0/bin/clang+0x43c91a)
Stack dump:
0. Program arguments: /usr/lib/llvm-6.0/bin/clang -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name TransBlockObjCVariable.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -momit-leaf-frame-pointer -ffunction-sections -fdata-sections -coverage-notes-file /build/llvm-build-Clang-release/tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.gcno -resource-dir /usr/lib/llvm-6.0/lib/clang/6.0.0 -dependency-file tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o.d -sys-header-deps -MT tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o -D GTEST_HAS_RTTI=0 -D _GNU_SOURCE -D __STDC_CONSTANT_MACROS -D __STDC_FORMAT_MACROS -D __STDC_LIMIT_MACROS -I tools/clang/lib/ARCMigrate -I /build/clang/lib/ARCMigrate -I /build/clang/include -I tools/clang/include -I /usr/include/libxml2 -I include -I /build/llvm/include -D NDEBUG -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward -internal-isystem /usr/include/clang/6.0.0/include/ -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-6.0/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Werror=date-time -Werror=unguarded-availability-new -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Woverloaded-virtual -Wno-nested-anon-types -pedantic -std=c++11 -fdeprecated-macro -fdebug-compilation-dir /build/llvm-build-Clang-release -ferror-limit 19 -fmessage-length 0 -fvisibility-inlines-hidden -fno-rtti -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o tools/clang/lib/ARCMigrate/CMakeFiles/clangARCMigrate.dir/TransBlockObjCVariable.cpp.o -x c++ /build/clang/lib/ARCMigrate/TransBlockObjCVariable.cpp

1.  <eof> parser at end of file
   

2.  Per-module optimization passes
   

3.  Running pass 'CallGraph Pass Manager' on module '/build/clang/lib/ARCMigrate/TransBlockObjCVariable.cpp'.
   

clang: error: unable to execute command: Segmentation fault
clang: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 6.0.0-svn312520-1~exp1 (trunk)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
clang: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg:

---

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/TransBlockObjCVariable-5dd049.cpp
clang: note: diagnostic msg: /tmp/TransBlockObjCVariable-5dd049.sh
clang: note: diagnostic msg:

---

ninja: build stopped: subcommand failed.

[LebedevRI]

1. i can not attach the /tmp/TransBlockObjCVariable-5dd049.cpp because even after compression, it is just a bit bigger than the limit.
2. it is nondeterministic, now it no longer crashes.

[LebedevRI]

Not sure what the problem is.
maybe fixed.
can not be reliably reproduced.

[LebedevRI]

Nope, still seeing it.

[LebedevRI]

Apparently still happens with current trunk (6.0.0-svn315825-1~exp1)

FAILED: tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o
/usr/local/bin/clang++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/lib/Sema -I/build/clang/lib/Sema -I/build/clang/include -Itools/clang/include -I/usr/include/libxml2 -Iinclude -I/build/llvm/include -fPIC -fvisibility-inlines-hidden -Werror -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fcolor-diagnostics -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -UNDEBUG -fno-exceptions -fno-rtti -MD -MT tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o -MF tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o.d -o tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o -c /build/clang/lib/Sema/SemaTemplateVariadic.cpp
#​0 0x00007fa497192eca llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x8bfeca)
#​1 0x00007fa4971911e6 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x8be1e6)
#​2 0x00007fa497191303 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x8be303)
#​3 0x00007fa49a20c0c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x110c0)
#​4 0x00007fa497e9481c llvm::CallGraphSCC::ReplaceNode(llvm::CallGraphNode*, llvm::CallGraphNode*) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x15c181c)
#​5 0x00007fa497cef592 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x141c592)
#​6 0x00007fa497e96d41 (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x15c3d41)
#​7 0x00007fa49726ccaf llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/lib/llvm-6.0/bin/../lib/libLLVM-6.0.so.1+0x999caf)
#​8 0x00005634e894cc01 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_deletellvm::raw_pwrite_stream >) (/usr/lib/llvm-6.0/bin/clang+0x495c01)
#​9 0x00005634e8eb7a8a (/usr/lib/llvm-6.0/bin/clang+0xa00a8a)
#​10 0x00005634e913db7a clang::ParseAST(clang::Sema&, bool, bool) (/usr/lib/llvm-6.0/bin/clang+0xc86b7a)
#​11 0x00005634e8eb6c71 clang::CodeGenAction::ExecuteAction() (/usr/lib/llvm-6.0/bin/clang+0x9ffc71)
#​12 0x00005634e8cfe716 clang::FrontendAction::Execute() (/usr/lib/llvm-6.0/bin/clang+0x847716)
#​13 0x00005634e8cca64c clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-6.0/bin/clang+0x81364c)
#​14 0x00005634e8da5983 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-6.0/bin/clang+0x8ee983)
#​15 0x00005634e88fad70 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/lib/llvm-6.0/bin/clang+0x443d70)
#​16 0x00005634e88e90ec main (/usr/lib/llvm-6.0/bin/clang+0x4320ec)
#​17 0x00007fa495a7b2e1 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e1)
#​18 0x00005634e88f8b7a _start (/usr/lib/llvm-6.0/bin/clang+0x441b7a)
Stack dump:
0. Program arguments: /usr/lib/llvm-6.0/bin/clang -cc1 -triple x86_64-pc-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name SemaTemplateVariadic.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -momit-leaf-frame-pointer -ffunction-sections -fdata-sections -coverage-notes-file /build/llvm-build-Clang-release/tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.gcno -resource-dir /usr/lib/llvm-6.0/lib/clang/6.0.0 -dependency-file tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o.d -sys-header-deps -MT tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o -D GTEST_HAS_RTTI=0 -D _DEBUG -D _GNU_SOURCE -D __STDC_CONSTANT_MACROS -D __STDC_FORMAT_MACROS -D __STDC_LIMIT_MACROS -I tools/clang/lib/Sema -I /build/clang/lib/Sema -I /build/clang/include -I tools/clang/include -I /usr/include/libxml2 -I include -I /build/llvm/include -U NDEBUG -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.1/../../../../include/c++/7.2.1 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.1/../../../../include/x86_64-linux-gnu/c++/7.2.1 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.1/../../../../include/x86_64-linux-gnu/c++/7.2.1 -internal-isystem /usr/lib64/gcc/x86_64-linux-gnu/7.2.1/../../../../include/c++/7.2.1/backward -internal-isystem /usr/include/clang/6.0.0/include/ -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-6.0/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Werror -Werror=date-time -Werror=unguarded-availability-new -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Woverloaded-virtual -Wno-nested-anon-types -pedantic -std=c++11 -fdeprecated-macro -fdebug-compilation-dir /build/llvm-build-Clang-release -ferror-limit 19 -fmessage-length 0 -fvisibility-inlines-hidden -fno-rtti -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o tools/clang/lib/Sema/CMakeFiles/clangSema.dir/SemaTemplateVariadic.cpp.o -x c++ /build/clang/lib/Sema/SemaTemplateVariadic.cpp

1. parser at end of file
2. Per-module optimization passes
3. Running pass 'CallGraph Pass Manager' on module '/build/clang/lib/Sema/SemaTemplateVariadic.cpp'.
   clang: error: unable to execute command: Segmentation fault
   clang: error: clang frontend command failed due to signal (use -v to see invocation)
   clang version 6.0.0-svn315825-1~exp1 (trunk)
   Target: x86_64-pc-linux-gnu
   Thread model: posix
   InstalledDir: /usr/local/bin
   clang: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
   clang: note: diagnostic msg:

---

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/SemaTemplateVariadic-348750.cpp
clang: note: diagnostic msg: /tmp/SemaTemplateVariadic-348750.sh
clang: note: diagnostic msg:

---

Still unreproducible.

[efriedma-quic]

Still unreproducible.

By unreproducible, do you mean you can't reproduce it at all, or just that it doesn't reproduce reliably? In some cases of intermittent crashes, I've managed to reproduce the issue just by invoking the compile command 100 times in a row, or something like that.

[LebedevRI]

Still unreproducible.

By unreproducible, do you mean you can't reproduce it at all, or just that
it doesn't reproduce reliably? In some cases of intermittent crashes, I've
managed to reproduce the issue just by invoking the compile command 100
times in a row, or something like that.

I reliably hit it several times when building LLVM, but when i try to run the script that clang writes after the crash, it does not reproduce the crash.
Yes, if i run it several times, it does crash sometimes.

So i can not reproduce it reliably.

[zmodem]

Can you upload the preprocessed source to Dropbox/Drive/something else? Without that, there's not much we can do.

[LebedevRI]

https://www.dropbox.com/s/c5pwj0q5h74png3/bug34480.tar.xz?dl=0

[zmodem]

I tried 100 compilations of the PaddingChecker-33e85e.cpp file with Clang from the 6.0 branch, built as a position-independent executable, but couldn't reproduce the crash.

Can you share the build flags used for the clang that's crashing?

[LebedevRI]

I tried 100 compilations of the PaddingChecker-33e85e.cpp file with Clang
from the 6.0 branch, built as a position-independent executable, but
couldn't reproduce the crash.

That's the general idea :(
It happens several times when building llvm, but it succeeds eventually.

Can you share the build flags used for the clang that's crashing?

It's just clang-7 for debian sid from http://apt.llvm.org/

[LebedevRI]

1. still happens
2. happens in release build mode (-O3 -DNDEBUG -g0), but not with -O0

[vchuravy]

Happens reliably for me on version 6 to 9, but only on AlpineLinux with musl, while crosscompiling LLVM to other platforms

• https://gist.githubusercontent.com/vchuravy/4147c7f28685461b352f48549ad8412f/raw/ab837858dcb29855b552463c9394cf9d8a530432/SemaTemplate-b62941.cpp
• https://gist.githubusercontent.com/vchuravy/4147c7f28685461b352f48549ad8412f/raw/ab837858dcb29855b552463c9394cf9d8a530432/SemaTemplate-b62941.sh

It took me a while, but I was able to catch it in rr and watch the pointer that seqfaults. For me it occurs during ReplaceNode on line

llvm-project/llvm/lib/Analysis/CallGraphSCCPass.cpp

Line 560 in 19a71f6

CGI->ReplaceNode(Old, New);

Which bogies down to scc_iterator::ReplaceNode and subsequently to DenseMap::grow

llvm-project/llvm/include/llvm/ADT/DenseMap.h

Lines 797 to 812 in 19a71f6

	void grow(unsigned AtLeast) {
	unsigned OldNumBuckets = NumBuckets;
	BucketT *OldBuckets = Buckets;
	allocateBuckets(std::max<unsigned>(64, static_cast<unsigned>(NextPowerOf2(AtLeast-1))));
	assert(Buckets);
	if (!OldBuckets) {
	this->BaseT::initEmpty();
	return;
	}
	this->moveFromOldBuckets(OldBuckets, OldBuckets+OldNumBuckets);
	// Free the old table.
	operator delete(OldBuckets);
	}

My current understanding is that clang then tries to access the pointer, that was just deleted in DenseMap::grow.

[Keno]

Candidate fix: https://reviews.llvm.org/D72529.

[wjristow]

Candidate fix: https://reviews.llvm.org/D72529.

Independently, we came across a problem that may be the same issue, and
proposed a similar fix: https://reviews.llvm.org/D72469

This fix was committed today in f7e9f4f.
However, see the above candidate fix, and also:
https://reviews.llvm.org/D72550

for more possible pending work.

[zmodem]

Candidate fix: https://reviews.llvm.org/D72529.

Independently, we came across a problem that may be the same issue, and
proposed a similar fix: https://reviews.llvm.org/D72469

This fix was committed today in f7e9f4f.
However, see the above candidate fix, and also:
https://reviews.llvm.org/D72550

for more possible pending work.

So should I merge f7e9f4f to the branch in the meantime, or wait for the possibly pending work?