llvm-tblgen crashes in x86 windows debug builds

[yurybura]

Bugzilla Link	44945
Resolution	FIXED
Resolved on	Feb 25, 2020 06:24
Version	10.0
OS	Windows NT
Blocks	#43900
CC	@efriedma-quic,@zmodem,@RKSimon

Extended Description

I compile LLVM 10.0.0-rc2 but llvm-tblgen crashes with this stack dump:

0. Program arguments: D:\libs\vcpkg\buildtrees\llvm\x86-windows-dbg\bin\llvm-tblgen.exe -gen-dfa-packetizer -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/Hexagon -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/include -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/Hexagon/Hexagon.td --write-if-changed -o lib/Target/Hexagon/HexagonGenDFAPacketizer.inc -d lib/Target/Hexagon/HexagonGenDFAPacketizer.inc.d
   0x7B6AB020 (0x0178F020 0x00000001 0x6703EAAE 0x0178EFF0), _calloc_base() + 0x7A0 bytes(s)
   0x7B6ADB3C (0x0178F020 0x00000001 0x00000000 0x00000000), _free_dbg() + 0x7C bytes(s)
   0x7B6AE110 (0x0178F020 0x0000054B 0x0178EE10 0x0178EE04), free() + 0x20 bytes(s)
   0x004A879F (0x0178EE10 0x0178EFF0 0x00494AFE 0x0178F098), llvm::SmallVectorImpl::~SmallVectorImpl() + 0x2F bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\include\llvm\ADT\SmallVector.h, line 336 + 0x11 byte(s)
   0x004A86F0 (0x0178F098 0x0178F020 0x00000001 0x00000004), llvm::SmallVector<unsigned __int64,4>::~SmallVector<unsigned __int64,4>() + 0x30 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\include\llvm\ADT\SmallVector.h, line 844 + 0x8 byte(s)
   0x00494AFE (0x0178EE10 0x0178F13C 0xCCCCCCCC 0x00000001), llvm::DfaEmitter::visitDfaState() + 0x37E bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAEmitter.cpp, line 85 + 0x8 byte(s)
   0x00494729 (0x0178F38C 0x0000054B 0xCCCCCCCC 0xCCCCCCCC), llvm::DfaEmitter::constructDfa() + 0xB9 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAEmitter.cpp, line 94 + 0x3F byte(s)
   0x0049411A (0x0178F1DC 0x00000007 0x0178F970 0x0178F65C), llvm::DfaEmitter::emit() + 0x2A bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAEmitter.cpp, line 100
   0x004B884D (0x0178F970 0x04D40994 0x04FBBC30 0xCCCCCC00), anonymous namespace'::DFAPacketizerEmitter::emitForItineraries() + 0x5DD bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAPacketizerEmitter.cpp, line 336 0x004B8DA5 (0x0178F970 0x0178F7C4 0xCCCCCCCC 0x04620AC8), anonymous namespace'::DFAPacketizerEmitter::run() + 0x205 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAPacketizerEmitter.cpp, line 227 + 0x27 byte(s)
   0x004B8EF2 (0x0178FB34 0x0178F970 0x0178FB74 0x0178F7D4), llvm::EmitDFAPacketizer() + 0x52 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\DFAPacketizerEmitter.cpp, line 360 + 0x17 byte(s)
   0x0069E9E5 (0x0178F970 0x0178FB34 0x0178FBB8 0x0020BC76), `anonymous namespace'::LLVMTableGenMain() + 0x165 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\TableGen.cpp, line 179 + 0xD byte(s)
   0x00834E67 (0x01808BB4 0x0069E880 0x0020BC76 0xCCCCCCCC), llvm::TableGenMain() + 0x237 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\lib\TableGen\Main.cpp, line 108 + 0x10 byte(s)
   0x0069EEF7 (0x0000000E 0x01808B78 0x01809458 0x0000000E), main() + 0x87 bytes(s), D:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\TableGen\TableGen.cpp, line 263 + 0x19 byte(s)
   0x008A0E93 (0x37EB8D13 0x0020BC76 0x0020BC76 0x00C9A000), invoke_main() + 0x33 bytes(s), d:\agent_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 78 + 0x2D byte(s)
   0x008A0D17 (0x0178FC44 0x008A0F18 0x0178FC54 0x747C6359), __scrt_common_main_seh() + 0x157 bytes(s), d:\agent_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 288 + 0x5 byte(s)
   0x008A0BAD (0x0178FC54 0x747C6359 0x00C9A000 0x747C6340), __scrt_common_main() + 0xD bytes(s), d:\agent_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 331
   0x008A0F18 (0x00C9A000 0x747C6340 0x0178FCB0 0x76F27B74), mainCRTStartup() + 0x8 bytes(s), d:\agent_work\6\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp, line 17
   0x747C6359 (0x00C9A000 0x0C3C8E3F 0x00000000 0x00000000), BaseThreadInitThunk() + 0x19 bytes(s)
   0x76F27B74 (0xFFFFFFFF 0x76F48EFD 0x00000000 0x00000000), RtlGetAppContainerNamedObjectPath() + 0xE4 bytes(s)
   0x76F27B44 (0x0020BC76 0x00C9A000 0x00000000 0x00000000), RtlGetAppContainerNamedObjectPath() + 0xB4 bytes(s)

I'm using MSVC 19.24.28316.0 compiler.

[zmodem]

I'm using MSVC 19.24.28316.0 compiler.

MSVC 19.24 is known to miscompile llvm, and the cmake step should error about it, see 5f94022

Does it still reproduce if you use another MSVC version?

[yurybura]

I've tested with MSVC 19.16.27035.0. The same problem:

FAILED: lib/Target/AMDGPU/R600GenDFAPacketizer.inc
cmd.exe /C "cd /D D:\libs\vcpkg\buildtrees\llvm\x86-windows-msvc141-dbg && D:\libs\vcpkg\buildtrees\llvm\x86-windows-msvc141-dbg\bin\llvm-tblgen.exe -gen-dfa-packetizer -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/AMDGPU -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/include -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/AMDGPU/R600.td --write-if-changed -o lib/Target/AMDGPU/R600GenDFAPacketizer.inc -d lib/Target/AMDGPU/R600GenDFAPacketizer.inc.d"
Stack dump:
0. Program arguments: D:\libs\vcpkg\buildtrees\llvm\x86-windows-msvc141-dbg\bin\llvm-tblgen.exe -gen-dfa-packetizer -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/AMDGPU -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/include -I D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target D:/libs/vcpkg/buildtrees/llvm/src/c8aac023bc-a535e33c48/llvm/lib/Target/AMDGPU/R600.td --write-if-changed -o lib/Target/AMDGPU/R600GenDFAPacketizer.inc -d lib/Target/AMDGPU/R600GenDFAPacketizer.inc.d
0x7A1CB020 (0x01B8EDE0 0x00000001 0xF4D705F3 0x01B8EDB0), _calloc_base() + 0x7A0 bytes(s)
0x7A1CDB3C (0x01B8EDE0 0x00000001 0x00000000 0x00000000), _free_dbg() + 0x7C bytes(s)
0x7A1CE110 (0x01B8EDE0 0x00000006 0x01B8EBD0 0x01B8EBC4), free() + 0x20 bytes(s)
0x00918C5F (0x01B8EBD0 0x01B8EDB0 0x008FEE0D 0x01B8EE58), llvm::SmallVectorImpl::~SmallVectorImpl() + 0x2F bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\include\llvm\adt\smallvector.h, line 336 + 0x11 byte(s)
0x00918BB0 (0x01B8EE58 0x01B8EDE0 0x00000001 0x00000004), llvm::SmallVector<unsigned __int64,4>::~SmallVector<unsigned __int64,4>() + 0x30 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\include\llvm\adt\smallvector.h, line 844 + 0x8 byte(s)
0x008FEE0D (0x01B8EBD0 0x01B8EF00 0xCCCCCCCC 0x00000001), llvm::DfaEmitter::visitDfaState() + 0x37D bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfaemitter.cpp, line 85 + 0x8 byte(s)
0x008FEA39 (0x01B8F158 0x00000006 0xCCCCCCCC 0xCCCCCCCC), llvm::DfaEmitter::constructDfa() + 0xB9 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfaemitter.cpp, line 94 + 0x3F byte(s)
0x008FE42A (0x01B8EFA0 0x00000004 0x01B8F75C 0x01B8F448), llvm::DfaEmitter::emit() + 0x2A bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfaemitter.cpp, line 100
0x0093387D (0x01B8F75C 0x044D936C 0x0489FBE8 0xCCCCCC00), anonymous namespace'::DFAPacketizerEmitter::emitForItineraries() + 0x5DD bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfapacketizeremitter.cpp, line 336 0x00933DD5 (0x01B8F75C 0x01B8F5B0 0xCCCCCCCC 0x044F9E08), anonymous namespace'::DFAPacketizerEmitter::run() + 0x205 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfapacketizeremitter.cpp, line 227 + 0x27 byte(s)
0x00933F22 (0x01B8F920 0x01B8F75C 0x01B8F960 0x01B8F5C0), llvm::EmitDFAPacketizer() + 0x52 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\dfapacketizeremitter.cpp, line 360 + 0x17 byte(s)
0x00BA4485 (0x01B8F75C 0x01B8F920 0x01B8F9A4 0x005CD332), `anonymous namespace'::LLVMTableGenMain() + 0x165 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\tablegen.cpp, line 179 + 0xD byte(s)
0x00D6F427 (0x01D0900C 0x00BA4320 0x005CD332 0xCCCCCCCC), llvm::TableGenMain() + 0x237 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\lib\tablegen\main.cpp, line 108 + 0x10 byte(s)
0x00BA4997 (0x0000000E 0x01D08FD0 0x01D09E90 0x01B8FA14), main() + 0x87 bytes(s), d:\libs\vcpkg\buildtrees\llvm\src\c8aac023bc-a535e33c48\llvm\utils\tablegen\tablegen.cpp, line 263 + 0x19 byte(s)
0x00DE971E (0x7CBEAE6F 0x005CD332 0x005CD332 0x010D6000), invoke_main() + 0x1E bytes(s), d:\agent_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 78 + 0x1B byte(s)
0x00DE95B7 (0x01B8FA24 0x00DE9798 0x01B8FA34 0x747C6359), __scrt_common_main_seh() + 0x157 bytes(s), d:\agent_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 288 + 0x5 byte(s)
0x00DE944D (0x01B8FA34 0x747C6359 0x010D6000 0x747C6340), __scrt_common_main() + 0xD bytes(s), d:\agent_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, line 331
0x00DE9798 (0x010D6000 0x747C6340 0x01B8FA90 0x76F27B74), mainCRTStartup() + 0x8 bytes(s), d:\agent_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp, line 17
0x747C6359 (0x010D6000 0xBEA72DD8 0x00000000 0x00000000), BaseThreadInitThunk() + 0x19 bytes(s)
0x76F27B74 (0xFFFFFFFF 0x76F48EF7 0x00000000 0x00000000), RtlGetAppContainerNamedObjectPath() + 0xE4 bytes(s)
0x76F27B44 (0x005CD332 0x010D6000 0x00000000 0x00000000), RtlGetAppContainerNamedObjectPath() + 0xB4 bytes(s)

Information about compilers from CMake log:
-- The C compiler identification is MSVC 19.16.27035.0
-- The CXX compiler identification is MSVC 19.16.27035.0
-- The ASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x86/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x86/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x86/cl.exe -- works

[zmodem]

Can you share the full cmake invocation and build command?

[yurybura]

Build log
Please find attached build log for following build script:

@​echo off
setlocal enabledelayedexpansion
cd /D "%~dp0"
cd llvm10.0.0-rc2
mkdir build 2>nul
cd build

set VSDEVCMD=C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\Tools\VsDevCmd.bat

set "VSCMD_START_DIR=%CD%"
call "%VSDEVCMD%"

set CC=
set CXX=

cmake -G "Ninja" ^
-DCMAKE_BUILD_TYPE=Debug ^
-DPYTHON_EXECUTABLE="c:\Python37-32\python.exe" ^
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;lld" ..\llvm

echo.
echo Press any key to start build
pause

cmake --build . --config Debug --target install -- -v

[zmodem]

Thank you! Using VS 2017 (19.12.25834) for x86, I can reproduce the crash like this:

cmake -GNinja -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_PROJECTS="llvm" ..\llvm && ninja lib/Target/AMDGPU/R600GenDFAPacketizer.inc

(When targeting x64, it doesn't crash. It also doesn't crash for Release builds.)

I also tried 19.23.28105.4 (VS 2019) for x86, and 19.25.28610.4 (VS 2019 preview) and they both crash.

[zmodem]

Bisection points to this:

---

commit 12092a9
Author: James Molloy jmolloy@google.com
Date: Thu Oct 17 08:34:29 2019 +0000

[DFAPacketizer] Use DFAEmitter. NFC.

Summary:
This is a NFC change that removes the NFA->DFA construction and emission logic from DFAPacketizerEmitter and instead uses the generic DFAEmitter logic. This allows DFAPacketizer to use the Automaton class from Support and remove a bunch of logic there too.

After this patch, DFAPacketizer is mostly logic for grepping Itineraries and collecting functional units, with no state machine logic. This will allow us to modernize by removing the 16-functional-unit limit and supporting non-itinerary functional units. This is all for followup patches.

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68992

llvm-svn: 375086

---

[zmodem]

Should be fixed by edae4be (2905a48 on the 10.x branch).

Please re-open if there is still a problem.

[llvmbot]

mentioned in issue llvm/llvm-bugzilla-archive#49097