Skip to content

CFI sanitizer errors on windows #57143

New issue
New issue
Open
Open
CFI sanitizer errors on windows#57143
Labels
compiler-rt:cfiControl Flow Integrityplatform:windows
@christianbrugger

Description

@christianbrugger
christianbrugger
opened on Aug 14, 2022

I have been trying out the examples from the clang-cfi-showcase with Clang 14. While on Ubuntu it mostly seems to work, on Windows I have many issues:

  1. Compiler crashes during compilation
  2. The violation is not detected
  3. False positive in ostream
  4. Forcing visibility hidden doesn't seem to be required

Test system:

Windows 10, 64 bit
Clang Version 14.0.6
Microsoft Visual Build Tools 2022 17.3.0 14.33.31629
Windows Kit 10.0.19041.0

Here is how I compiled the examples with clang++:

clang++ -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp

And with clang-cl:

clang-cl -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp

Each example should compile, but generate a CFI violation. When false positives where found, I fixed them and ran the example again. Here is the summary:

Example clang++ clang-cl
cfi_cast_strict.cpp false positive
not-found		 not-found
cfi_derived_cast.cpp compile crash compile crash
cfi_icall.c found found
cfi_nvcall.cpp compile crash compile crash
cfi_unrelated_cast.cpp found found
cfi_vcall.cpp false positive
found		 found

In bold are the ones that are problematic. Lets go through the failure cases one by one:

1. Compiler Crashes

The compiler crashes for cfi_derived_cast.cpp and the cfi_nvcall.cpp example. Here is the output the first with clang++. I added an additional -v flag:

> clang++ -v -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld cfi_derived_cast.cpp
clang version 14.0.6
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
 "C:\\Program Files\\LLVM\\bin\\clang++.exe" -cc1 -triple x86_64-pc-windows-msvc19.33.31629 -emit-llvm-bc -flto=full -flto-unit -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name cfi_derived_cast.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -v "-fcoverage-compilation-dir=C:\\Users\\brugg\\Desktop\\cfi_test" -resource-dir "C:\\Program Files\\LLVM\\lib\\clang\\14.0.6" -internal-isystem "C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\include" -internal-isystem "C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\cppwinrt" -fdeprecated-macro "-fdebug-compilation-dir=C:\\Users\\brugg\\Desktop\\cfi_test" -ferror-limit 19 -fmessage-length=114 --dependent-lib=clang_rt.ubsan_standalone-x86_64.lib --dependent-lib=clang_rt.ubsan_standalone_cxx-x86_64.lib -fsanitize=cfi-derived-cast,cfi-icall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall "-fsanitize-system-ignorelist=C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\share\\cfi_ignorelist.txt" -fsanitize-cfi-canonical-jump-tables -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.33.31629 -std=c++14 -fdelayed-template-parsing -fcxx-exceptions -fexceptions -fcolor-diagnostics -fsplit-lto-unit -faddrsig -o "C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-2067bb.o" -x c++ cfi_derived_cast.cpp
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-pc-windows-msvc
ignoring nonexistent directory "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\atlmfc\include"
#include "..." search starts here:
#include <...> search starts here:
 C:\Program Files\LLVM\lib\clang\14.0.6\include
 C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\include
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\shared
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\um
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\winrt
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\cppwinrt
End of search list.
 "C:\\Program Files\\LLVM\\bin\\lld-link" -out:a.exe -defaultlib:libcmt -defaultlib:oldnames "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\um\\x64" "-libpath:C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\lib\\windows" -nologo "C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-2067bb.o"
LLVM ERROR: Associative COMDAT symbol '?hex@std@@YAAEAVios_base@1@AEAV21@@Z' is not a key for its COMDAT.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: "C:\\Program Files\\LLVM\\bin\\lld-link" -out:a.exe -defaultlib:libcmt -defaultlib:oldnames "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\um\\x64" "-libpath:C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\lib\\windows" -nologo C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-2067bb.o
1.      Running pass 'Function Pass Manager' on module 'ld-temp.o'.
2.      Running pass 'X86 Assembly Printer' on function '@"?hex@std@@YAAEAVios_base@1@AEAV21@@Z.cfi"'
 #0 0x00007ff697bfaa56 C:\Program Files\LLVM\bin\lld-link.exe 0x3aa56 (C:\Program Files\LLVM\bin\lld-link.exe+0x3aa56)
 #1 0x00007ff697bfaa56
 #2 0x00007ff697bfaa56 (C:\Program Files\LLVM\bin\lld-link.exe+0x3aa56)
 #3 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x45d42 C:\Program Files\LLVM\bin\lld-link.exe 0x183fd66
 #4 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x183ff4a C:\Program Files\LLVM\bin\lld-link.exe 0x1bf4d50
 #5 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1bf4dbc C:\Program Files\LLVM\bin\lld-link.exe 0x154345f
 #6 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x15449f0 C:\Program Files\LLVM\bin\lld-link.exe 0x117fcfd
 #7 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x18b846d C:\Program Files\LLVM\bin\lld-link.exe 0x2720b0f
 #8 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x2726b73 C:\Program Files\LLVM\bin\lld-link.exe 0x2721110
 #9 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1828819 C:\Program Files\LLVM\bin\lld-link.exe 0x18276e7
#10 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x181c496 C:\Program Files\LLVM\bin\lld-link.exe 0x181bb2f
#11 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0xe95ec C:\Program Files\LLVM\bin\lld-link.exe 0xbcba7
#12 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x95595 C:\Program Files\LLVM\bin\lld-link.exe 0x8e35c
#13 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1c6a C:\Program Files\LLVM\bin\lld-link.exe 0x1464
#14 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x28b5e2c (C:\Windows\System32\ucrtbase.dll+0x71881)
#15 0x00007fff046a1881
#16 0x00007fff046a1881 (C:\Windows\System32\ucrtbase.dll+0x71881)
0x00007FF697BFAA56 (0x000017C8DB958623 0x0000000000000024 0x0000000000000016 0x00007FF697BFAA50)
0x00007FFF046A1881 (0x00000000FD58AB01 0x0000005100000000 0x0000000000000000 0x00000051FD58AAC0), raise() + 0x1E1 bytes(s)
0x00007FFF046A2851 (0x0000000000000003 0x0000000000000003 0x00000051FD58AB40 0x00007FF697BF346F), abort() + 0x31 bytes(s)
0x00007FF697C05D42 (0xCE7A6101634B54F3 0x0000CA0F153C5BC6 0x0000015CBA8368D8 0x00007FF697C65385)
0x00007FF6993FFD66 (0x0000015CB7725AC8 0x0000CA0F153C5C36 0x0000015CBA8368D8 0x00007FF69A298EB9)
0x00007FF6993FFF4A (0x0000000300000020 0x0000015CBAB8BF90 0x0000000000000001 0x00000000D00010C0)
0x00007FF6997B4D50 (0x0000CA0F153C5F46 0x0000000000000000 0x00007FF69C31D430 0x0000000000000003)
0x00007FF6997B4DBC (0x0000015CB49D0000 0x00007FF69A475993 0x00000051FD58B3A8 0x0000015CBAD49500)
0x00007FF69910345F (0x00000051FD58B2A0 0x0000000800000000 0x0000CA0F153C4276 0x0000015CBAAAC730)
0x00007FF6991049F0 (0x00007FF69C31D430 0x0000CA0F153C34C6 0x0000000000000000 0x0000015CBA8368D8)
0x00007FF698D3FCFD (0x0000015CBA8368D8 0x0000015CBA7E4450 0x0000000000000000 0x0000015CBAD8A7B0)
0x00007FF69947846D (0x00000051FD58CA50 0x00007FF600000000 0x0000015CBA7D3888 0x0000000000000000)
0x00007FF69A2E0B0F (0x0000CA0F153C3AD6 0x0000015CB773EDF0 0x0000000000000001 0x0000CA0F153C39F6)
0x00007FF69A2E6B73 (0x000000000000002F 0x0000CA0F153C3946 0x0000015CBAA54430 0x0000000000000000)
0x00007FF69A2E1110 (0x00007FF6990B8DC0 0x0000005100000000 0x0000015CBAA8E850 0x0000000000000000)
0x00007FF6993E8819 (0x00000051FD58D5A0 0x0000015CBAAF5D4A 0x0000015CBAAF5D40 0x00007FF69A2E9264)
0x00007FF6993E76E7 (0x0000015CB7217B50 0x00007FF60000000F 0x0000015CBA835E28 0x0000015CB7663950)
0x00007FF6993DC496 (0x00000000000E006C 0x0000000000000014 0x0000015CBA879800 0x0000015CB49D0000)
0x00007FF6993DBB2F (0x0000000000000013 0x0000000000000000 0x0000015CB4ADAFA0 0x00000000001A0003)
0x00007FF697CA95EC (0x0000000000000000 0x00000000000013B8 0x0000000000000000 0x0000000000000000)
0x00007FF697C7CBA7 (0x01D8AFF181F462B3 0x0000000000000056 0x0000000000000000 0x0000000000000008)
0x00007FF697C55595 (0x00000000FFFFFFFE 0x0000000000000004 0x0000000000000003 0x00007FF697BD75E0)
0x00007FF697C4E35C (0x0000004C004C0044 0x00000051FD58ECC0 0x00000051FD58ECF0 0x00007FFF06C16630)
0x00007FF697BC1C6A (0x0000000000000000 0x00007FFF046426EE 0x00007FF69C4F0C58 0x00000051FD58F710)
0x00007FF697BC1464 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FF69A475E2C (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FFF05937034 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), BaseThreadInitThunk() + 0x14 bytes(s)
0x00007FFF06B02651 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), RtlUserThreadStart() + 0x21 bytes(s)
clang++: error: linker command failed due to signal (use -v to see invocation)

And here with clang-cl:

> clang-cl -v -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld cfi_derived_cast.cpp
clang version 14.0.6
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
 "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" -cc1 -triple x86_64-pc-windows-msvc19.33.31629 -emit-llvm-bc -flto=full -flto-unit -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name cfi_derived_cast.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -relaxed-aliasing -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -mllvm -x86-asm-syntax=intel -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fms-volatile -fdiagnostics-format msvc -v "-fcoverage-compilation-dir=C:\\Users\\brugg\\Desktop\\cfi_test" -resource-dir "C:\\Program Files\\LLVM\\lib\\clang\\14.0.6" -internal-isystem "C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\include" -internal-isystem "C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\include" -internal-isystem "C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.19041.0\\cppwinrt" -fdeprecated-macro "-fdebug-compilation-dir=C:\\Users\\brugg\\Desktop\\cfi_test" -ferror-limit 19 -fmessage-length=114 --dependent-lib=clang_rt.ubsan_standalone-x86_64.lib --dependent-lib=clang_rt.ubsan_standalone_cxx-x86_64.lib -fsanitize=cfi-derived-cast,cfi-icall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall "-fsanitize-system-ignorelist=C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\share\\cfi_ignorelist.txt" -fsanitize-cfi-canonical-jump-tables -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.33.31629 -std=c++14 -fdelayed-template-parsing -fcolor-diagnostics -fsplit-lto-unit -faddrsig -o "C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-de4461.obj" -x c++ cfi_derived_cast.cpp
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-pc-windows-msvc
ignoring nonexistent directory "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\atlmfc\include"
#include "..." search starts here:
#include <...> search starts here:
 C:\Program Files\LLVM\lib\clang\14.0.6\include
 C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\include
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\shared
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\um
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\winrt
 C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\cppwinrt
End of search list.
 "C:\\Program Files\\LLVM\\bin\\lld-link" -out:cfi_derived_cast.exe "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\um\\x64" "-libpath:C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\lib\\windows" -nologo "C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-de4461.obj"
LLVM ERROR: Associative COMDAT symbol '?hex@std@@YAAEAVios_base@1@AEAV21@@Z' is not a key for its COMDAT.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: "C:\\Program Files\\LLVM\\bin\\lld-link" -out:cfi_derived_cast.exe "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\atlmfc\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\um\\x64" "-libpath:C:\\Program Files\\LLVM\\lib\\clang\\14.0.6\\lib\\windows" -nologo C:\\Users\\brugg\\AppData\\Local\\Temp\\cfi_derived_cast-de4461.obj
1.      Running pass 'Function Pass Manager' on module 'ld-temp.o'.
2.      Running pass 'X86 Assembly Printer' on function '@"?hex@std@@YAAEAVios_base@1@AEAV21@@Z.cfi"'
 #0 0x00007ff697bfaa56 C:\Program Files\LLVM\bin\lld-link.exe 0x3aa56 (C:\Program Files\LLVM\bin\lld-link.exe+0x3aa56)
 #1 0x00007ff697bfaa56
 #2 0x00007ff697bfaa56 (C:\Program Files\LLVM\bin\lld-link.exe+0x3aa56)
 #3 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x45d42 C:\Program Files\LLVM\bin\lld-link.exe 0x183fd66
 #4 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x183ff4a C:\Program Files\LLVM\bin\lld-link.exe 0x1bf4d50
 #5 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1bf4dbc C:\Program Files\LLVM\bin\lld-link.exe 0x154345f
 #6 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x15449f0 C:\Program Files\LLVM\bin\lld-link.exe 0x117fcfd
 #7 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x18b846d C:\Program Files\LLVM\bin\lld-link.exe 0x2720b0f
 #8 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x2726b73 C:\Program Files\LLVM\bin\lld-link.exe 0x2721110
 #9 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1828819 C:\Program Files\LLVM\bin\lld-link.exe 0x18276e7
#10 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x181c496 C:\Program Files\LLVM\bin\lld-link.exe 0x181bb2f
#11 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0xe95ec C:\Program Files\LLVM\bin\lld-link.exe 0xbcba7
#12 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x95595 C:\Program Files\LLVM\bin\lld-link.exe 0x8e35c
#13 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x1c6a C:\Program Files\LLVM\bin\lld-link.exe 0x1464
#14 0x00007fff046a1881 C:\Program Files\LLVM\bin\lld-link.exe 0x28b5e2c (C:\Windows\System32\ucrtbase.dll+0x71881)
#15 0x00007fff046a1881
#16 0x00007fff046a1881 (C:\Windows\System32\ucrtbase.dll+0x71881)
0x00007FF697BFAA56 (0x0000857144A5A3FC 0x0000000000000024 0x0000000000000016 0x00007FF697BFAA50)
0x00007FFF046A1881 (0x000000001798AD01 0x0000004600000000 0x0000000000000000 0x000000461798ACB0), raise() + 0x1E1 bytes(s)
0x00007FFF046A2851 (0x0000000000000003 0x0000000000000003 0x000000461798AD30 0x00007FF697BF346F), abort() + 0x31 bytes(s)
0x00007FF697C05D42 (0xCE7A6101634B54F3 0x0000A502520194A3 0x0000018D98044AD8 0x00007FF697C65385)
0x00007FF6993FFD66 (0x0000018D94EF43D8 0x0000A50252019773 0x0000018D98044AD8 0x00007FF69A298EB9)
0x00007FF6993FFF4A (0x0000000300000020 0x0000000000000192 0x0000000000000001 0x0000000051000051)
0x00007FF6997B4D50 (0x0000A50252018823 0x0000000000000000 0x00007FF69C31D430 0x0000000000000003)
0x00007FF6997B4DBC (0x0000018D92250000 0x00007FF69A475993 0x0000000000001400 0x0000018D922527C0)
0x00007FF69910345F (0x000000461798B490 0x0000000800000000 0x0000000000000220 0x0000018D922F0000)
0x00007FF6991049F0 (0x00007FF69C31D430 0x0000A5025201FFA3 0x0000000000000000 0x0000018D98044AD8)
0x00007FF698D3FCFD (0x0000000000000062 0x0000018D9801D3E0 0x0000018D922F6874 0x00000000FD0000FD)
0x00007FF69947846D (0x000000461798CC40 0x00007FF600000000 0x0000018D94EC7968 0x0000000000000000)
0x00007FF69A2E0B0F (0x0000A5025201F553 0x0000018D94E79870 0x0000000000000001 0x0000A5025201F2B3)
0x00007FF69A2E6B73 (0x000000000000002F 0x0000A5025201F223 0x0000018D98296850 0x0000000000000000)
0x00007FF69A2E1110 (0x00007FF6990B8DC0 0x0000004600000000 0x0000018D98209520 0x0000000000000000)
0x00007FF6993E8819 (0x000000461798D790 0x0000018D98183B4A 0x0000018D98183B40 0x00007FF69A2E9264)
0x00007FF6993E76E7 (0x0000018D94EEDED0 0x00007FF600000002 0x0000018D9805E808 0x0000018D97FA0080)
0x00007FF6993DC496 (0x0000000000620051 0x0000000000000052 0x0000018D9802E130 0x0000018D92250000)
0x00007FF6993DBB2F (0x000000000000000F 0x0000000000000000 0x0000018D922F86C0 0x00000000000F02C0)
0x00007FF697CA95EC (0x0000000000000000 0x00000000000013B8 0x0000000000000000 0x0000000000000000)
0x00007FF697C7CBA7 (0x01D8AFF1A44E3D26 0x0000000000000056 0x0000000000000000 0x0000000000000008)
0x00007FF697C55595 (0x00000000FFFFFFFE 0x0000000000000004 0x0000000000000003 0x00007FF697BD75E0)
0x00007FF697C4E35C (0x0000004C004C0044 0x000000461798EEB0 0x000000461798EEE0 0x00007FFF06C16630)
0x00007FF697BC1C6A (0x0000000000000000 0x00007FFF046426EE 0x00007FF69C4F0C58 0x000000461798F900)
0x00007FF697BC1464 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FF69A475E2C (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000)
0x00007FFF05937034 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), BaseThreadInitThunk() + 0x14 bytes(s)
0x00007FFF06B02651 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), RtlUserThreadStart() + 0x21 bytes(s)
clang-cl: error: linker command failed due to signal (use -v to see invocation)

It has to do with std::cout <<. So if we convert those to printf statements, the examples compile and the violations are found. Still this is far from ideal. The example compile normally without fsanitize=cfi, so there is nothing wrong with the examples, besides the violations.

2. Violations are not detected

For example cfi_cast_strict.cpp the violation is not detected when compiling it with clang-cl as above. Here is the application output:

> clang-cl -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp
> example.exe
Base: func

When compiling it with clang++ it first finds a false positive in iostream. We come to that later. If we include cstdio instead of iostream and change the print function like this printf("Base: %p \n", (void *)__FUNCTION__);, it is also not found:

> clang++ -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp
> a.exe
Base: 00007FF6DBA0A967

I found out it works if we add cfi-cast-strict in addition cfi. This will give us the correct behavior for both cases:

> clang++ -fsanitize=cfi,cfi-cast-strict -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp
> clang-cl -fsanitize=cfi,cfi-cast-strict -flto -fno-sanitize-trap=all -fuse-ld=lld example.cpp
wrong_cast.cpp:22:18: runtime error: control flow integrity check for type 'Derived' failed during base-to-derived cast (vtable address 0x7ff659688468)
0x7ff659688468: note: vtable is of type 'struct Base'
 f6 7f 00 00  a0 11 65 59 f6 7f 00 00  e0 11 65 59 f6 7f 00 00  01 00 00 00 00 00 00 00  00 00 00 00
              ^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior wrong_cast.cpp:22:18 in

The documentation clearly says:

Available schemes are:

  • -fsanitize=cfi-cast-strict: Enables strict cast checks.
  • ...
  • -fsanitize=cfi-mfcall: Indirect call via a member function pointer with wrong dynamic type.

You can use -fsanitize=cfi to enable all the schemes.

This is not correct, cfi-cast-strict is not included in cfi. This should be fixed.

3. False Positive

In two examples clang++ seems to have false positives. Here is the cfi_cast_strict.cpp example:

> clang++ -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld cfi_cast_strict.cpp
> a.exe
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\include\ostream:768:32: runtime error: control flow integrity check for type 'std::ios_base' failed during non-virtual call (vtable address 0x7ff6e8253970)
0x7ff6e8253970: note: invalid vtable
 f6 7f 00 00  d0 9c 21 e8 f6 7f 00 00  00 00 00 00 10 00 00 00  80 ea 25 e8 f6 7f 00 00  dc 9a 21 e8

I looks like it has to do again with std::cout <<. Even a simple test triggers the behavior:

#include <iostream>

int main(int argc, const char *argv[]) {
    std::cout << "test" << "\n";
    return 0;
}
> clang++ -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld test.cpp
> a.exe
::ios_base' failed during non-virtual call (vtable address 0x7ff602b13970)
0x7ff602b13970: note: invalid vtable
 f6 7f 00 00  20 9b ad 02 f6 7f 00 00  00 00 00 00 10 00 00 00  f0 e9 b1 02 f6 7f 00 00  2c 99 ad 02
              ^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\include\ostream:768:32 in

The same example works when using clang-cl:

> clang-cl -fsanitize=cfi -flto -fno-sanitize-trap=all -fuse-ld=lld test.cpp
> test.exe
test

This doesn't happen under Linux.

4. Visibility hidden doesn't seem to be required

In the documentation it is stated:

The compiler will only produce CFI checks for a class if it can infer hidden LTO visibility for that class.

The -fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast} flags require that a -fvisibility= flag also be specified. This is because the default visibility setting is -fvisibility=default, which would disable CFI checks for classes without visibility attributes. Most users will want to specify -fvisibility=hidden, which enables CFI checks for such classes.

Two observations:

First, it is not documented how a hidden visibility could be set when using clang-cl. Can it even be done?

Second, the compiler seems to be much better in inferring a hidden visibility on Windows. For clang++ in the examples above it doesn't matter if we define -fvisibility=hidden or not. On my Ubuntu installation and if I set -fvisibility=default almost no CFI violation is detected. Also on Windows the setting is not enforced, while on Linux it is, so the compiler is aware of some difference here.

One might wonder if it is even required, or possible, to set the visibility to hidden in Windows even in clang++. In my opinion this should be documented.

Here is the behavior on Ubuntu with Clang 14:

Example visibilit=hidden visibility=default
cfi_cast_strict.cpp found not found
cfi_derived_cast.cpp found not found
cfi_icall.c found found
cfi_nvcall.cpp found not found
cfi_unrelated_cast.cpp found not found
cfi_vcall.cpp found not found

And on Windows clang++, when avoiding the problematic std::cout:

Example visibilit=hidden visibility=default
cfi_cast_strict.cpp found found
cfi_derived_cast.cpp found found
cfi_icall.c found found
cfi_nvcall.cpp found found
cfi_unrelated_cast.cpp found found
cfi_vcall.cpp found found

Metadata

Metadata

Assignees

No one assigned

    Labels

    compiler-rt:cfiControl Flow Integrityplatform:windows

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions