diff between Phase 2 and 3 object files when building release_90 on Linux

[zmodem]

Bugzilla Link	42739
Resolution	FIXED
Resolved on	Aug 06, 2019 03:58
Version	9.0
OS	Linux
Blocks	#41819
CC	@rovka,@pogo59

Extended Description

$ utils/release/test-release.sh -svn-path branches/release_90 -triple x86_64-unknown-linux-gnu

...

Comparing Phase 2 and Phase 3 files

file JITLinkGeneric.cpp.o differs between phase 2 and phase 3
file MachO_x86_64.cpp.o differs between phase 2 and phase 3
file llvm-config.cpp.o differs between phase 2 and phase 3

[pogo59]

Does this procedure also look at the unittest files? We're kind of
grasping at straws re. llvm/llvm-bugzilla-archive#42719 .

[zmodem]

Does this procedure also look at the unittest files? We're kind of
grasping at straws re. llvm/llvm-bugzilla-archive#42719 .

Yes, it compares all object files in the build directories, which includes the unit test files.

[zmodem]

Two of the same files also differ on Mac:

Comparing Phase 2 and Phase 3 files

file JITLinkGeneric.cpp.o differs between phase 2 and phase 3
file MachO_x86_64.cpp.o differs between phase 2 and phase 3

[rovka]

Same 3 files on aarch64-linux-gnu.

[rovka]

*** Bug llvm/llvm-bugzilla-archive#42855 has been marked as a duplicate of this bug. ***

[zmodem]

Actually, it looks like the two first files are not different:

$ for x in JITLinkGeneric.cpp.o MachO_x86_64.cpp.o ; do sha1sum find . -name $x | grep 'Phase[23]' ; done
0e80e918dbab8719f0379e5c758f341169ed2269 ./Phase2/Release/llvmCore-test-branches_release_90.obj/lib/ExecutionEngine/JITLink/CMakeFiles/LLVMJITLink.dir/JITLinkGeneric.cpp.o
0e80e918dbab8719f0379e5c758f341169ed2269 ./Phase3/Release/llvmCore-test-branches_release_90.obj/lib/ExecutionEngine/JITLink/CMakeFiles/LLVMJITLink.dir/JITLinkGeneric.cpp.o
06445f3c3aa24bfc343169951b31f0a76cc2473c ./Phase2/Release/llvmCore-test-branches_release_90.obj/lib/ExecutionEngine/JITLink/CMakeFiles/LLVMJITLink.dir/MachO_x86_64.cpp.o
06445f3c3aa24bfc343169951b31f0a76cc2473c ./Phase3/Release/llvmCore-test-branches_release_90.obj/lib/ExecutionEngine/JITLink/CMakeFiles/LLVMJITLink.dir/MachO_x86_64.cpp.o

test-release.sh reports them different because of the sed command that substitutes occurrences of Phase2 for Phase3 in the Phase2 files. Unfortunately, the object files now include strings like "ZN4llvm7jitlink13JITLinkerBase10linkPhase2".
This should be easy to fix.

For llvm-config.o, it looks like we're writing some strings in a non-deterministic order. From a hexdump of Phase2:

00007be0: 6976 6549 6e73 7443 6f6d 6269 6e65 0061 iveInstCombine.a
00007bf0: 6c6c 0078 7261 7900 6f62 6a65 6374 7961 ll.xray.objectya
00007c00: 6d6c 006c 696e 6565 6469 746f 7200 6f72 ml.lineeditor.or
00007c10: 636a 6974 0063 6f72 6f75 7469 6e65 7300 cjit.coroutines.
00007c20: 6774 6573 745f 6d61 696e 0066 757a 7a6d gtest_main.fuzzm
00007c30: 7574 6174 6500 7465 7374 696e 6773 7570 utate.testingsup
00007c40: 706f 7274 0074 6578 7461 7069 006c 746f port.textapi.lto
00007c50: 0073 796d 626f 6c69 7a65 0074 6162 6c65 .symbolize.table
00007c60: 6765 6e00 656e 6769 6e65 0063 6f76 6572 gen.engine.cover
00007c70: 6167 6500 6e61 7469 7665 636f 6465 6765 age.nativecodege
00007c80: 6e00 7769 6e64 6f77 736d 616e 6966 6573 n.windowsmanifes
00007c90: 7400 6d63 6100 646c 6c74 6f6f 6c64 7269 t.mca.dlltooldri
00007ca0: 7665 7200 6465 6275 6769 6e66 6f67 7379 ver.debuginfogsy
00007cb0: 6d00 6c69 6264 7269 7665 7200 696e 7465 m.libdriver.inte
00007cc0: 7270 7265 7465 7200 616c 6c2d 7461 7267 rpreter.all-targ
00007cd0: 6574 7300 616d 6467 7075 0061 726d 0062 ets.amdgpu.arm.b
00007ce0: 7066 0068 6578 6167 6f6e 006c 616e 6169 pf.hexagon.lanai
00007cf0: 006d 6970 7300 6d73 7034 3330 006e 7670 .mips.msp430.nvp

From Phase3:

00007be0: 6976 6549 6e73 7443 6f6d 6269 6e65 0061 iveInstCombine.a
00007bf0: 6c6c 006c 6962 6472 6976 6572 0074 6162 ll.libdriver.tab
00007c00: 6c65 6765 6e00 6465 6275 6769 6e66 6f67 legen.debuginfog
00007c10: 7379 6d00 616c 6c2d 7461 7267 6574 7300 sym.all-targets.
00007c20: 7465 7374 696e 6773 7570 706f 7274 0066 testingsupport.f
00007c30: 757a 7a6d 7574 6174 6500 646c 6c74 6f6f uzzmutate.dlltoo
00007c40: 6c64 7269 7665 7200 656e 6769 6e65 006f ldriver.engine.o
00007c50: 7263 6a69 7400 6f62 6a65 6374 7961 6d6c rcjit.objectyaml
00007c60: 006c 746f 0069 6e74 6572 7072 6574 6572 .lto.interpreter
00007c70: 0063 6f72 6f75 7469 6e65 7300 7872 6179 .coroutines.xray
00007c80: 006d 6361 0067 7465 7374 5f6d 6169 6e00 .mca.gtest_main.
00007c90: 636f 7665 7261 6765 0073 796d 626f 6c69 coverage.symboli
00007ca0: 7a65 0074 6578 7461 7069 006e 6174 6976 ze.textapi.nativ
00007cb0: 6563 6f64 6567 656e 006c 696e 6565 6469 ecodegen.lineedi
00007cc0: 746f 7200 7769 6e64 6f77 736d 616e 6966 tor.windowsmanif
00007cd0: 6573 7400 616d 6467 7075 0061 726d 0062 est.amdgpu.arm.b
00007ce0: 7066 0068 6578 6167 6f6e 006c 616e 6169 pf.hexagon.lanai
00007cf0: 006d 6970 7300 6d73 7034 3330 006e 7670 .mips.msp430.nvp

[zmodem]

Yes, the llvm-config diff can be seen in llvmCore-test-branches_release_90.obj/tools/llvm-config/LibraryDependencies.inc

The RequiredLibraries array for the "all" component is ordered differently.

LibraryDependencies.inc is generated at CMake time by invoking

/work/llvm.monorepo/llvm/utils/llvm-build/llvm-build --native-target X86 --enable-targets AArch64;AMDGPU;ARM;BPF;Hexagon;Lanai;Mips;MSP430;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86;XCore --enable-optional-components --write-library-table /work/llvm.monorepo/build.release/tools/llvm-config/LibraryDependencies.inc --write-cmake-fragment /work/llvm.monorepo/build.release/LLVMBuild.cmake

[zmodem]

Committed r367846 and r367847 to address this.

I'll run the script on the branch to verify that it worked...

[zmodem]

Committed r367846 and r367847 to address this.

I'll run the script on the branch to verify that it worked...

Also needed r368004, but now everything compares equal.

[rovka]

mentioned in issue llvm/llvm-bugzilla-archive#42855