clangd/unittests/TUSchedulerTests.cpp fails and timesout frequently

[hiraditya]

For more logs see for example:

• https://ci.android.com/builds/branches/aosp-llvm-toolchain/grid?head=10685355&tail=10685355
• https://ci.android.com/builds/branches/aosp-llvm-toolchain/grid?head=10684480&tail=10684480
• https://ci.android.com/builds/branches/aosp-llvm-toolchain/grid?head=10684020&tail=10684020

Test logs pasted here.

Driver produced command: cc1 -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name no_cmd.h -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fcoverage-compilation-dir=/clangd-test -resource-dir lib/clang/17 -D MAIN -internal-isystem lib/clang/17/include -internal-isystem /usr/local/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/clangd-test -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -no-round-trip-args -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c++-header /clangd-test/no_cmd.h
Rebuilding invalidated preamble for /clangd-test/no_cmd.h version null (previous was version null)
out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1354: Failure
Value of: S.blockUntilIdle(timeoutSeconds(10))
  Actual: false
Expected: true
Built preamble of size 217192 for file /clangd-test/no_cmd.h version null in 11.22 seconds
ASTWorker building file /clangd-test/unreliable.h version null with command inferred from main.cpp
[/clangd-test]
clang -DMAIN -x c++-header -- /clangd-test/unreliable.h
Driver produced command: cc1 -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name unreliable.h -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fcoverage-compilation-dir=/clangd-test -resource-dir lib/clang/17 -D MAIN -internal-isystem lib/clang/17/include -internal-isystem /usr/local/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/clangd-test -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -no-round-trip-args -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c++-header /clangd-test/unreliable.h
Building first preamble for /clangd-test/unreliable.h version null
out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1354: Failure
Value of: S.blockUntilIdle(timeoutSeconds(10))
  Actual: false
Expected: true
Built preamble of size 217204 for file /clangd-test/unreliable.h version null in 10.22 seconds
ASTWorker building file /clangd-test/ok.h version null with command 
[/clangd-test]
clang -xobjective-c++-header /clangd-test/ok.h
Driver produced command: cc1 -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name ok.h -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fcoverage-compilation-dir=/clangd-test -resource-dir lib/clang/17 -internal-isystem lib/clang/17/include -internal-isystem /usr/local/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/clangd-test -ferror-limit 19 -fgnuc-version=4.2.1 -fobjc-runtime=gcc -fobjc-encode-cxx-class-template-spec -fobjc-exceptions -fcxx-exceptions -fexceptions -no-round-trip-args -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x objective-c++-header /clangd-test/ok.h
Building first preamble for /clangd-test/ok.h version null
Built preamble of size 220364 for file /clangd-test/ok.h version null in 0.06 seconds
out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1354: Failure
Value of: S.blockUntilIdle(timeoutSeconds(10))
  Actual: false
Expected: true
ASTWorker building file /clangd-test/not_included.h version null with command clangd fallback
[/clangd-test]

In another run:

out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1354: Failure
Value of: S.blockUntilIdle(timeoutSeconds(10))
  Actual: false
Expected: true

Built preamble of size 207320 for file /clangd-test/foo.cpp version 3 in 5.44 seconds
Reusing preamble version 3 for version 4 of /clangd-test/foo.cpp
out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1318: Failure
Value of: Collector.diagVersions().back()
Expected: has a first field that is equal to "3", and has a second field that is equal to "3"
  Actual: ("3", "4"), whose second field does not match

out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1311
Value of: Collector.diagVersions().back()
Expected: has a first field that is equal to "3", and has a second field that is equal to "3"
  Actual: ("2", "3"), whose first field does not match
out/llvm-project/clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp:1318
Value of: Collector.diagVersions().back()
Expected: has a first field that is equal to "3", and has a second field that is equal to "3"
  Actual: ("3", "4"), whose second field does not match

[llvmbot]

@llvm/issue-subscribers-bug

[llvmbot]

@llvm/issue-subscribers-clangd

[hiraditya]

Created a patch to configure clangd tests apart from clang tools tests. https://reviews.llvm.org/D158566

[HighCommander4]

See also #59644, #50117, clangd/clangd#1712

[kadircet]

Thanks for the report @hiraditya, the correctness issue is because we weren't asserting for executing all the work (hence we falsely, succeeded). Sent out https://reviews.llvm.org/D159337.

Regarding the timeouts, the machine seems to be really busy again, building trivial preambles take ~11 secs: Built preamble of size 217192 for file /clangd-test/no_cmd.h version null in 11.22 seconds. Bumping timeouts in https://reviews.llvm.org/D159338.

Considering these timeout issues are started happening recently, I wonder if we're spawning too many threads with the recent test execution strategy changes. Can you give some details of the machine you're running the tests/builds on @hiraditya ?

[hiraditya]

Can you give some details of the machine you're running the tests/builds on @hiraditya ?

I dont have the details, these are MacOS ci machines. Maybe there is something in the artifact directory you find helpful: https://ci.android.com/builds/submitted/10685355/darwin_mac/latest ?