Skip to content

Conversation

@zmodem
Copy link
Collaborator

@zmodem zmodem commented Sep 30, 2025

Developers reported non-deterministic output from -module-file-info, thinking this reflected non-determinism in the .pcm files themselves. However, it turned out it was the printing that was non-deterministic:

$ cat /tmp/a.h
#define FOO 1
#define BAR 2

$ build/bin/clang -cc1 -std=c++20 -x c++ -emit-header-unit /tmp/a.h -o /tmp/a.pcm

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     FOO
     BAR

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     BAR
     FOO

Making the output deterministic also simplifies the test.

This is a follow-up to 360c5fe

@zmodem zmodem requested a review from ChuanqiXu9 September 30, 2025 07:56
@zmodem zmodem added the clang:modules C++20 modules and Clang Header Modules label Sep 30, 2025
@llvmbot llvmbot added the clang Clang issues not falling into any other category label Sep 30, 2025
@llvmbot
Copy link
Member

llvmbot commented Sep 30, 2025

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-modules

Author: Hans Wennborg (zmodem)

Changes

Developers reported non-deterministic output from -module-file-info, thinking this reflected non-determinism in the .pcm files themselves. However, it turned out it was the printing that was non-deterministic:

$ cat /tmp/a.h
#define FOO 1
#define BAR 2

$ build/bin/clang -cc1 -std=c++20 -x c++ -emit-header-unit /tmp/a.h -o /tmp/a.pcm

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     FOO
     BAR

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     BAR
     FOO

Making the output deterministic also simplifies the test.

This is a follow-up to 360c5fe


Full diff: https://github.com/llvm/llvm-project/pull/161332.diff

2 Files Affected:

  • (modified) clang/lib/Frontend/FrontendActions.cpp (+11-8)
  • (modified) clang/test/Modules/cxx20-module-file-info-macros.cpp (+10-10)
diff --git a/clang/lib/Frontend/FrontendActions.cpp b/clang/lib/Frontend/FrontendActions.cpp
index 7424958d46612..d7d56b8166350 100644
--- a/clang/lib/Frontend/FrontendActions.cpp
+++ b/clang/lib/Frontend/FrontendActions.cpp
@@ -971,14 +971,17 @@ void DumpModuleInfoAction::ExecuteAction() {
     // Emit the macro definitions in the module file so that we can know how
     // much definitions in the module file quickly.
     // TODO: Emit the macro definition bodies completely.
-    if (auto FilteredMacros = llvm::make_filter_range(
-            R->getPreprocessor().macros(),
-            [](const auto &Macro) { return Macro.first->isFromAST(); });
-        !FilteredMacros.empty()) {
-      Out << "   Macro Definitions:\n";
-      for (/*<IdentifierInfo *, MacroState> pair*/ const auto &Macro :
-           FilteredMacros)
-        Out << "     " << Macro.first->getName() << "\n";
+    {
+      std::vector<StringRef> MacroNames;
+      for (const auto &M : R->getPreprocessor().macros()) {
+        if (M.first->isFromAST())
+          MacroNames.push_back(M.first->getName());
+      }
+      llvm::sort(MacroNames);
+      if (!MacroNames.empty())
+        Out << "   Macro Definitions:\n";
+      for (StringRef Name : MacroNames)
+        Out << "     " << Name << "\n";
     }
 
     // Now let's print out any modules we did not see as part of the Primary.
diff --git a/clang/test/Modules/cxx20-module-file-info-macros.cpp b/clang/test/Modules/cxx20-module-file-info-macros.cpp
index 3b67e9b9acd41..431c967fbbccd 100644
--- a/clang/test/Modules/cxx20-module-file-info-macros.cpp
+++ b/clang/test/Modules/cxx20-module-file-info-macros.cpp
@@ -36,28 +36,28 @@
 #define REDEFINE
 
 // CHECK: Macro Definitions:
-// CHECK-DAG: REDEFINE
-// CHECK-DAG: FUNC_Macro
-// CHECK-DAG: CONSTANT
-// CHECK-DAG: FOO
+// CHECK: CONSTANT
+// CHECK: FOO
+// CHECK: FUNC_Macro
+// CHECK: REDEFINE
 // CHECK-NEXT: ===
 
 //--- include_foo.h
 #include "foo.h"
 #undef REDEFINE
 // CHECK: Macro Definitions:
-// CHECK-DAG: CONSTANT
-// CHECK-DAG: FUNC_Macro
-// CHECK-DAG: FOO
+// CHECK: CONSTANT
+// CHECK: FOO
+// CHECK: FUNC_Macro
 // CHECK-NEXT: ===
 
 //--- import_foo.h
 import "foo.h";
 #undef REDEFINE
 // CHECK: Macro Definitions:
-// CHECK-DAG: CONSTANT
-// CHECK-DAG: FUNC_Macro
-// CHECK-DAG: FOO
+// CHECK: CONSTANT
+// CHECK: FOO
+// CHECK: FUNC_Macro
 // CHECK-NEXT: ===
 
 //--- named_module.cppm

Copy link
Member

@ChuanqiXu9 ChuanqiXu9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. And I am curious to see the report if available.

@zmodem
Copy link
Collaborator Author

zmodem commented Sep 30, 2025

LGTM. And I am curious to see the report if available.

This is where it came up: https://crbug.com/448270644

@zmodem zmodem enabled auto-merge (squash) September 30, 2025 08:16
@ChuanqiXu9
Copy link
Member

Got it. And if the pcm are non determinism, it is super bad. It will be pretty appreciated to fix it. Such problems generally come from DenseMap

@zmodem zmodem merged commit 170b5fd into llvm:main Sep 30, 2025
12 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 30, 2025

LLVM Buildbot has detected a new failure on builder cross-project-tests-sie-ubuntu-dwarf5 running on doug-worker-1b while building clang at step 6 "test-build-unified-tree-check-cross-project".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/163/builds/27340

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-cross-project) failure: test (failure)
******************** TEST 'cross-project-tests :: debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
clang++ -O0 -glldb -std=gnu++11 /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp -o /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/projects/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/Output/limit_steps_expect_value.cpp.tmp # RUN: at line 4
+ clang++ -O0 -glldb -std=gnu++11 /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp -o /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/projects/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/Output/limit_steps_expect_value.cpp.tmp
"/usr/bin/python3.10" "/home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/dexter.py" test --fail-lt 1.0 -w -v --debugger lldb-dap --lldb-executable "/home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/bin/lldb-dap" --dap-message-log=-e --binary /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/projects/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/Output/limit_steps_expect_value.cpp.tmp -- /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp | /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/bin/FileCheck /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp # RUN: at line 5
+ /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/bin/FileCheck /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp
+ /usr/bin/python3.10 /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/dexter.py test --fail-lt 1.0 -w -v --debugger lldb-dap --lldb-executable /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/bin/lldb-dap --dap-message-log=-e --binary /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/projects/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/Output/limit_steps_expect_value.cpp.tmp -- /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/llvm-project/cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/limit_steps/limit_steps_expect_value.cpp
note: Opening DAP server: /home/buildbot/buildbot-root/cross-project-tests-sie-ubuntu-dwarf5/build/bin/lldb-dap
-> {
  "type": "request",
  "command": "initialize",
  "arguments": {
    "clientID": "dexter",
    "adapterID": "lldb-dap",
    "pathFormat": "path",
    "linesStartAt1": true,
    "columnsStartAt1": true,
    "supportsVariableType": true,
    "supportsVariablePaging": true,
    "supportsRunInTerminalRequest": false
  },
  "seq": 1
}
<- {
  "body": {
    "$__lldb_version": "lldb version 22.0.0git (https://github.com/llvm/llvm-project.git revision 170b5fde8fc23525049ebbde8377f59971ee9b3f)\n  clang revision 170b5fde8fc23525049ebbde8377f59971ee9b3f\n  llvm revision 170b5fde8fc23525049ebbde8377f59971ee9b3f",
    "completionTriggerCharacters": [
      ".",
      " ",
      "\t"
    ],
    "exceptionBreakpointFilters": [
      {
        "description": "C++ Catch",
        "filter": "cpp_catch",
        "label": "C++ Catch",
        "supportsCondition": true
      },
      {
        "description": "C++ Throw",
        "filter": "cpp_throw",
        "label": "C++ Throw",
        "supportsCondition": true
      },
      {
        "description": "Objective-C Catch",
        "filter": "objc_catch",
...

mahesh-attarde pushed a commit to mahesh-attarde/llvm-project that referenced this pull request Oct 3, 2025
…rder (llvm#161332)

Developers reported non-deterministic output from `-module-file-info`,
thinking this reflected non-determinism in the .pcm files themselves.
However, it turned out it was the printing that was non-deterministic:

```
$ cat /tmp/a.h
#define FOO 1
#define BAR 2

$ build/bin/clang -cc1 -std=c++20 -x c++ -emit-header-unit /tmp/a.h -o /tmp/a.pcm

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     FOO
     BAR

$ build/bin/clang -cc1 -module-file-info /tmp/a.pcm | grep -A2 Definitions
   Macro Definitions:
     BAR
     FOO
```

Making the output deterministic also simplifies the test.

This is a follow-up to 360c5fe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants