[DebugInfo] Use locations inside macros (rather than the macro usage location) for instruction source locations in debug info

[SergejSalnikov]

Look at the following example: https://godbolt.org/z/zTnao79xr

The whole macro invocation was folded to a single line. It means that macros are practically not debuggable.
It would be very handy to preserve/generate the correct line numbers while expanding the macro.

Update:
PoC

[SergejSalnikov]

@AaronBallman, do you know which dev has the most context on this topic? I'd like to contribute is noone is willing to pick it up.

[SergejSalnikov]

And here are 2 examples of the same code with and without macro: https://godbolt.org/z/3eoExadMG
I imagine a day when stepping through the expanded version is not any different to stepping through a version with macro.

[llvmbot]

@llvm/issue-subscribers-clang-frontend

Author: None (SergejSalnikov)

Look at the following example: https://godbolt.org/z/zTnao79xr

The whole macro invocation was folded to a single line. It means that macros are practically not debuggable.
It would be very handy to preserve/generate the correct line numbers while expanding the macro.

[SergejSalnikov]

@hokein, do you know if that's feasible to fix?

[SergejSalnikov]

@akyrtzi, adding people who has touched that code. It seems what a small change might be needed to TokenLexer.cc

[AaronBallman]

I'm not certain I understand the issue yet.

The whole macro invocation was folded to a single line.

Line splicing tokens work that way. In Phase 2 of translation, line splices are deleted to form logical source lines.

It means that macros are practically not debuggable.

In general, macros are not debuggable (you can't run them under a debugger like you can non-preprocessor code).

It would be very handy to preserve/generate the correct line numbers while expanding the macro.

We already preserve that information within the compiler: https://godbolt.org/z/xhxaqqrxv where you'll see output like:

identifier 'x'	 [StartOfLine] [LeadingSpace]	Loc=<<source>:11:5 <Spelling=<source>:12:9>>
plusequal '+='	 [LeadingSpace]	Loc=<<source>:11:5 <Spelling=<source>:2:5>>
numeric_constant '3'	 [LeadingSpace]	Loc=<<source>:11:5 <Spelling=<source>:2:8>>

The Loc is telling you where the expansion happened at, and Spelling tells you were the original source was written within the macro expansion list.

[SergejSalnikov]

Watch my PoC:
https://github.com/user-attachments/assets/994fb89f-83be-4c21-a79c-f8e51d818f7b

The macro can be stepped through. But with todays clang implementation the whole macro is rendered on a single line, so there is not much you get from the debugging. It's like trying to debug the code which is written on a single line.

[SergejSalnikov]

I think that debug symbols are built from Loc=<<source>:11:5. We want to use source location for macro arguments and Loc for macro body expansion.

[SergejSalnikov]

Here is the output that will work better I think.

int 'int'        [StartOfLine]  Loc=<main4.cpp:5:1>
identifier 'f'   [LeadingSpace] Loc=<main4.cpp:5:5>
l_paren '('             Loc=<main4.cpp:5:6>
r_paren ')'             Loc=<main4.cpp:5:7>
l_brace '{'      [LeadingSpace] Loc=<main4.cpp:5:9>
return 'return'  [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:6:5>
numeric_constant '5'     [LeadingSpace] Loc=<main4.cpp:6:12>
semi ';'                Loc=<main4.cpp:6:13>
r_brace '}'      [StartOfLine]  Loc=<main4.cpp:7:1>
int 'int'        [StartOfLine]  Loc=<main4.cpp:10:1>
identifier 'square_macro'        [LeadingSpace] Loc=<main4.cpp:10:5>
l_paren '('             Loc=<main4.cpp:10:17>
int 'int'               Loc=<main4.cpp:10:18>
identifier 'x'   [LeadingSpace] Loc=<main4.cpp:10:22>
r_paren ')'             Loc=<main4.cpp:10:23>
l_brace '{'      [LeadingSpace] Loc=<main4.cpp:10:25>

identifier 'x' [StartOfLine] [LeadingSpace] Loc=<main4.cpp:12:9 <Spelling=main4.cpp:12:9>>
plusequal '+=' [StartOfLine] [LeadingSpace] Loc=<main4.cpp:11:5 <Spelling=main4.cpp:2:5>>
numeric_constant '3' [LeadingSpace] Loc=<main4.cpp:11:5 <Spelling=main4.cpp:2:8>>\

semi ';'                Loc=<main4.cpp:11:5 <Spelling=main4.cpp:2:9>>
identifier 'x'   [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:12:9 <Spelling=main4.cpp:12:9>>
plusequal '+='   [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:11:5 <Spelling=main4.cpp:3:5>>
identifier 'f'   [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:13:9 <Spelling=main4.cpp:13:9>>
l_paren '('             Loc=<main4.cpp:13:10 <Spelling=main4.cpp:13:10>>
r_paren ')'             Loc=<main4.cpp:13:11 <Spelling=main4.cpp:13:11>>
plus '+'         [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:14:10 <Spelling=main4.cpp:14:10>>
identifier 'f'   [LeadingSpace] Loc=<main4.cpp:14:12 <Spelling=main4.cpp:14:12>>
l_paren '('             Loc=<main4.cpp:14:13 <Spelling=main4.cpp:14:13>>
r_paren ')'             Loc=<main4.cpp:14:14 <Spelling=main4.cpp:14:14>>
semi ';'         [StartOfLine]  Loc=<main4.cpp:11:5 <Spelling=main4.cpp:3:9>>
return 'return'  [StartOfLine] [LeadingSpace]   Loc=<main4.cpp:16:5>
identifier 'x'   [LeadingSpace] Loc=<main4.cpp:16:12>
semi ';'                Loc=<main4.cpp:16:13>
r_brace '}'      [StartOfLine]  Loc=<main4.cpp:17:1>
eof ''          Loc=<main4.cpp:17:2>

[AaronBallman]

CC @echristo @dwblaikie for debug info details

[dwblaikie]

@AaronBallman - mostly I /think/ the debug info being generated from this is probably fine/worth dabbling with (though I expect there might be some complications from different levels of macro depth, etc, which thing is the right thing to step into/onto/etc - but I'm willing to give it a go if other Debug Info stakeholders (@adrian-prantl @JDevlieghere @jmorse ) are on board)

But yeah, my understanding I think matches yours - the compiler already preserves this information in the AST/source location representation. So this is just a debug info bug and the fix should be in CGDebugInfo.cpp most likely, navigating the different dimensions of source location information in such a way as to produce the desired debug info metadata source locations on the instructions.