lldb crashes because of stack overflow in clang::ASTContext::getASTRecordLayout

[vadimcn]

Bugzilla Link	44259
Version	unspecified
OS	Linux
CC	@dwblaikie,@JDevlieghere,@Teemperor

Extended Description

lldb was built from top of the master (d5acc83), but the same happens with lldb-7.

[Teemperor]

Do you see any way how I could reproduce this?

If not, could you go to RecordLayoutBuilder.cpp and add a D->dumpColor(); to the first line of the function:
const ASTRecordLayout &ASTContext::getASTRecordLayout(const RecordDecl *D) const

This will print the Clang representation of the type that we fail to layout. It should produce quite a lot of output because we probably do an infinite recursion.

(Marking as confirmed because multiple people already reported this).

[vadimcn]

The code is pretty simple:

#include <Eigen/Core>
#include

using namespace Eigen;

int main() {
Matrix4f mat = Matrix4f::Identity();
return 0;
}

Set a breakpoint on return, then execute p mat.cols()
I used Eigen 3.3.7 and GCC 7.4.0.

[Teemperor]

I don't have GCC 7.4 around, but when I try this with clang 9.0.0 or GCC 9.2.0 then I can't reproduce this with ToT-lldb or lldb 9.0.

Can you post your compiler invocation (for compiling that example) and the exact LLDB commands you are running (copy-pasting your LLDB session is enough).

[vadimcn]

Ah, I see: bugzilla ate the reproducer package I tried to attach. (1MB might be too small these days - mine was 8MB after zipping)
Anyhow:
$ g++ --version
g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
...
$ g++ eigen-debuggee.cpp -I eigen-3.3.7 -g -o eigen-debuggee
$ lldb eigen-debuggee
(lldb) target create "eigen-debuggee"
Current executable set to 'eigen-debuggee' (x86_64).
(lldb) break set -f eigen-debuggee.cpp -l 8
Breakpoint 1: where = eigen-debuggee`main + 54 at eigen-debuggee.cpp:8, address = 0x0000000000000930
(lldb) run
Process 29214 launched: '/tmp/eigen-debuggee/eigen-debuggee' (x86_64)
Process 29214 stopped

• thread #​1, name = 'eigen-debuggee', stop reason = breakpoint 1.1
  frame #​0: 0x0000555555554930 eigen-debuggee`main at eigen-debuggee.cpp:8
  5
  6 int main() {
  7 Matrix4f mat = Matrix4f::Identity();
  -> 8 return 0;
  9 }
  (lldb) p mat.cols()

[Teemperor]

The record that we seem to recurse on is:
ClassTemplateSpecializationDecl 0x55c6f815bd00 <> class DenseCoeffsBase definition
|-DefinitionData pass_in_registers empty standard_layout trivially_copyable trivial has_constexpr_non_copy_move_ctor can_const_default_init
| |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
| |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
| |-MoveConstructor exists simple trivial needs_implicit
| |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
| |-MoveAssignment exists simple trivial needs_implicit
| -Destructor simple irrelevant trivial needs_implicit |-public 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>' |-TemplateArgument type 'Eigen::Matrix<float, 4, 4, 4, 4, 4>' |-TemplateArgument integral 4 |-TypedefDecl 0x55c6f815dda0 <<invalid sloc>> <invalid sloc> Scalar 'Eigen::internal::traits<Eigen::Matrix<float, 4, 4, 4, 4, 4> >::Scalar':'float' | -TypedefType 0x55c6f815dd70 'Eigen::internal::traits<Eigen::Matrix<float, 4, 4, 4, 4, 4> >::Scalar' sugar
| |-Typedef 0x55c6f815dd10 'Scalar'
| -BuiltinType 0x55c6f80760b0 'float' |-CXXMethodDecl 0x55c6f815e168 <<invalid sloc>> <invalid sloc> coeffRef 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index, Eigen::Index)' | |-ParmVarDecl 0x55c6f815e098 <<invalid sloc>> <invalid sloc> 'Eigen::Index':'long' | |-ParmVarDecl 0x55c6f815e100 <<invalid sloc>> <invalid sloc> 'Eigen::Index':'long' | -AsmLabelAttr 0x55c6f815e208 <> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE8coeffRefEll"
|-CXXMethodDecl 0x55c6f815e3a0 <> coeffRefByOuterInner 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index, Eigen::Index)'
| |-ParmVarDecl 0x55c6f815e2d0 <> 'Eigen::Index':'long'
| |-ParmVarDecl 0x55c6f815e338 <> 'Eigen::Index':'long'
| -AsmLabelAttr 0x55c6f815e440 <<invalid sloc>> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE20coeffRefByOuterInnerEll" |-CXXMethodDecl 0x55c6f815e5e8 <<invalid sloc>> <invalid sloc> operator() 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index, Eigen::Index)' | |-ParmVarDecl 0x55c6f815e518 <<invalid sloc>> <invalid sloc> 'Eigen::Index':'long' | |-ParmVarDecl 0x55c6f815e580 <<invalid sloc>> <invalid sloc> 'Eigen::Index':'long' | -AsmLabelAttr 0x55c6f815e688 <> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EEclEll"
|-CXXMethodDecl 0x55c6f815e828 <> coeffRef 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index)'
| |-ParmVarDecl 0x55c6f815e7c0 <> 'Eigen::Index':'long'
| -AsmLabelAttr 0x55c6f815e8c8 <<invalid sloc>> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE8coeffRefEl" |-CXXMethodDecl 0x55c6f815e9f0 <<invalid sloc>> <invalid sloc> operator[] 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index)' | |-ParmVarDecl 0x55c6f815e988 <<invalid sloc>> <invalid sloc> 'Eigen::Index':'long' | -AsmLabelAttr 0x55c6f815ea90 <> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EEixEl"
|-CXXMethodDecl 0x55c6f8160c08 <> operator() 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &(Eigen::Index)'
| |-ParmVarDecl 0x55c6f8160ba0 <> 'Eigen::Index':'long'
| -AsmLabelAttr 0x55c6f8160ca8 <<invalid sloc>> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EEclEl" |-CXXMethodDecl 0x55c6f8160db8 <<invalid sloc>> <invalid sloc> x 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &()' | -AsmLabelAttr 0x55c6f8160e58 <> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE1xEv"
|-CXXMethodDecl 0x55c6f8160f08 <> y 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &()'
| -AsmLabelAttr 0x55c6f8160fa8 <<invalid sloc>> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE1yEv" |-CXXMethodDecl 0x55c6f8161058 <<invalid sloc>> <invalid sloc> z 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &()' | -AsmLabelAttr 0x55c6f81610f8 <> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE1zEv"
|-CXXMethodDecl 0x55c6f81611a8 <> w 'Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 4, 4, 4>, 4>::Scalar &()'
| -AsmLabelAttr 0x55c6f8161248 <<invalid sloc>> Implicit "_ZN5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi1EE1wEv" |-CXXMethodDecl 0x55c6f81613e8 <<invalid sloc>> <invalid sloc> innerStride 'Eigen::Index () const' | -AsmLabelAttr 0x55c6f8161488 <> Implicit "_ZNK5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi3EE11innerStrideEv"
|-CXXMethodDecl 0x55c6f8161548 <> outerStride 'Eigen::Index () const'
| -AsmLabelAttr 0x55c6f81615e8 <<invalid sloc>> Implicit "_ZNK5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi3EE11outerStrideEv" |-CXXMethodDecl 0x55c6f81616a8 <<invalid sloc>> <invalid sloc> stride 'Eigen::Index () const' | -AsmLabelAttr 0x55c6f8161748 <> Implicit "_ZNK5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi3EE6strideEv"
|-CXXMethodDecl 0x55c6f8161800 <> rowStride 'Eigen::Index () const'
| -AsmLabelAttr 0x55c6f81618a0 <<invalid sloc>> Implicit "_ZNK5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi3EE9rowStrideEv" -CXXMethodDecl 0x55c6f8161958 <> colStride 'Eigen::Index () const'
`-AsmLabelAttr 0x55c6f81619f8 <> Implicit "_ZNK5Eigen15DenseCoeffsBaseINS_6MatrixIfLi4ELi4ELi0ELi4ELi4EEELi3EE9colStrideEv"

[Teemperor]

Here is a static Linux executable reproducing this: http://teemperor.de/pub/eigen-debuggee

Reproduce with: lldb eigen-debuggee -o "break set -f eigen3.cpp -l 8" -o r -o "p mat.cols()"

[vadimcn]

This is still reproducible with g++ 7.5.0 and lldb from release/14.x branch.

[vadimcn]

I've managed to reduce repro to this:

template<int Level>
class Class;

template<>
class Class<0>
{};

template<>
class Class<1> : public Class<0>
{};

int main()
{
  Class<1> x;
  return 0; // 'p x' crashes lldb
}

g++-7 debug info (crashes lldb):

0x0000002d:   DW_TAG_class_type
                DW_AT_name	("Class<0>")
                DW_AT_byte_size	(0x01)
                DW_AT_decl_file	("repro.cpp")
                DW_AT_decl_line	(5)

0x00000035:   DW_TAG_class_type
                DW_AT_name	("Class<1>")
                DW_AT_byte_size	(0x01)
                DW_AT_decl_file	("repro.cpp")
                DW_AT_decl_line	(10)
                DW_AT_sibling	(0x00000049)

0x00000041:     DW_TAG_inheritance
                  DW_AT_type	(0x0000002d "Class<0>")
                  DW_AT_data_member_location	(0x00)
                  DW_AT_accessibility	(DW_ACCESS_public)

0x00000048:     NULL

g++-9 debug info (okay with lldb):

0x0000002d:   DW_TAG_class_type
                DW_AT_name	("Class<0>")
                DW_AT_byte_size	(0x01)
                DW_AT_decl_file	("repro.cpp")
                DW_AT_decl_line	(5)
                DW_AT_decl_column	(0x07)
                DW_AT_sibling	(0x00000045)

0x0000003a:     DW_TAG_template_value_parameter
                  DW_AT_name	("Level")
                  DW_AT_type	(0x00000094 "int")
                  DW_AT_const_value	(0x00)

0x00000044:     NULL

0x00000045:   DW_TAG_class_type
                DW_AT_name	("Class<1>")
                DW_AT_byte_size	(0x01)
                DW_AT_decl_file	("repro.cpp")
                DW_AT_decl_line	(10)
                DW_AT_decl_column	(0x07)
                DW_AT_sibling	(0x00000064)

0x00000052:     DW_TAG_inheritance
                  DW_AT_type	(0x0000002d "Class<0>")
                  DW_AT_data_member_location	(0x00)
                  DW_AT_accessibility	(DW_ACCESS_public)

0x00000059:     DW_TAG_template_value_parameter
                  DW_AT_name	("Level")
                  DW_AT_type	(0x00000094 "int")
                  DW_AT_const_value	(0x01)

0x00000063:     NULL

[Teemperor]

LLDB is asked to create a Clang AST (that is, create a template declaration and specialization) and the only info it has is the string "Class<0>" but no information what the template arguments (and their types) actually are. However we need those things for creating a proper Clang AST.

I think #51160 has some more details but the short answer is: LLDB has to make something up to make this work. Either by guessing types or making up some bogus args that at least stop the crash (e.g., in this case just create a bunch of random but unique arguments). But there is no nice and easy solution here.

FWIW, it's not LLDB that actually makes the self-inheritance thing here. LLDB creates two unique templates but they happen to both have no arguments so Clang's ASTImporter merges the two classes into one class (that inherits from itself...)

[vadimcn]

Why does ASTImporter rely on textual matching of class names? To deduplicate types between modules?
Also, after thinking about it some more, I've one more question: Why does the record layout get computed from AST? Isn't precise layout already available in the debug info?

[Teemperor]

IIRC, The problem is rather that the class name and everything else about the declarations LLDB is creating is the same.

LLDB creates:

template<> Class<> {};
template<> Class<> : /*refers to the one above*/Class<> {};

ASTImporter turns that into:

template<> Class<> : /*now refers to itself*/ Class<> {};

You can see the AST LLDB creates by doing target modules dump ast.

[vadimcn]

The problem is rather that the class name and everything else about the declarations LLDB is creating is the same.

Yeah, I was just a bit surprised to see that record layout is computed from AST, rather than being imported. I mean, isn't there a chance that clang will compute a different layout than the original compiler did?

Anyhow, getting back to the original problem... How do we fix this? I think that the first order goal should be not crashing the debugger, even if that means not being able to evaluate expressions involving certain types.
Can LLDB detect that it is about to create a "bad" AST? Or should the ASTImporter be fixed to detect and reject such ASTs?

[Teemperor]

Yeah, I was just a bit surprised to see that record layout is computed from AST, rather than being imported. I mean, isn't there a chance that clang will compute a different layout than the original compiler did?

Jepp, and there is a some stuff in Clang that only exists to allow LLDB to fixup member offsets etc.

Anyhow, getting back to the original problem... How do we fix this? I think that the first order goal should be not crashing the debugger, even if that means not being able to evaluate expressions involving certain types.
Can LLDB detect that it is about to create a "bad" AST? Or should the ASTImporter be fixed to detect and reject such ASTs?

The ASTImporter should actually reject this stuff because it's an ODR violation but in LLDB most of these checks are disabled (and it's a bunch of work to re-enable them without breaking everything...).

Anyway, you might want to check out this patch: https://reviews.llvm.org/D92425 It should change the AST we create to:

struct Class<0> {};
struct Class<1> : Class<0> {};

(Note that this is now a non-template class named 'Class<1>' which is as weird as it sounds, but it should at least fix this specific).

I don't think that patch will fix the whole issue though as it essentially reverts code that was introduced after this bug was filed. So unless we fix the other underlying bug in the meantime this might be just fix a bug on top of the one that caused the crash in the original report.

So let's see if it fixes this issue and if not we can check what is the new reason for why this crashes. But let's hope for the best here.