Segfault on kernel debug info

[vries]

$ ./build/pahole -C  mem_cgroup ~/vmlinux-5.3.18-107.g1c7b2b9-default.debug
tag__recode_dwarf_type: couldn't find 0x734b27 type for 0x3e (pointer_type)!
tag__recode_dwarf_type: couldn't find 0x22 type for 0x75 (array_type)!
  ...
tag__recode_dwarf_type: couldn't find 0x181e5 type for 0x452c8 (formal_parameter)!
tag__recode_dwarf_type: couldn't find 0x734b75 type for 0x452cd (formal_parameter)!
Segmentation fault (core dumped)

[vries]

file is too big to attach, can be found in https://download.opensuse.org/repositories/Kernel:/SLE15-SP2/standard/x86_64/kernel-default-debuginfo-5.3.18-107.1.g1c7b2b9.x86_64.rpm

[vries]

We segfault because tag is NULL:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f65d92 in tag__is_pointer (tag=0x0) at /home/vries/dwarves/src/dwarves.h:446
446             return tag->tag == DW_TAG_pointer_type;

Backtrace:

(gdb) bt
#0  0x00007ffff7f65d92 in tag__is_pointer (tag=0x0) at /home/vries/dwarves/src/dwarves.h:446
#1  0x00007ffff7f69914 in tag__natural_alignment (tag=0x0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1428
#2  0x00007ffff7f69ad4 in type__natural_alignment (type=0x15867f0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1469
#3  0x00007ffff7f699c0 in tag__natural_alignment (tag=0x15867f0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1435
#4  0x00007ffff7f69bc1 in type__check_structs_at_unnatural_alignments (type=0x15263f0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1494
#5  0x00007ffff7f69cd4 in class__infer_packed_attributes (cls=0x15263f0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1521
#6  0x00007ffff7f70521 in __class__fprintf (class=0x15263f0, cu=0xcd6520, conf=0x7fffffffb610, fp=0x7ffff7e5e500 <_IO_2_1_stdout_>)
    at /home/vries/dwarves/src/dwarves_fprintf.c:1351
#7  0x00007ffff7f720f0 in tag__fprintf (tag=0x15263f0, cu=0xcd6520, conf=0x40e2a0 <conf>, fp=0x7ffff7e5e500 <_IO_2_1_stdout_>)
    at /home/vries/dwarves/src/dwarves_fprintf.c:1852
#8  0x0000000000407c51 in pahole_stealer (cu=0xcd6520, conf_load=0x40e300 <conf_load>) at /home/vries/dwarves/src/pahole.c:2481
#9  0x00007ffff7f811f1 in finalize_cu (cus=0x436870, cu=0xcd6520, dcu=0x7fffffffb850, conf=0x40e300 <conf_load>)
    at /home/vries/dwarves/src/dwarf_loader.c:2216
#10 0x00007ffff7f8122a in finalize_cu_immediately (cus=0x436870, cu=0xcd6520, dcu=0x7fffffffb850, conf=0x40e300 <conf_load>)
    at /home/vries/dwarves/src/dwarf_loader.c:2225
#11 0x00007ffff7f818d3 in cus__load_module (cus=0x436870, conf=0x40e300 <conf_load>, mod=0x436570, dw=0x432c50, elf=0x436890, 
    filename=0x7fffffffe1a7 "/home/vries/vmlinux-5.3.18-107.g1c7b2b9-default.debug") at /home/vries/dwarves/src/dwarf_loader.c:2378
#12 0x00007ffff7f819e7 in cus__process_dwflmod (dwflmod=0x436570, userdata=0x436580, 
    name=0x435d60 "/home/vries/vmlinux-5.3.18-107.g1c7b2b9-default.debug", base=18446744071578845184, arg=0x7fffffffda70)
    at /home/vries/dwarves/src/dwarf_loader.c:2423
#13 0x00007ffff7ee96c1 in dwfl_getmodules () from /usr/lib64/libdw.so.1
#14 0x00007ffff7f81aa7 in cus__process_file (cus=0x436870, conf=0x40e300 <conf_load>, fd=3, 
    filename=0x7fffffffe1a7 "/home/vries/vmlinux-5.3.18-107.g1c7b2b9-default.debug") at /home/vries/dwarves/src/dwarf_loader.c:2476
#15 0x00007ffff7f81b23 in dwarf__load_file (cus=0x436870, conf=0x40e300 <conf_load>, 
    filename=0x7fffffffe1a7 "/home/vries/vmlinux-5.3.18-107.g1c7b2b9-default.debug") at /home/vries/dwarves/src/dwarf_loader.c:2493
#16 0x00007ffff7f6ac7e in cus__load_file (cus=0x436870, conf=0x40e300 <conf_load>, 
    filename=0x7fffffffe1a7 "/home/vries/vmlinux-5.3.18-107.g1c7b2b9-default.debug") at /home/vries/dwarves/src/dwarves.c:1941
#17 0x00007ffff7f6b7af in cus__load_files (cus=0x436870, conf=0x40e300 <conf_load>, filenames=0x7fffffffdcf0)
    at /home/vries/dwarves/src/dwarves.c:2299
#18 0x00000000004082c8 in main (argc=4, argv=0x7fffffffdcd8) at /home/vries/dwarves/src/pahole.c:2675

[vries]

Tag is NULL because member_type is NULL:

(gdb) up
#2  0x00007ffff7f69ad4 in type__natural_alignment (type=0x15867f0, cu=0xcd6520) at /home/vries/dwarves/src/dwarves.c:1469
1469                    size_t member_natural_alignment = tag__natural_alignment(member_type, cu);
(gdb) p member_type
$1 = (struct tag *) 0x0

which was set just before:
1468 struct tag *member_type = tag__strip_typedefs_and_modifiers(&member->tag, cu);
but ends up as NULL because member->tag->type is 0:

(gdb) p *member
$3 = {tag = {node = {next = 0x15869e0, prev = 0x1586920}, type = 0, tag = 13, visited = false, top_level = false, recursivity_level = 0, 
    priv = 0x1585e30}, name = 55224, bit_offset = 128, bit_size = 0, byte_offset = 16, byte_size = 0, bitfield_offset = 0 '\000', 
  bitfield_size = 0 '\000', bit_hole = 0 '\000', bitfield_end = 0 '\000', const_value = 0, alignment = 8, visited = 0 '\000', 
  is_static = 0 '\000', accessibility = 0 '\000', virtuality = 0 '\000', hole = 0}

Looking a bit more in detail:

(gdb) p /x *((struct dwarf_tag *)member.tag.priv)
$6 = {hash_node = {next = 0x1574700, pprev = 0x15758d0}, type = {from_types = 0x0, off = 0x726e}, id = 0x4273c, {abstract_origin = {
      from_types = 0x0, off = 0x0}, containing_type = {from_types = 0x0, off = 0x0}}, tag = 0x1586980, small_id = 0x0, decl_file = 0x0, 
  decl_line = 0x0}

we can see that the member is a DIE at 0x4273c with type DIE 0x726e, which corresponds to one of the messages:
namespace__recode_dwarf_types: couldn't find 0x726e type for 0x4273c (member)!

[acmel]

I'm downloading the file to try and fix this.

[acmel]

I couldn't find the SLES15-SP2 directory under Kernel:/, can you please check and provide some other URL for download?

[vries]

I couldn't find the SLES15-SP2 directory under Kernel:/, can you please check and provide some other URL for download?

https://download.opensuse.org/repositories/Kernel:/SLE15-SP2/standard/x86_64/kernel-default-debuginfo-5.3.18-109.1.g8ff6392.x86_64.rpm

[vries]

I couldn't find the SLES15-SP2 directory under Kernel:/,

FWIW, you tried SLES15-SP2 but the dir is called SLE15-SP2.

[acmel]

It works now, but I had just copy'n'pasted the provided URL to use with wget, it didn't work, so I accessed the https://download.opensuse.org/repositories/Kernel:/ directory, there was nothing starting with SLE, I used SLES out of muscle memory for SuSE's SLES Enterprise distro :-) Maybe some mirror without that dir, anyway, looking at it.

[acmel]

Yeah, SUSE is using partial units, probably using some kernel config option to reduce the size of the kernel debuginfo packages:

<0>: Abbrev Number: 45 (DW_TAG_partial_unit)
DW_AT_stmt_list : 0x185
<10> DW_AT_comp_dir : (indirect string, offset: 0x1bc53f): /usr/src/debug/kernel-default-5.3.18-109.1.g8ff6392.x86_64/linux-5.3/linux-obj

And that isn't yet supported in dwarves.

The fact that SuSE ships it this way makes supporting DW_TAG_partial_unit a priority for dwarves 1.19, I'll try and get v1.18 out of the door and then work on this, thanks for the report.

[vries]

Thanks for the feedback.

Just curious, are you planning to fix the segfault in v1.18 in a conservative way?

[acmel]

Em Mon, Sep 14, 2020 at 12:27:31AM -0700, Tom de Vries escreveu:

Thanks for the feedback. Just curious, are you planning to fix the segfault in v1.18 in a conservative way?

Yes, I'll probably check if there are DW_TAG_partial_unit and stop at that point, can you provide me with this kernel .config file? I want to check what options are making the compiler use partial units, probably Fedora has: [acme@quaco perf]$ grep DEBUG_INFO /boot/config-5.8.8-200.fc32.x86_64 CONFIG_DEBUG_INFO=y # CONFIG_DEBUG_INFO_REDUCED is not set # CONFIG_DEBUG_INFO_COMPRESSED is not set # CONFIG_DEBUG_INFO_SPLIT is not set # CONFIG_DEBUG_INFO_DWARF4 is not set CONFIG_DEBUG_INFO_BTF=y [acme@quaco perf]$ And doesn't have partial units, so I want to provide a nice error message, providing a possible workaround. - Arnaldo

…

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #10 (comment)

-- - Arnaldo

[vries]

Em Mon, Sep 14, 2020 at 12:27:31AM -0700, Tom de Vries escreveu:

can you provide me with this kernel .config file?

I downloaded https://download.opensuse.org/repositories/Kernel:/SLE15-SP2/standard/x86_64/kernel-default-5.3.18-112.1.gd6c1198.x86_64.rpm and got:

$ grep DEBUG_INFO boot/config-5.3.18-112.gd6c1198-default
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
CONFIG_DEBUG_INFO_DWARF4=y
# CONFIG_DEBUG_INFO_BTF is not set

I'm not sure tif the partial units are caused by a kernel option.

It might just be a consequence of rpm's find-debuginfo.sh calling dwz.

[acmel]

Em Thu, Sep 17, 2020 at 10:34:59AM -0700, Tom de Vries escreveu:

> Em Mon, Sep 14, 2020 at 12:27:31AM -0700, Tom de Vries escreveu: > > can you provide me with this kernel .config file? I downloaded https://download.opensuse.org/repositories/Kernel:/SLE15-SP2/standard/x86_64/kernel-default-5.3.18-112.1.gd6c1198.x86_64.rpm and got: ``` $ grep DEBUG_INFO boot/config-5.3.18-112.gd6c1198-default CONFIG_DEBUG_INFO=y # CONFIG_DEBUG_INFO_REDUCED is not set # CONFIG_DEBUG_INFO_SPLIT is not set CONFIG_DEBUG_INFO_DWARF4=y # CONFIG_DEBUG_INFO_BTF is not set ``` I'm not sure tif the partial units are caused by a kernel option. It might just be a consequence of rpm's find-debuginfo.sh calling dwz.

I think that is the reason, it'll compress the DWARF by introducing DW_TAG_partial_unit and DW_TAG_imported_unit, etc.

…

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #10 (comment)