[GR-44851] A common base library for DWARF

[flohuemer]

During the implementation of DWARF debugging information support for GraalWasm, the idea of a common base library for DWARF on the GraalVM arose. This is mainly motivated by the fact that SVM, Sulong, and GraalWasm have a lot of code duplication in regard to the DWARF debugging information format. Especially encoding information like tags, attributes, and value encodings are present in all three locations. While SVM generates DWARF information, Sulong and GraalWasm read DWARF information.

Since I only know the requirements for GraalWasm, and I am not an experts in regard to DWARF, this should be the start of a discussion about if a common base library for DWARF on the GraalVM would be useful and how such a library could look like so that all three implementations can share as much code as possible.

The big advantage of a common base library for DWARF would be that changes, like supporting a new DWARF version or fixing a bug, would only have to be implemented in one location with little adjustment effort in the individual suites.

• Involved SVM package
• Involved Sulong package
• Involved GraalWasm package

[adinn]

@flohuemer @olpaw

Hi Florian. Firstly, I agree that this sounds like a good idea. How much can be shared is nto clear but there are definitely some good candidates I can think of for including in some library classes.

The obvious place to look for sharing is in the code that describes the basic DWARF layout elements, i.e. Compile Units, DIEs (DWARF Information Entries) and DWARF attributes, their forms (i.e. essentially types) and their pre-defined, possible values, plus perhaps some generic aspects of how those elements are organized such as sizings. There may be some opportunity also to draw in some shared concerns that relate to ELF. However, initially I think it would be best to focus on the interpretation of the DWARF byte sequences themselves rather than how they are embedded in an ELF container.

I don't think there is going to be a lot more opportunity to share code beyond the elements defined above. Firstly, a lot of the code serves different needs. The SVM code is primarily concerned with marshalling a specific Java code model into byte sequence in a valid DWARF format. It sounds like Sulong and Wasm are mostly going to be concerned with parsing DWARF format byte sequences into a code model appropriate to some appropriate source language.

In the former case the task is a relatively simply one of traversing the input model and encoding the correct sequence of values in the correct order. The only complexity is tracking cross-references within and between sections so that the relevant offsets can be filled in. In the latter case the bulk of the work is not decoding those same base values from the byte representation but rather that of parsing these input values as they are encountered, in order to identify both how to proceed with the next decode and how to build the decoded information into a suitable program model.

If there is any commonality it is that any routine which can be used to encode a basic value (of some known type) will often be matched with a corresponding routine that can be used to decode a basic value (of the same known type). It might be useful to ensure these pairs are provided in a common library, at least for some cases. Perhaps there are further opportunities for refactoring but for now I'll identify what I think exists on the SVM side as far as these two categories are concerned.

The most useful thing to do right away would be to factor out all of the DWARF-specific constants that currently reside in class DwarfDebuginfo (perhaps, relocating them in a class called DwarfConstants?). The obvious set of common elements that could usefully be shared includes:

• DW_TAG_XXX tags identifying different types of DIEs
• DW_AT_XXX attribute ids identifying different attributes that may reside in a DIE
• DW_FORM_XXX ids specifying an available value type for a DWARF attribute
• various predefined DW_XXX attribute values and other constants that can appear in DIEs such as DW_CHILDREN_no, DW_FLAG_true, DW_INL_not_inlined, DW_ACCESS_public, etc.
• DW_CFA_XXX constants for use in defining call frame layouts
• DW_OP_XXX constants used to build DWARF location expressions

These are all named following to the DWARF (4) standard, mostly as int constants (since this is generally what the spec defines) but, in the case of attribute or (DIE/CU) header values occasionally as byte values because that is how they appear in the byte sequence. Clearly, some of these values ought to be unsigned but that is not possible in Java.

It might also be useful to share the standard names used for DWARF sections and, possibly, some of the ELF sections they may need to reference i.e.

    public static final String TEXT_SECTION_NAME = ".text";
    public static final String HEAP_BEGIN_NAME = "__svm_heap_begin";
    public static final String DW_STR_SECTION_NAME = ".debug_str";
    public static final String DW_LINE_SECTION_NAME = ".debug_line";
    . . .

Note that the DW_ABBREV_CODE_XXX constants which appear in this file are all specific to DWARF record layouts employed by SVM so they are unlikely to be of any use for Sulong or Wasm DWARF debug info.

There are a few other useful constants defined in subclasses of DwarfDebugInfo that model specific DWARF sections which might also be relocated to a DwarfConstants class:

1. DwarfARangesSectionImpl defines constants DW_AR_HEADER_SIZE and DW_AR_HEADER_PAD_SIZE
2. DwarfFrameSectionImpl defines constant PADDING_NOPS_ALIGNMENT
3. DwarfInfoSectionImpl defines constant DW_DIE_HEADER_SIZE
4. DwarfLineSectionImpl defines a variety of constants with names like DW_LN_XXX, DW_LNS_XXX and DW_LNE_XXX.

Only the last of these correspond to values defined in the DWARF spec. The first 3 list values implied by the definitions provided in the spec given certain choices for the CU and DIE layout. Those values may not be appropriate if Wasm or Sulong choose to use different layouts. Also, the values specified by DW_LN_LINE_BASE, DW_LN_LINE_RANGE and DW_LN_OPCODE_BASE are (the default) choices used when writing line info and, once again, Wasm or Sulong might choose differently.

Apart from these constants the only other obvious code that might usefully be put into a general purpose library (DwarfUtils?) is the routines in class DwarfSectionImpl which encode a variety of types of value into a byte buffer. The basic write operations start with putByte() and continue through to putUTF8StringBytes. Most (but not all) of these routines might usefully be paired in a library with a corresponding getXXX() routine that reads a data value of the same type.

The same class defines a suite of writeXXX() routines which follow the putXXX routines. These are provided for two purposes. Firstly, they can be called with a null byte array in order to size a write operation. They only insert data when the byte array is not null (they can delegate that decision to writeXXX methods that they call but must always check before calling a putXXX method). Secondly, these methods often write complex sequences of data by sequencing calls to other write or put methods.

In many cases, but perhaps not all, that means that there will be no use for an equivalent 'readXXX()` method. That depends on whether:

1. Sulong or Wasm employ the same composite layouts
2. it is possible to determine in advance that some encoded bytes that are to be read conform to that layout
3. there is some easy way to parameterize the read with a callback (say, a lambda) that installs the component values into the target code model and notifies the caller how many bytes have been read

Some of the other section implementation classes also provide writeXXX routines that might be usefully matched with equivalent readXXX routines. For example

• DwarfAbbrevSectionImpl provides writeAttrType and writeAttrForm
• DwarfFrameSectionImpl provides methods for writing various frame records and their elements (used by the different implementations in DwarframeSectionImplAArch64 and DwarfFrameSectionImplx86_64
• DwarfInfoSectionImpl includes a variety of write routines starting with writeAttrRef4, including writeAttrString and writeULEB
• DwarfLineSectionImpl includes methods to write the line number state transition table starting at writeCopyOp through to writeSpecialOpcode