libdrgn: dwarf_index: rework file reporting

Currently, the interface between the DWARF index, libdwfl, and the code
which finds and reports vmlinux/kernel modules is spaghetti. The DWARF
index tracks Dwfl_Modules via their userdata. However, despite
conceptually being owned by the DWARF index, the reporting code reports
the Dwfl_Modules and sets up the userdata. These Dwfl_Modules and
drgn_dwfl_module_userdatas are messy to track and pass between the
layers.

This reworks the architecture so that the DWARF index owns the Dwfl
instance and files are reported to the DWARF index; the DWARF index
takes care of reporting to libdwfl internally. In addition to making the
interface for the reporter much cleaner, this improves a few things as a
side-effect:

- We now deduplicate on build ID in addition to path.
- We now skip searching for vmlinux and/or kernel modules if they were
  already indexed.
- We now support compressed ELF files via libdwelf.
- We can now load default debug info at the same time as additional
  debug info.

Author: osandov

docs/api_reference.rst

@@ -266,33 +266,33 @@ Programs
 
         :param int pid: Process ID.
 
-    .. method:: load_debug_info(paths)
+    .. method:: load_debug_info(paths, default=False)
 
         Load debugging information for a list of executable or library files.
 
-        If an error is encountered while loading any file, no new debugging
-        information is loaded.
-
         Note that this is parallelized, so it is usually faster to load
         multiple files at once rather than one by one.
 
         :param paths: Paths of binary files.
         :type paths: Iterable[str, bytes, or os.PathLike]
+        :param bool default: Also load debugging information which can
+            automatically be determined from the program.
+
+            For the Linux kernel, this tries to load ``vmlinux`` and any loaded
+            kernel modules from a few standard locations.
+
+            For userspace programs, this tries to load the executable and any
+            loaded libraries.
+        :raises MissingDebugInfoError: if debugging information was not
+            available for some files; other files with debugging information
+            are still loaded
 
     .. method:: load_default_debug_info()
 
         Load debugging information which can automatically be determined from
         the program.
 
-        For the Linux kernel, this tries to load ``vmlinux`` and any loaded
-        kernel modules from a few standard locations.
-
-        For userspace programs, this tries to load the executable and any
-        loaded libraries.
-
-        :raises MissingDebugInfoError: if debugging information was not
-            available for some files; other files with debugging information
-            are still loaded if this is raised
+        This is equivalent to ``load_debug_info([], True)``.
 
     .. attribute:: cache
 

drgn/internal/cli.py

@@ -85,18 +85,11 @@ def main() -> None:
         prog.set_pid(args.pid or os.getpid())
     else:
         prog.set_kernel()
-    if args.default_symbols:
-        try:
-            prog.load_default_debug_info()
-        except drgn.MissingDebugInfoError as e:
-            if not args.quiet:
-                print(str(e), file=sys.stderr)
-    if args.symbols:
-        try:
-            prog.load_debug_info(args.symbols)
-        except (drgn.MissingDebugInfoError, OSError) as e:
-            if not args.quiet:
-                print(e, file=sys.stderr)
+    try:
+        prog.load_debug_info(args.symbols or [], args.default_symbols)
+    except drgn.MissingDebugInfoError as e:
+        if not args.quiet:
+            print(str(e), file=sys.stderr)
 
     init_globals: Dict[str, Any] = {'prog': prog}
     if args.script:

libdrgn/drgn.h

@@ -1117,16 +1117,15 @@ struct drgn_error *drgn_program_set_kernel(struct drgn_program *prog);
  */
 struct drgn_error *drgn_program_set_pid(struct drgn_program *prog, pid_t pid);
 
-/** Load debugging information for a list of executable or library files. */
-struct drgn_error *drgn_program_load_debug_info(struct drgn_program *prog,
-						const char **paths, size_t n);
-
 /**
- * Load debugging information which can automatically be determined from the
- * program.
+ * Load debugging information for a list of executable or library files.
+ *
+ * @param[in] load_default Whether to also load debugging information which can
+ * automatically be determined from the program.
  */
-struct drgn_error *
-drgn_program_load_default_debug_info(struct drgn_program *prog);
+struct drgn_error *drgn_program_load_debug_info(struct drgn_program *prog,
+						const char **paths, size_t n,
+						bool load_default);
 
 /**
  * Create a @ref drgn_program from a core dump file.