Skip to content

Commit 348be32

Browse files
dominikmascherbauerolpaw
authored andcommitted
Added GDB Python script (gdb-debughelpers.py) to improve the Native Image debugging experience
1 parent 87db358 commit 348be32

File tree

18 files changed

+3538
-4
lines changed

18 files changed

+3538
-4
lines changed

docs/reference-manual/native-image/guides/debug-native-executables-with-gdb.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -564,4 +564,5 @@ void svm_dbg_print_locationInfo(graal_isolatethread_t* thread, size_t mem);
564564

565565
### Related Documentation
566566

567-
* [Debug Info Feature](../DebugInfo.md)
567+
* [Debug Info Feature](../DebugInfo.md)
568+
* [Debug Native Executables with a Python Helper Script](debug-native-executables-with-python-helper.md)
Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
---
2+
layout: ni-docs
3+
toc_group: how-to-guides
4+
link_title: Debug Native Executables with a Python Helper Script
5+
permalink: /reference-manual/native-image/guides/debug-native-image-process-with-python-helper-script/
6+
---
7+
# Debug Native Executables with a Python Helper Script
8+
9+
Additionally to the [GDB debugging](debug-native-executables-with-gdb.md), you can debug a `native-image` process using a Python helper script, _gdb-debughelpers.py_.
10+
The [GDB Python API](https://sourceware.org/gdb/current/onlinedocs/gdb/Python.html) is used to provide a reasonably good experience for debugging native executables or shared libraries.
11+
It requires GDB with Python support.
12+
The debugging extension is tested against GDB 13.2 and supports the new debuginfo generation introduced in GraalVM for JDK 17 and later.
13+
14+
> Note: The _gdb-debughelpers.py_ file does not work with versions older than version 13.2 of `gdb` or versions older than GraalVM for JDK 17.
15+
16+
The Python script _gdb-debughelpers.py_ can be found in the _<GRAALVM\_HOME>/lib/svm/debug_ directory.
17+
If debuginfo generation is enabled (see [Build a Native Executable with Debug Information](debug-native-executables-with-gdb.md#build-a-native-executable-with-debug-information)), the script is copied to the build directory.
18+
The `native-image` tool adds the debugging section `.debug_gdb_scripts` to the debug info file, which causes GDB to automatically load _gdb-debughelpers.py_ from the current working directory.
19+
20+
For [security reasons](https://sourceware.org/gdb/current/onlinedocs/gdb/Auto_002dloading-safe-path.html)
21+
the first time GDB encounters a native executable or shared library that requests a specific Python file to be loaded it will print a warning:
22+
23+
> warning: File "<CWD>/gdb-debughelpers.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
24+
>
25+
> To enable execution of this file add
26+
>         add-auto-load-safe-path <CWD>/gdb-debughelpers.py
27+
> line to your configuration file "<HOME>/.gdbinit".
28+
> To completely disable this security protection add
29+
>         add-auto-load-safe-path /
30+
> line to your configuration file "<HOME>/.gdbinit".
31+
> For more information about this security protection see the
32+
> "Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
33+
>         info "(gdb)Auto-loading safe path"
34+
35+
To solve this, either add the current working directory to _~/.gdbinit_ as follows:
36+
37+
echo "add-auto-load-safe-path <CWD>/gdb-debughelpers.py" >> ~/.gdbinit
38+
39+
or pass the path as a command line argument to `gdb`:
40+
41+
gdb -iex "set auto-load safe-path <CWD>/gdb-debughelpers.py" <binary-name>
42+
43+
Both enable GDB to auto-load _gdb-debughelpers.py_ from the current working directory.
44+
45+
Auto-loading is the recommended way to provide the script to GDB.
46+
However, it is possible to manually load the script from GDB explicitly with:
47+
48+
(gdb) source gdb-debughelpers.py
49+
50+
## Pretty Printing Support
51+
52+
Loading _gdb-debughelpers.py_ registers a new pretty printer to GDB, which adds an extra level of convenience for debugging native executables or shared libraries.
53+
This pretty printer handles the printing of Java Objects, Arrays, Strings, and Enums for debugging native executables or shared libraries.
54+
If the Java application uses `@CStruct` and `@CPointer` annotations to access C data structures, the pretty printer will also try to print them as if they were Java data structures.
55+
If the C data structures cannot be printed by the pretty printer, printing is performed by GDB.
56+
57+
The pretty printer also prints of the primitive value of a boxed primitive (instead of a Java Object).
58+
59+
Whenever printing is done via the `p` alias of the `print` command the pretty printer intercepts that call to perform type casts to the respective runtime types of Java Objects.
60+
This also applies for auto-completion when using the `p` alias.
61+
This means that if the static type is different to the runtime type, the `print` command uses the static type, which leaves the user to discover the runtime type and typecast it.
62+
Additionally, the `p` alias understands Java field and array access and function calls for Java Objects.
63+
64+
#### Limitations
65+
66+
The `print` command still uses its default implementation, as there is no way to overwrite it, while still keeping the capability of the default `print` command.
67+
Overriding would cause printing non-Java Objects to not work properly.
68+
Therefore, only the `p` alias for the `print` command is overwritten by the pretty printer, such that the user can still make use of the default GDB `print` command.
69+
70+
### Options to Control the Pretty Printer Behavior
71+
72+
In addition to the enhanced `p` alias, _gdb-debughelpers.py_ introduces some GDB parameters to customize the behavior of the pretty printer.
73+
Parameters in GDB can be controlled with `set <param> <value>` and `show <param>` commands, and thus integrate with GDB's customization options.
74+
75+
* #### svm-print on/off
76+
77+
Use this command to enable/disable the pretty printer.
78+
This also resets the `print` command alias `p` to its default behavior.
79+
Alternatively pretty printing can be suppressed with the
80+
[`raw` printing option of GDB's `print` command](https://sourceware.org/gdb/current/onlinedocs/gdb/Output-Formats.html):
81+
82+
(gdb) show svm-print
83+
The current value of 'svm-print' is "on".
84+
85+
(gdb) print str
86+
$1 = "string"
87+
88+
(gdb) print/r str
89+
$2 = (java.lang.String *) 0x7ffff689d2d0
90+
91+
(gdb) set svm-print off
92+
1 printer disabled
93+
1 of 2 printers enabled
94+
95+
(gdb) print str
96+
$3 = (java.lang.String *) 0x7ffff689d2d0
97+
98+
* #### svm-print-string-limit &lt;int&gt;
99+
100+
Customizes the maximum length for pretty printing a Java String.
101+
The default value is `200`.
102+
Set to `-1` or `unlimited` for unlimited printing of a Java String.
103+
This does not change the limit for a C String, which can be controlled with GDB's `set print characters` command.
104+
105+
* #### svm-print-element-limit &lt;int&gt;
106+
107+
Customizes the maximum number of elements for pretty printing a Java Array, ArrayList, and HashMap.
108+
The default value is `10`.
109+
Set to `-1` or `unlimited` to print an unlimited number of elements.
110+
This does not change the limit for a C array, which can be controlled with GDB's `set print elements` command.
111+
However, GDB's parameter `print elements` is the upper bound for `svm-print-element-limit`.
112+
113+
* #### svm-print-field-limit &lt;int&gt;
114+
115+
Customizes the maximum number of elements for pretty printing fields of a Java Object.
116+
The default value is `50`.
117+
Set to `-1` or `unlimited` to print an unlimited number of fields.
118+
GDB's parameter `print elements` is the upper bound for `svm-print-field-limit`.
119+
120+
* #### svm-print-depth-limit &lt;int&gt;
121+
122+
Customizes the maximum depth of recursive pretty printing.
123+
The default value is `1`.
124+
The children of direct children are printed (a sane default to make contents of boxed values visible).
125+
Set to `-1` or `unlimited` to print unlimited depth.
126+
GDB's parameter `print max-depth` is the upper bound for `svm-print-depth-limit`.
127+
128+
* #### svm-use-hlrep on/off
129+
130+
Enables/disables pretty printing for higher level representations.
131+
It provides a more data-oriented view on some Java data structures with a known internal structure such as Lists or Maps.
132+
Currently supports ArrayList and HashMap.
133+
134+
* #### svm-infer-generics &lt;int&gt;
135+
136+
Customizes the number of elements taken into account to infer the generic type of higher level representations.
137+
The default value is `10`.
138+
Set to `0` to not infer generic types and `-1` or `unlimited` to infer the generic type of all elements.
139+
140+
* #### svm-print-address absolute/on/off
141+
142+
Enables/disables printing of addresses in addition to regular pretty printing.
143+
When `absolute` mode is used even compressed references are shown as absolute addresses.
144+
Printing addresses is disabled by default.
145+
146+
* #### svm-print-static-fields on/off
147+
148+
Enables/disables printing of static fields for a Java Object.
149+
Printing static fields is disabled by default.
150+
151+
* #### svm-complete-static-variables on/off
152+
153+
Enables/disables auto-completion of static field members for the enhanced `p` alias.
154+
Auto-completion of static fields is enabled by default.
155+
156+
* #### svm-selfref-check on/off
157+
158+
Enables/disables self-reference checks for data structures.
159+
The pretty printer detects a self-referential data structure and prevents further expansion to avoid endless recursion.
160+
Self-reference checks are enabled by default.
161+
For testing, this feature can be temporary disabled (usually you wouldn't want to do this).
162+
163+
### Related Documentation
164+
165+
* [Debug Info Feature](../DebugInfo.md)
166+
* [Debug Native Executables with GDB](debug-native-executables-with-gdb.md)

docs/reference-manual/native-image/guides/guides.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Here you will learn how to:
2525
- [Containerize a Native Executable and Run in a Docker Container](containerise-native-executable-with-docker.md)
2626
- [Create a Heap Dump from a Native Executable](create-heap-dump-from-native-executable.md)
2727
- [Debug Native Executables with GDB](debug-native-executables-with-gdb.md)
28+
- [Debug Native Executables with a Python Helper Script](debug-native-executables-with-python-helper.md)
2829
- [Include Reachability Metadata Using the Native Image Gradle Plugin](include-reachability-metadata-gradle.md)
2930
- [Include Reachability Metadata Using the Native Image Maven Plugin](include-reachability-metadata-maven.md)
3031
- [Include Resources in a Native Executable](include-resources.md)

substratevm/CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22

33
This changelog summarizes major changes to GraalVM Native Image.
44

5+
## GraalVM for JDK 24 (Internal Version 24.2.0)
6+
* (GR-48384) Added a GDB Python script (`gdb-debughelpers.py`) to improve the Native Image debugging experience.
7+
58
## GraalVM for JDK 23 (Internal Version 24.1.0)
69
* (GR-51520) The old class initialization strategy, which was deprecated in GraalVM for JDK 22, is removed. The option `StrictImageHeap` no longer has any effect.
710
* (GR-51106) Fields that are accessed via a `VarHandle` or `MethodHandle` are no longer marked as "unsafe accessed" when the `VarHandle`/`MethodHandle` can be fully intrinsified.

substratevm/mx.substratevm/mx_substratevm.py

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -986,6 +986,125 @@ def build_debug_test(variant_name, image_name, extra_args):
986986
os.environ.update({'debuginfotest_isolates' : 'yes'})
987987
mx.run([os.environ.get('GDB_BIN', 'gdb'), '-ex', 'python "ISOLATES=True"', '-x', gdb_utils_py, '-x', testhello_py, hello_binary])
988988

989+
990+
def _gdbdebughelperstest(native_image, path, with_isolates_only, build_only, test_only, args):
991+
test_proj = mx.dependency('com.oracle.svm.test')
992+
test_source_path = test_proj.source_dirs()[0]
993+
tutorial_proj = mx.dependency('com.oracle.svm.tutorial')
994+
tutorial_c_source_dir = join(tutorial_proj.dir, 'native')
995+
tutorial_source_path = tutorial_proj.source_dirs()[0]
996+
997+
gdbdebughelpers_py = join(mx.dependency('com.oracle.svm.hosted.image.debug').output_dir(), 'gdb-debughelpers.py')
998+
999+
test_python_source_dir = join(test_source_path, 'com', 'oracle', 'svm', 'test', 'debug', 'helper')
1000+
test_pretty_printer_py = join(test_python_source_dir, 'test_pretty_printer.py')
1001+
test_cinterface_py = join(test_python_source_dir, 'test_cinterface.py')
1002+
test_class_loader_py = join(test_python_source_dir, 'test_class_loader.py')
1003+
test_settings_py = join(test_python_source_dir, 'test_settings.py')
1004+
test_svm_util_py = join(test_python_source_dir, 'test_svm_util.py')
1005+
1006+
test_pretty_printer_args = [
1007+
'-cp', classpath('com.oracle.svm.test'),
1008+
# We do not want to step into class initializer, so initialize everything at build time.
1009+
'--initialize-at-build-time=com.oracle.svm.test.debug.helper',
1010+
'com.oracle.svm.test.debug.helper.PrettyPrinterTest'
1011+
]
1012+
test_cinterface_args = [
1013+
'--shared',
1014+
'-Dcom.oracle.svm.tutorial.headerfile=' + join(tutorial_c_source_dir, 'mydata.h'),
1015+
'-cp', tutorial_proj.output_dir()
1016+
]
1017+
test_class_loader_args = [
1018+
'-cp', classpath('com.oracle.svm.test'),
1019+
'-Dsvm.test.missing.classes=' + classpath('com.oracle.svm.test.missing.classes'),
1020+
'--initialize-at-build-time=com.oracle.svm.test.debug.helper',
1021+
# We need the static initializer of the ClassLoaderTest to run at image build time
1022+
'--initialize-at-build-time=com.oracle.svm.test.missing.classes',
1023+
'com.oracle.svm.test.debug.helper.ClassLoaderTest'
1024+
]
1025+
1026+
gdb_args = [
1027+
os.environ.get('GDB_BIN', 'gdb'),
1028+
'--nx',
1029+
'-q', # do not print the introductory and copyright messages
1030+
'-iex', 'set logging overwrite on',
1031+
'-iex', 'set logging redirect on',
1032+
'-iex', 'set logging enabled on',
1033+
]
1034+
1035+
def run_debug_test(image_name: str, testfile: str, source_path: str, with_isolates: bool = True,
1036+
build_cinterfacetutorial: bool = False, extra_args: list[str] = None, skip_build: bool = False):
1037+
extra_args = [] if extra_args is None else extra_args
1038+
build_dir = join(path, image_name + ("" if with_isolates else "_no_isolates"))
1039+
1040+
if not test_only and not skip_build:
1041+
# clean / create output directory
1042+
if exists(build_dir):
1043+
mx.rmtree(build_dir)
1044+
mx.ensure_dir_exists(build_dir)
1045+
1046+
build_args = args + [
1047+
'-H:CLibraryPath=' + source_path,
1048+
'--native-image-info',
1049+
'-Djdk.graal.LogFile=graal.log',
1050+
'-g', '-O0',
1051+
] + svm_experimental_options([
1052+
'-H:+VerifyNamingConventions',
1053+
'-H:+SourceLevelDebug',
1054+
'-H:+IncludeDebugHelperMethods',
1055+
'-H:DebugInfoSourceSearchPath=' + source_path,
1056+
]) + extra_args
1057+
1058+
if not with_isolates:
1059+
build_args += svm_experimental_options(['-H:-SpawnIsolates'])
1060+
1061+
if build_cinterfacetutorial:
1062+
build_args += ['-o', join(build_dir, 'lib' + image_name)]
1063+
else:
1064+
build_args += ['-o', join(build_dir, image_name)]
1065+
1066+
mx.log(f"native_image {' '.join(build_args)}")
1067+
native_image(build_args)
1068+
1069+
if build_cinterfacetutorial:
1070+
if mx.get_os() != 'windows':
1071+
c_command = ['cc', '-g', join(tutorial_c_source_dir, 'cinterfacetutorial.c'),
1072+
'-I.', '-L.', '-lcinterfacetutorial',
1073+
'-ldl', '-Wl,-rpath,' + build_dir,
1074+
'-o', 'cinterfacetutorial']
1075+
1076+
else:
1077+
c_command = ['cl', '-MD', join(tutorial_c_source_dir, 'cinterfacetutorial.c'), '-I.',
1078+
'libcinterfacetutorial.lib']
1079+
mx.log(' '.join(c_command))
1080+
mx.run(c_command, cwd=build_dir)
1081+
if not build_only and mx.get_os() == 'linux':
1082+
# copying the most recent version of gdb-debughelpers.py (even if the native image was not built)
1083+
mx.log(f"Copying {gdbdebughelpers_py} to {build_dir}")
1084+
mx.copyfile(gdbdebughelpers_py, join(build_dir, 'gdb-debughelpers.py'))
1085+
1086+
gdb_command = gdb_args + [
1087+
'-iex', f"set auto-load safe-path {join(build_dir, 'gdb-debughelpers.py')}",
1088+
'-x', testfile, join(build_dir, image_name)
1089+
]
1090+
mx.log(' '.join(gdb_command))
1091+
# unittest may result in different exit code, nonZeroIsFatal ensures that we can go on with other test
1092+
mx.run(gdb_command, cwd=build_dir, nonZeroIsFatal=False)
1093+
1094+
if not with_isolates_only:
1095+
run_debug_test('prettyPrinterTest', test_pretty_printer_py, test_source_path, False,
1096+
extra_args=test_pretty_printer_args)
1097+
run_debug_test('prettyPrinterTest', test_pretty_printer_py, test_source_path, extra_args=test_pretty_printer_args)
1098+
run_debug_test('prettyPrinterTest', test_settings_py, test_source_path, extra_args=test_pretty_printer_args, skip_build=True)
1099+
run_debug_test('prettyPrinterTest', test_svm_util_py, test_source_path, extra_args=test_pretty_printer_args, skip_build=True)
1100+
1101+
run_debug_test('cinterfacetutorial', test_cinterface_py, tutorial_source_path, build_cinterfacetutorial=True,
1102+
extra_args=test_cinterface_args)
1103+
1104+
run_debug_test('classLoaderTest', test_class_loader_py, test_source_path, extra_args=test_class_loader_args)
1105+
1106+
1107+
9891108
def _javac_image(native_image, path, args=None):
9901109
args = [] if args is None else args
9911110
mx_util.ensure_dir_exists(path)
@@ -1520,6 +1639,30 @@ def debuginfotestshared(args, config=None):
15201639
# ideally we ought to script a gdb run
15211640
native_image_context_run(_cinterfacetutorial, all_args)
15221641

1642+
@mx.command(suite_name=suite.name, command_name='gdbdebughelperstest', usage_msg='[options]')
1643+
def gdbdebughelperstest(args, config=None):
1644+
"""
1645+
builds and tests gdb-debughelpers.py with multiple native images with debuginfo
1646+
"""
1647+
parser = ArgumentParser(prog='mx gdbdebughelperstest')
1648+
all_args = ['--output-path', '--with-isolates-only', '--build-only', '--test-only']
1649+
masked_args = [_mask(arg, all_args) for arg in args]
1650+
parser.add_argument(all_args[0], metavar='<output-path>', nargs=1, help='Path of the generated image', default=[join(svmbuild_dir(), "gdbdebughelperstest")])
1651+
parser.add_argument(all_args[1], action='store_true', help='Only build and test the native image with isolates')
1652+
parser.add_argument(all_args[2], action='store_true', help='Only build the native image')
1653+
parser.add_argument(all_args[3], action='store_true', help='Only run the tests')
1654+
parser.add_argument('image_args', nargs='*', default=[])
1655+
parsed = parser.parse_args(masked_args)
1656+
output_path = unmask(parsed.output_path)[0]
1657+
with_isolates_only = parsed.with_isolates_only
1658+
build_only = parsed.build_only
1659+
test_only = parsed.test_only
1660+
native_image_context_run(
1661+
lambda native_image, a:
1662+
_gdbdebughelperstest(native_image, output_path, with_isolates_only, build_only, test_only, a), unmask(parsed.image_args),
1663+
config=config
1664+
)
1665+
15231666
@mx.command(suite_name=suite.name, command_name='helloworld', usage_msg='[options]')
15241667
def helloworld(args, config=None):
15251668
"""
@@ -1878,6 +2021,17 @@ def isJDKDependent(self):
18782021
return True
18792022

18802023

2024+
class GDBDebugHelpers(mx.ArchivableProject):
2025+
def output_dir(self):
2026+
return os.path.join(self.dir, 'src', self.name, 'gdbpy')
2027+
2028+
def archive_prefix(self):
2029+
return ''
2030+
2031+
def getResults(self):
2032+
return [os.path.join(self.output_dir(), 'gdb-debughelpers.py')]
2033+
2034+
18812035
class SubstrateCompilerFlagsBuilder(mx.ArchivableProject):
18822036

18832037
flags_build_dependencies = [

0 commit comments

Comments
 (0)