Skip to content

[GR-43385] Group by modules and JARs in code area breakdown. #5705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 9, 2023
Merged

[GR-43385] Group by modules and JARs in code area breakdown. #5705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
56a17c6
Group by JAR file in code area breakdown.
fniephaus Jan 4, 2023
ab7ca75
Group by module name in code area breakdown.
fniephaus Jan 6, 2023
e06ca9d
Document origins of code area.
fniephaus Jan 6, 2023
0b52fa9
Special case Native Image module.
fniephaus Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 44 additions & 37 deletions docs/reference-manual/native-image/BuildOutput.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,51 +20,50 @@ Below is the example output when building a native executable of the `HelloWorld
================================================================================
GraalVM Native Image: Generating 'helloworld' (executable)...
================================================================================
[1/8] Initializing... (2.5s @ 0.21GB)
Version info: 'GraalVM dev Java 11 CE'
C compiler: gcc (linux, x86_64, 9.3.0)
[1/8] Initializing... (3.3s @ 0.25GB)
Version info: 'GraalVM dev Java 19+36-jvmci-23.0-b01 CE'
Java version info: '19+36-jvmci-23.0-b01'
C compiler: gcc (linux, x86_64, 11.3.0)
Garbage collector: Serial GC (max heap size: unlimited)
[2/8] Performing analysis... [*******] (5.6s @ 0.46GB)
2,718 (72.93%) of 3,727 types reachable
3,442 (53.43%) of 6,442 fields reachable
12,128 (44.82%) of 27,058 methods reachable
27 types, 0 fields, and 271 methods registered for reflection
58 types, 59 fields, and 52 methods registered for JNI access
[2/8] Performing analysis... [****] (6.2s @ 0.47GB)
2,880 (71.50%) of 4,028 types reachable
3,519 (51.06%) of 6,892 fields reachable
13,339 (45.11%) of 29,570 methods reachable
879 types, 0 fields, and 356 methods registered for reflection
57 types, 56 fields, and 52 methods registered for JNI access
4 native libraries: dl, pthread, rt, z
[3/8] Building universe... (0.5s @ 0.61GB)
[4/8] Parsing methods... [*] (0.5s @ 0.86GB)
[5/8] Inlining methods... [****] (0.5s @ 0.73GB)
[6/8] Compiling methods... [**] (3.7s @ 2.38GB)
[7/8] Layouting methods... [*] (0.5s @ 0.71GB)
[8/8] Creating image... (2.1s @ 1.04GB)
4.00MB (28.31%) for code area: 7,073 compilation units
5.90MB (41.70%) for image heap: 83,319 objects and 5 resources
3.24MB (22.91%) for debug info generated in 1.0s
1.00MB ( 7.08%) for other data
14.15MB in total
[3/8] Building universe... (1.1s @ 2.26GB)
[4/8] Parsing methods... [*] (1.0s @ 2.76GB)
[5/8] Inlining methods... [***] (0.8s @ 0.99GB)
[6/8] Compiling methods... [***] (6.4s @ 4.86GB)
[7/8] Layouting methods... [**] (4.2s @ 3.98GB)
[8/8] Creating image... (4.0s @ 2.04GB)
4.52MB (22.97%) for code area: 7,470 compilation units
7.06MB (35.87%) for image heap: 101,764 objects and 5 resources
7.52MB (38.24%) for debug info generated in 1.8s
590.19KB ( 2.93%) for other data
19.68MB in total
--------------------------------------------------------------------------------
Top 10 packages in code area: Top 10 object types in image heap:
632.68KB java.util 871.62KB byte[] for code metadata
324.42KB java.lang 798.53KB java.lang.String
223.90KB java.util.regex 774.91KB byte[] for general heap data
221.62KB java.text 614.06KB java.lang.Class
198.30KB com.oracle.svm.jni 492.51KB byte[] for java.lang.String
166.02KB java.util.concurrent 314.81KB java.util.HashMap$Node
115.44KB java.math 233.58KB c.o.s.c.h.DynamicHubCompanion
98.48KB sun.text.normalizer 154.84KB java.lang.String[]
97.42KB java.util.logging 139.54KB byte[] for embedded resources
95.18KB c.oracle.svm.core.genscavenge 139.04KB char[]
1.83MB for 118 more packages 1.29MB for 753 more object types
Top 10 origins of code area: Top 10 object types in image heap:
3.43MB java.base 1.01MB byte[] for code metadata
760.98KB svm.jar (Native Image) 1000.72KB java.lang.String
102.06KB java.logging 884.18KB byte[] for general heap data
48.03KB org.graalvm.nativeimage.base 686.91KB byte[] for java.lang.String
40.49KB jdk.proxy1 659.87KB java.lang.Class
38.23KB jdk.proxy3 247.50KB c.o.s.c.h.DynamicHubCompanion
25.73KB jdk.internal.vm.ci 239.25KB java.lang.Object[]
23.55KB org.graalvm.sdk 226.08KB java.util.HashMap$Node
11.10KB jdk.proxy2 173.15KB java.lang.String[]
8.10KB jdk.internal.vm.compiler 163.22KB j.u.c.ConcurrentHashMap$Node
1.39KB for 2 more origins 1.70MB for 808 more object types
--------------------------------------------------------------------------------
0.9s (5.6% of total time) in 17 GCs | Peak RSS: 3.22GB | CPU load: 10.87
0.5s (1.8% of total time) in 24 GCs | Peak RSS: 5.62GB | CPU load: 8.92
--------------------------------------------------------------------------------
Produced artifacts:
/home/janedoe/helloworld/helloworld (executable)
/home/janedoe/helloworld/helloworld (executable, debug_info)
/home/janedoe/helloworld/sources (debug_info)
/home/janedoe/helloworld/helloworld (debug_info)
/home/janedoe/helloworld/helloworld.build_artifacts.txt
================================================================================
Finished generating 'helloworld' in 16.2s.
Finished generating 'helloworld' in 27.4s.
```

## Build Stages
Expand Down Expand Up @@ -154,6 +153,14 @@ Debug info is also generated as part of this stage (if requested).
The code area contains machine code produced by the Graal compiler for all reachable methods.
Therefore, reducing the number of [reachable methods](#glossary-reachability) also reduces the size of the code area.

##### <a name="glossary-code-area-origins"></a>Origins of Code Area
To help users understand where the machine code of the code area comes from, the build output shows a breakdown of the top origins.
An origin is a group of Java sources and can be a JAR file, a package name, or a class name, depending on the information available.
The [`java.base` module](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/module-summary.html), for example, contains base classes from the JDK.
The `svm.jar` file, the `org.graalvm.nativeimage.base` module, and similar origins contain internal sources for the Native Image runtime.
To reduce the size of the code area and with that, the total size of the native executable, re-evaluate the dependencies of your application based on the code area breakdown.
Some libraries and frameworks are better prepared for Native Image than others, and newer versions of a library or framework may improve (or worsen) their code footprint.

#### <a name="glossary-image-heap"></a>Image Heap
The heap contains reachable objects such as static application data, metadata, and `byte[]` for different purposes (see below).

Expand Down
50 changes: 41 additions & 9 deletions substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ProgressReporter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import java.lang.management.OperatingSystemMXBean;
import java.lang.reflect.Field;
import java.nio.file.Path;
import java.security.CodeSource;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Comparator;
Expand Down Expand Up @@ -95,6 +96,7 @@
import com.oracle.svm.hosted.image.AbstractImage.NativeImageKind;
import com.oracle.svm.hosted.image.NativeImageHeap.ObjectInfo;
import com.oracle.svm.hosted.meta.HostedMetaAccess;
import com.oracle.svm.hosted.meta.HostedMethod;
import com.oracle.svm.hosted.reflect.ReflectionHostedSupport;
import com.oracle.svm.hosted.util.VMErrorReporter;
import com.oracle.svm.util.ImageBuildStatistics;
Expand All @@ -109,8 +111,6 @@ public class ProgressReporter {
private static final boolean IS_CI = System.console() == null || System.getenv("CI") != null;
private static final boolean IS_DUMB_TERM = isDumbTerm();
private static final int MAX_NUM_BREAKDOWN = 10;
private static final String CODE_BREAKDOWN_TITLE = String.format("Top %d packages in code area:", MAX_NUM_BREAKDOWN);
private static final String HEAP_BREAKDOWN_TITLE = String.format("Top %d object types in image heap:", MAX_NUM_BREAKDOWN);
private static final String STAGE_DOCS_URL = "https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md";
private static final double EXCESSIVE_GC_MIN_THRESHOLD_MILLIS = 15_000;
private static final double EXCESSIVE_GC_RATIO = 0.5;
Expand Down Expand Up @@ -490,15 +490,45 @@ public void createBreakdowns(HostedMetaAccess metaAccess, Collection<CompileTask

private void calculateCodeBreakdown(Collection<CompileTask> compilationTasks) {
for (CompileTask task : compilationTasks) {
String classOrPackageName = task.method.format("%H");
int lastDotIndex = classOrPackageName.lastIndexOf('.');
if (lastDotIndex > 0) {
classOrPackageName = classOrPackageName.substring(0, lastDotIndex);
String key = null;
Class<?> javaClass = task.method.getDeclaringClass().getJavaClass();
Module module = javaClass.getModule();
if (module.isNamed()) {
key = module.getName();
if ("org.graalvm.nativeimage.builder".equals(key)) {
key = "svm.jar (Native Image)";
}
} else {
key = findJARFile(javaClass);
if (key == null) {
key = findPackageOrClassName(task.method);
}
}
codeBreakdown.merge(classOrPackageName, (long) task.result.getTargetCodeSize(), Long::sum);
codeBreakdown.merge(key, (long) task.result.getTargetCodeSize(), Long::sum);
}
}

private static String findJARFile(Class<?> javaClass) {
CodeSource codeSource = javaClass.getProtectionDomain().getCodeSource();
if (codeSource != null && codeSource.getLocation() != null) {
String path = codeSource.getLocation().getPath();
if (path.endsWith(".jar")) {
// Use String API to determine basename of path to handle both / and \.
return path.substring(Math.max(path.lastIndexOf('/') + 1, path.lastIndexOf('\\') + 1));
}
}
return null;
}

private static String findPackageOrClassName(HostedMethod method) {
String qualifier = method.format("%H");
int lastDotIndex = qualifier.lastIndexOf('.');
if (lastDotIndex > 0) {
qualifier = qualifier.substring(0, lastDotIndex);
}
return qualifier;
}

private void calculateHeapBreakdown(HostedMetaAccess metaAccess, Collection<ObjectInfo> heapObjects) {
long stringByteLength = 0;
for (ObjectInfo o : heapObjects) {
Expand Down Expand Up @@ -558,7 +588,9 @@ private void printBreakdowns() {
.sorted(Entry.comparingByValue(Comparator.reverseOrder())).iterator();

final TwoColumnPrinter p = new TwoColumnPrinter();
p.l().yellowBold().a(CODE_BREAKDOWN_TITLE).jumpToMiddle().a(HEAP_BREAKDOWN_TITLE).reset().flushln();
p.l().yellowBold().a(String.format("Top %d ", MAX_NUM_BREAKDOWN)).doclink("origins", "#glossary-code-area-origins").a(" of code area:")
.jumpToMiddle()
.a(String.format("Top %d object types in image heap:", MAX_NUM_BREAKDOWN)).reset().flushln();

long printedCodeBytes = 0;
long printedHeapBytes = 0;
Expand Down Expand Up @@ -596,7 +628,7 @@ private void printBreakdowns() {
int numHeapItems = heapBreakdown.size();
long totalCodeBytes = codeBreakdown.values().stream().collect(Collectors.summingLong(Long::longValue));
long totalHeapBytes = heapBreakdown.values().stream().collect(Collectors.summingLong(Long::longValue));
p.l().a(String.format("%9s for %s more packages", Utils.bytesToHuman(totalCodeBytes - printedCodeBytes), numCodeItems - printedCodeItems))
p.l().a(String.format("%9s for %s more ", Utils.bytesToHuman(totalCodeBytes - printedCodeBytes), numCodeItems - printedCodeItems)).doclink("origins", "#glossary-code-area-origins")
.jumpToMiddle()
.a(String.format("%9s for %s more object types", Utils.bytesToHuman(totalHeapBytes - printedHeapBytes), numHeapItems - printedHeapItems)).flushln();
}
Expand Down