8244778: Archive full module graph in CDS #80

iklam · 2020-09-08T15:59:33Z

This is the same patch as 8244778-archive-full-module-graph.v03 published in [email protected].

The rest of the review will continue on GitHub. I will add new commits to respond to comments to the above e-mail.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8244778: Archive full module graph in CDS

Reviewers

Erik Joelsson (@erikj79 - Reviewer) ⚠️ Review applies to 89f3327
Coleen Phillimore (@coleenp - Reviewer) ⚠️ Review applies to 89f3327
Lois Foltan (@lfoltan - Reviewer) ⚠️ Review applies to 89f3327
Claes Redestad (@cl4es - Reviewer)
alanb - Reviewer ⚠️ Added manually
mchung - Reviewer ⚠️ Added manually

Download

$ git fetch https://git.openjdk.java.net/jdk pull/80/head:pull/80
$ git checkout pull/80

iklam · 2020-09-08T16:02:05Z

/cc core-libs

bridgekeeper · 2020-09-08T16:02:15Z

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-09-08T16:03:12Z

@iklam
The core-libs label was successfully added.

openjdk · 2020-09-08T16:03:48Z

@iklam The following labels will be automatically applied to this pull request: build hotspot security.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label (add|remove) "label" command.

iklam · 2020-09-08T16:04:17Z

/reviewer add lfoltan,coleenp,alanb,mchung

openjdk · 2020-09-08T16:04:19Z

@iklam
Reviewer lfoltan successfully added.

Reviewer coleenp successfully added.

Reviewer alanb successfully added.

Reviewer mchung successfully added.

openjdk · 2020-09-08T16:24:59Z

@iklam This change now passes all automated pre-integration checks. In addition to the automated checks, the change must also fulfill all project specific requirements

After integration, the commit message will be:

8244778: Archive full module graph in CDS

Reviewed-by: erikj, coleenp, lfoltan, redestad, alanb, mchung

If you would like to add a summary, use the /summary command.
To credit additional contributors, use the /contributor command.
To add additional solved issues, use the /issue command.

There are currently no new commits on the master branch since the last update of the source branch of this PR. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you would like to avoid potential automatic rebasing, specify the current head hash when integrating, like this: /integrate 998ce78e530ccb52a76369ea6f5bdd9a3f90601c.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

mlbridge · 2020-09-08T16:28:48Z

Webrevs

iklam · 2020-09-08T16:41:46Z

In response to Lois Foltain's comments on [email protected]:

Minor nit in moduleEntry.cpp & packageEntry.cpp when dealing with the ModuleEntry's reads list and a PackageEntry's exports list. The names of the methods to write and read those arrays is somewhat confusing.

ModuleEntry::write_archived_entry_array
ModuleEntry::read_archived_entry_array

At first I thought you were reading/writing an array of archived entries, not the array within an archived entry itself. I was trying to think of a better name. Please consider adding a comment at line #400 & line #417 ahead of those methods in moduleEntry.cpp to indicate that they are used for both reading/writing a ModuleEntry's reads list and a PackageEntry's exports list.

I renamed the functions to ModuleEntry's::write_growable_array and ModuleEntry::restore_growable_array, and added comments as you suggested. See commit 4f90e77

// This function is used to archive ModuleEntry::_reads and PackageEntry::_qualified_exports.
// GrowableArray cannot be directly archived, as it needs to be expandable at runtime.
// Write it out as an Array, and convert it back to GrowableArray at runtime.
Array<ModuleEntry*>* ModuleEntry::write_growable_array(GrowableArray<ModuleEntry*>* array) {

A question about this because a user's program can define modules post module initialization via ModuleDescriptor.newModule(). See for example, tests within open/test/hotspot/jtreg/runtime/module/AccessCheck. So all of these tests would trigger check_cds_restrictions() if -Xshare:dump was turned on. Is that a concern?

Arbitrary user code cannot be executed during -Xshare:dump. The only way to do it is to use a JVMTI agent, which requires specifying -XX:+AllowArchivingWithJavaAgent. You can see an example in the GCDuringDump.java test. If the agent tries to define an extra module, it will get an UnsupportedOperationException thrown by check_cds_restrictions().

iklam · 2020-09-08T16:45:28Z

/label remove build,hotspot

iklam · 2020-09-08T16:45:40Z

/cc hotspot-runtime

openjdk · 2020-09-08T16:45:49Z

@iklam
The build label was successfully removed.

The hotspot label was successfully removed.

openjdk · 2020-09-08T16:46:11Z

@iklam
The hotspot-runtime label was successfully added.

iklam · 2020-09-08T16:46:51Z

/label remove security

openjdk · 2020-09-08T16:47:00Z

@iklam
The security label was successfully removed.

erikj79

Build changes look good.

src/hotspot/share/classfile/classLoaderDataShared.cpp

src/hotspot/share/classfile/classLoaderDataShared.hpp

src/hotspot/share/classfile/modules.cpp

src/hotspot/share/classfile/classLoaderDataShared.cpp

lfoltan

Thanks Ioi for addressing my review comments. Overall, looks great!

src/hotspot/share/classfile/moduleEntry.cpp

src/hotspot/share/oops/instanceKlass.cpp

coleenp · 2020-09-09T22:18:49Z

Ok thanks! So many emails ...

cl4es

Excellent work!

Only a few minor comments inline, which you can choose to ignore.

cl4es · 2020-09-12T12:16:04Z

src/hotspot/share/classfile/classLoaderDataShared.cpp

+  assert(DumpSharedSpaces, "must be");
+  assert_valid(loader_data);
+  if (loader_data != NULL) {
+    // We can't create hashtables at dump time because the hashcode dependes on the


dependes -> depends

cl4es · 2020-09-12T12:16:56Z

src/hotspot/share/classfile/classLoaderDataShared.cpp

+  if (loader_data != NULL) {
+    // We can't create hashtables at dump time because the hashcode dependes on the
+    // address of the Symbols, which may be relocated at run time due to ASLR.
+    // So we store the packages/modules in a Arrays. At run time, we create


run time -> runtime
a Arrays -> Arrays

cl4es · 2020-09-12T12:20:23Z

src/hotspot/share/classfile/javaClasses.cpp

  if (klass == SystemDictionary::ClassLoader_klass() ||  // ClassLoader::loader_data is malloc'ed.
-      klass == SystemDictionary::Module_klass() ||       // Module::module_entry is malloc'ed
      // The next 3 classes are used to implement java.lang.invoke, and are not used directly in
      // regular Java code. The implementation of java.lang.invoke uses generated anonymoys classes


pre-existing: anonymoys

cl4es · 2020-09-12T12:23:23Z

src/hotspot/share/classfile/modules.cpp

+  assert(UseSharedSpaces && MetaspaceShared::use_full_module_graph(), "must be");
+
+  // We don't want the classes used by the archived full module graph to be redefined by JVMTI.
+  // Luckily, such classes are loaded in the JVMTI "early" phase, and CDS is disable if a JVMTI


cl4es · 2020-09-12T12:34:06Z

src/hotspot/share/memory/heapShared.cpp

-  {"java/lang/Character$CharacterCache",       "archivedCache"},
-  {"java/util/jar/Attributes$Name",            "KNOWN_NAMES"},
-  {"sun/util/locale/BaseLocale",               "constantBaseLocales"},
+  {"java/lang/Integer$IntegerCache",              0, "archivedCache"},


Could the changes here be simplified or clarified? I think the new field should be a bool, or we could instead introduce a new array for the fields archived only when archiving the full module graph (the field is ignored on iteration over closed_archive_subgraph_entry_fields anyhow)

I split out the new fields into a separate array as you suggested. Also fixed the typos you found. See commit e987110.

…issing TRAPS param

iklam · 2020-09-13T14:44:29Z

/integrate

openjdk · 2020-09-13T14:45:23Z

@iklam Pushed as commit 03a4df0.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Restore looks like this now: ``` 0x0000000106e4dfcc: movk x9, #0x5e4, lsl openjdk#16 0x0000000106e4dfd0: movk x9, #0x1, lsl openjdk#32 0x0000000106e4dfd4: blr x9 0x0000000106e4dfd8: ldp x2, x3, [sp, openjdk#16] 0x0000000106e4dfdc: ldp x4, x5, [sp, openjdk#32] 0x0000000106e4dfe0: ldp x6, x7, [sp, openjdk#48] 0x0000000106e4dfe4: ldp x8, x9, [sp, openjdk#64] 0x0000000106e4dfe8: ldp x10, x11, [sp, openjdk#80] 0x0000000106e4dfec: ldp x12, x13, [sp, openjdk#96] 0x0000000106e4dff0: ldp x14, x15, [sp, openjdk#112] 0x0000000106e4dff4: ldp x16, x17, [sp, openjdk#128] 0x0000000106e4dff8: ldp x0, x1, [sp], openjdk#144 0x0000000106e4dffc: ldp xzr, x19, [sp], openjdk#16 0x0000000106e4e000: ldp x22, x23, [sp, openjdk#16] 0x0000000106e4e004: ldp x24, x25, [sp, openjdk#32] 0x0000000106e4e008: ldp x26, x27, [sp, openjdk#48] 0x0000000106e4e00c: ldp x28, x29, [sp, openjdk#64] 0x0000000106e4e010: ldp x30, xzr, [sp, openjdk#80] 0x0000000106e4e014: ldp x20, x21, [sp], openjdk#96 0x0000000106e4e018: ldur x12, [x29, #-24] 0x0000000106e4e01c: ldr x22, [x12, openjdk#16] 0x0000000106e4e020: add x22, x22, #0x30 0x0000000106e4e024: ldr x8, [x28, openjdk#8] ```

* Use JOL to compute size of AllocObject and enforce this as minimum object size * Include array overhead in total

openjdk bot added the core-libs [email protected] label Sep 8, 2020

openjdk bot added security [email protected] hotspot [email protected] build [email protected] labels Sep 8, 2020

iklam added 4 commits September 8, 2020 09:11

8244778: Archive full module graph in CDS

d2008cb

Update to latest repo (JDK-8251557); added comments

5213a02

Renamed ModuleEntry::write_growable_array

4f90e77

fixed trailing spaces

89f3327

iklam force-pushed the 8244778-archive-full-module-graph branch from eaa9125 to 89f3327 Compare September 8, 2020 16:24

openjdk bot added ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 8, 2020

openjdk bot removed the build [email protected] label Sep 8, 2020

openjdk bot added hotspot-runtime [email protected] and removed hotspot [email protected] labels Sep 8, 2020

openjdk bot removed the security [email protected] label Sep 8, 2020

erikj79 approved these changes Sep 8, 2020

View reviewed changes

coleenp approved these changes Sep 9, 2020

View reviewed changes

lfoltan approved these changes Sep 9, 2020

View reviewed changes

src/hotspot/share/classfile/moduleEntry.cpp Show resolved Hide resolved

src/hotspot/share/oops/instanceKlass.cpp Outdated Show resolved Hide resolved

iklam added 3 commits September 9, 2020 14:44

Merge branch 'master' into 8244778-archive-full-module-graph

8b62dd7

Removed TODO comment referring to JBS issue

92b4202

Feedback from Coleen

e541890

Merge branch 'master' into 8244778-archive-full-module-graph

a1abed8

cl4es approved these changes Sep 12, 2020

View reviewed changes

iklam added 2 commits September 12, 2020 11:32

Claes feedback (typos, refactored archived fields init); also added m…

e987110

…issing TRAPS param

fixed failure in runtime/cds/DeterministicDump.java

97706f5

cl4es approved these changes Sep 13, 2020

View reviewed changes

openjdk bot closed this Sep 13, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 13, 2020

iklam deleted the 8244778-archive-full-module-graph branch February 18, 2021 00:46

RealLucy mentioned this pull request Jan 17, 2022

8278302: [s390] Implement fast-path for ASCII-compatible CharsetEncoders #6738

Closed

3 tasks

scientificware mentioned this pull request Aug 12, 2022

JDK-8292276 : Add named colors from CSS Color Module Level 4 scientificware/jdk#12

Open

RealFYang mentioned this pull request Oct 27, 2022

8295926: RISC-V: C1: Fix LIRGenerator::do_LibmIntrinsic #10867

Closed

3 tasks

JoshuaZhuwj mentioned this pull request Feb 23, 2024

8326541: [AArch64] ZGC C2 load barrier stub considers the length of live registers when spilling registers #17974

Closed

3 tasks

pf0n pushed a commit to pf0n/jdk that referenced this pull request Jul 9, 2025

Use JOL to determine AllocObject overhead and minimum size (openjdk#80)

79e5ebf

* Use JOL to compute size of AllocObject and enforce this as minimum object size * Include array overhead in total

8244778: Archive full module graph in CDS #80

8244778: Archive full module graph in CDS #80

Uh oh!

Conversation

iklam commented Sep 8, 2020 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Download

Uh oh!

iklam commented Sep 8, 2020

Uh oh!

bridgekeeper bot commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

iklam commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mlbridge bot commented Sep 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

iklam commented Sep 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iklam commented Sep 8, 2020

Uh oh!

iklam commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

iklam commented Sep 8, 2020

Uh oh!

openjdk bot commented Sep 8, 2020

Uh oh!

erikj79 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lfoltan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coleenp commented Sep 9, 2020

Uh oh!

cl4es left a comment

Choose a reason for hiding this comment

Uh oh!

cl4es Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

cl4es Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

cl4es Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

cl4es Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

cl4es Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

iklam Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

iklam commented Sep 8, 2020 •

edited by openjdk bot

Loading

openjdk bot commented Sep 8, 2020 •

edited

Loading

mlbridge bot commented Sep 8, 2020 •

edited

Loading

iklam commented Sep 8, 2020 •

edited

Loading