8252103: Parallel heap inspection for ParallelScavengeHeap

[linzang]

• Parallel heap iteration support for PSS
• JBS: https://bugs.openjdk.java.net/browse/JDK-8252103

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Testing

	Linux x64	Linux x86	Windows x64	macOS x64
Build	✔️ (5/5 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)
Test (tier1)	✔️ (9/9 passed)	✔️ (9/9 passed)	✔️ (9/9 passed)	✔️ (9/9 passed)

Issue

• JDK-8252103: Parallel heap inspection for ParallelScavengeHeap

Reviewers

• Stefan Johansson (@kstefanj - Reviewer) ⚠️ Review applies to 6b4600a
• Thomas Schatzl (@tschatzl - Reviewer) ⚠️ Review applies to ee92086

Download

$ git fetch https://git.openjdk.java.net/jdk pull/25/head:pull/25
$ git checkout pull/25

[bridgekeeper]

👋 Welcome back lzang! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@linzang The following labels will be automatically applied to this pull request: hotspot-gc. When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label (add|remove) "label" command.

[mlbridge]

Webrevs

• 15: Full - Incremental (4cb0866)
• 14: Full - Incremental (6b4600a)
• 13: Full - Incremental (ee92086)
• 12: Full - Incremental (2668fe6)
• 11: Full - Incremental (dacae05)
• 10: Full - Incremental (046638c)
• 09: Full - Incremental (cb75521)
• 08: Full - Incremental (bd06f91)
• 07: Full - Incremental (161b595)
• 06: Full - Incremental (b17c822)
• 05: Full - Incremental (2d432f9)
• 04: Full - Incremental (d1ca71d)
• 03: Full - Incremental (2878fc0)
• 02: Full - Incremental (05f3c64f701f2ab87a16a32addf7f0b805049601)
• 01: Full - Incremental (3672513aaf006b5a320409cc81889816241fc1c1)
• 00: Full (23e09ec4efedfebb199f44f55ac6fa4543c24f33)

[linzang]

/label add "serviceablility"

[openjdk]

@linzang Usage: /label <add|remove> [label[, label, ...]]wherelabel` is an additional classification that should be applied to this PR. These labels are valid:

• serviceability
• hotspot
• sound
• hotspot-compiler
• kulla
• i18n
• shenandoah
• jdk
• javadoc
• 2d
• security
• swing
• hotspot-runtime
• jmx
• build
• nio
• beans
• core-libs
• compiler
• net
• hotspot-gc
• hotspot-jfr
• awt

[linzang]

/label add serviceability

[openjdk]

@linzang
The serviceability label was successfully added.

[linzang]

Dear All,
May I ask your help to review this PR? Thanks!
-Lin

[bridgekeeper]

@linzang This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[kstefanj]

Hi Lin,

Sorry for not getting to this sooner. One of the reasons is that I haven't had time to explore a better solution, but the above notification triggered me to at least share my thoughts.

I would like us to avoid having the logic that check how many workers are doing the work and instead have some mechanism that claim different chunks of work. You can compare it a bit to G1 where we have the HeapRegionClaimer that make sure only one thread handles a given region. This claimer needs to be a bit different and allow claiming of eden, to-space, from-space and then multiple chunks of old. But I believe such solution would both be more efficient (since all threads can help out on old in the end) and easier to follow (no special cases).

So basically the PSScavengeParallelObjectIterator need to set up this claimer and pass it down to the workers and all workers than try to do all work, but only the one getting the claim will do the actual work. What do you think about this approach? Do you understand what I'm after?

[linzang]

Hi @kstefanj, Thanks a lot for your comments.
I think there can be a reclaimer that treat eden space, from space, to space and "blocks" in old space as parallel iteration candidates, then every workers try to claim their ownership of the candidate that it is going to scan. So there is no need to check worker numbers and Id for processing different spaces.
what do you think?

BTW, Paul ([email protected]) also helped review this patch and send the comment in mail list, I will paste his comments here and then try to polish a patch with all of your comments considered.

[linzang]

Here is the review comments from Paul, copied from hotspot-gc-dev digest:

Message: 3
Date: Mon, 28 Sep 2020 22:39:38 +0000
From: "Hohensee, Paul" [email protected]
To: Lin Zang [email protected],
"[email protected]" [email protected],
"[email protected]"
[email protected]
Subject: RE: RFR: 8252103: Parallel heap inspection for
ParallelScavengeHeap
Message-ID: [email protected]
Content-Type: text/plain; charset="utf-8"

I'm not a GC specialist, but your approach looks reasonable to me.

parallelScavengeHeap.cpp:

"less that 1 workers" -> "less that 1 worker"

psOldGen.cpp:

">=2" -> ">= 2"

"thread_num-2 worker" -> "(thread_num -2) workers"

"cross blocks" -> "crosses blocks"

"that the object start address locates in the related block" -> "that owns the block within which the object starts"

blocks_iterate_parallel() relies on BLOCK_SIZE being an integral multiple of a start_array() block (i.e., ObjectStartArray::block_size). I'd add an assert to that effect.

The comment on the definition of BLOCK_SIZE is "64 KB", but you're doing pointer arithmetic in terms of HeapWords, which > means the actual byte value is either 256KB or 512KB for 32- and 64-bit respectively. You could use

const int BLOCK_SIZE = (64 * 1024 / HeapWordSize);

"// There is objects" -> "There are objects"

"reture" -> "return"

", here only focus objects" -> ". Here only focus on objects"

It's a matter of taste, but imo the loop in blocks_iterate_parallel() would be more clear as

for (HeapWord* p = start; p < end; p += oop(p)->size()) {
cl->do_object(oop(p));
}

psOldGen.hpp:

blocks_iterate_parallel() is pure implementation and I don't see a reference outside psOldGen.cpp. So, it should be private, > not public.

psYoungGen.cpp:

"<=1" -> "<= 1"

Thanks,
Paul

Thanks for Paul's help and effort for reviewing, I will make a refined patch based on your comments.

[linzang]

Dear @stefank,
I have update this PR that use a claimer to help worker thread do parallel iteration. would you like to help review again?
Thanks,
Lin

[openjdk]

@linzang Usage: /label <add|remove> [label[, label, ...]]wherelabel` is an additional classification that should be applied to this PR. These labels are valid:

• serviceability
• hotspot
• sound
• hotspot-compiler
• kulla
• i18n
• shenandoah
• jdk
• javadoc
• 2d
• security
• swing
• hotspot-runtime
• jmx
• build
• nio
• beans
• core-libs
• compiler
• net
• hotspot-gc
• hotspot-jfr
• awt

[openjdk]

@linzang The serviceability label was already applied.

[openjdk]

@linzang To determine the appropriate audience for reviewing this pull request, one or more labels corresponding to different subsystems will normally be applied automatically. However, no automatic labelling rule matches the changes in this pull request. In order to have an "RFR" email sent to the correct mailing list, you will need to add one or more applicable labels manually using the /label pull request command.

Applicable Labels

• 2d
• awt
• beans
• build
• compiler
• core-libs
• hotspot
• hotspot-compiler
• hotspot-gc
• hotspot-jfr
• hotspot-runtime
• i18n
• javadoc
• jdk
• jmx
• kulla
• net
• nio
• security
• serviceability
• shenandoah
• sound
• swing

[openjdk]

@linzang Usage: /label <add|remove> [label[, label, ...]]wherelabel` is an additional classification that should be applied to this PR. These labels are valid:

• serviceability
• hotspot
• sound
• hotspot-compiler
• kulla
• i18n
• shenandoah
• jdk
• javadoc
• 2d
• security
• swing
• hotspot-runtime
• jmx
• build
• nio
• beans
• core-libs
• compiler
• net
• hotspot-gc
• hotspot-jfr
• awt

[openjdk]

@linzang
The serviceability label was successfully added.

[openjdk]

@linzang Usage: /label <add|remove> [label[, label, ...]]wherelabel` is an additional classification that should be applied to this PR. These labels are valid:

• serviceability
• hotspot
• sound
• hotspot-compiler
• kulla
• i18n
• shenandoah
• jdk
• javadoc
• 2d
• security
• swing
• hotspot-runtime
• jmx
• build
• nio
• beans
• core-libs
• compiler
• net
• hotspot-gc
• hotspot-jfr
• awt

[openjdk]

@linzang The serviceability label was already applied.

[tschatzl]

I recommend changing the (size_t)-1 assignment to SIZE_MAX.

While there is (to my knowledge) no compiler (that supports the JVM) that does not use two's complement for integer representation, officially only with C++20 (or later) the result of this operation is exactly defined. We know that other code just does the cast as this does ;), but if you use SIZE_MAX, the comment above the definition of InvalidIndex can also be removed then.

I would also move InvalidIndex as the first in the list of constants because it's not really related to numbers assigned because of the spaces (while NumNonOldGenClaims is).

Lgtm otherwise. Thanks for bearing with us.

[linzang]

Dear @tschatzl @kstefanj and @albertnetymk,
So appericated for your suggestion and guidance. I have upload a new commit.
would you like to help review it again.

BRs,
Lin

[kstefanj]

I'm good with the current state of the code. Will do some additional testing before being ready to sponsor though.

[openjdk]

@linzang This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8252103: Parallel heap inspection for ParallelScavengeHeap

Reviewed-by: sjohanss, tschatzl

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 119 new commits pushed to the master branch:

• cdef186: 8232856: jshell crashes when pressing up arrow after /!
• 700447f: 8255909: Remove unused delayed_value methods
• 397972e: 8255848: -Xlog:gc+heap+exit shows "used 0K"
• eb85b8d: 8255665: C2 should aggressively remove temporary hook nodes
• 18bc95b: 8250625: Compiler implementation of Pattern Matching for instanceof (Final)
• 60e4aca: 8255890: Zero: remove unused methods from BytecodeInterpreter
• 397bae2: 8255860: Clean up CDS logging related to lambda
• 97a81ce: 8253385: annotation processors remove varargs information from record components
• 166c728: 8255858: Add debug agent support for storing thread names
• a0ade22: 8255900: x86: Reduce impact when VerifyOops is disabled
• ... and 109 more: https://git.openjdk.java.net/jdk/compare/56eb5f54f40d2e1c3bdfbf440f2c5e1572639bc9...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@kstefanj, @tschatzl) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[tschatzl]

Please remove the size_t cast before SIZE_MAX. SIZE_MAX defines the maximum value that a size_t can hold, so it must be of that type.

Approving with that minor fix request. @kstefanj will sponsor it.

[kstefanj]

@linzang the testing looks good and we are ready to take this in. Before doing /integrate please merge the latest changes into your branch, I will issue the sponsor command once this is done.

[linzang]

Hi @kstefanj,
Thanks for help push it .
The latest changes has already at branch jmap-par on linzang/jdk. I will add "/integrate"
BRs,
Lin

[linzang]

/integrate

[openjdk]

@linzang
Your change (at version 4cb0866) is now ready to be sponsored by a Committer.

[kstefanj]

@linzang, sorry I was unclear. I meant merge the latest master, but since this change is not touching to many files and no conflicts are reported I guess its fine to let the bots to the automatic rebase.

[kstefanj]

/sponsor

[openjdk]

@kstefanj @linzang Since your change was applied there have been 119 commits pushed to the master branch:

• cdef186: 8232856: jshell crashes when pressing up arrow after /!
• 700447f: 8255909: Remove unused delayed_value methods
• 397972e: 8255848: -Xlog:gc+heap+exit shows "used 0K"
• eb85b8d: 8255665: C2 should aggressively remove temporary hook nodes
• 18bc95b: 8250625: Compiler implementation of Pattern Matching for instanceof (Final)
• 60e4aca: 8255890: Zero: remove unused methods from BytecodeInterpreter
• 397bae2: 8255860: Clean up CDS logging related to lambda
• 97a81ce: 8253385: annotation processors remove varargs information from record components
• 166c728: 8255858: Add debug agent support for storing thread names
• a0ade22: 8255900: x86: Reduce impact when VerifyOops is disabled
• ... and 109 more: https://git.openjdk.java.net/jdk/compare/56eb5f54f40d2e1c3bdfbf440f2c5e1572639bc9...master

Your commit was automatically rebased without conflicts.

Pushed as commit a6ce6a5.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.