8361842: Move input validation checks to Java for java.lang.StringCoding intrinsics #25998

vy · 2025-06-26T10:48:37Z

Validate input in java.lang.StringCoding intrinsic Java wrappers, improve their documentation, enhance the checks in the associated IR or assembly code, and adapt them to cause VM crash on invalid input.

Implementation notes

The goal of the associated umbrella issue JDK-8156534 is to, for java.lang.String* classes,

Move @IntrinsicCandidate-annotated public methods¹ (in Java code) to private ones, and wrap them with a public "front door" method
Since we moved the @IntrinsicCandidate annotation to a new method, intrinsic mappings – i.e., associated do_intrinsic() calls in vmIntrinsics.hpp – need to be updated too
Add necessary input validation (range, null, etc.) checks to the newly created public front door method
Place all input validation checks in the intrinsic code (add if missing!) behind a VerifyIntrinsicChecks VM flag

Following preliminary work needs to be carried out as well:

Add a new VerifyIntrinsicChecks VM flag
Update generate_string_range_check to produce a HaltNode. That is, crash the VM if VerifyIntrinsicChecks is set and a Java wrapper fails to spot an invalid input.

¹ @IntrinsicCandidate-annotated constructors are not subject to this change, since they are a special case.

Functional and performance tests

tier1 (which includes test/hotspot/jtreg/compiler/intrinsics/string) passes on several platforms. Further tiers will be executed after integrating reviewer feedback.
Performance impact is still actively monitored using test/micro/org/openjdk/bench/java/lang/String{En,De}code.java, among other tests. If you have suggestions on benchmarks, please share in the comments.

Verification of the VM crash

I've tested the VM crash scenario as follows:

Created the following test program:

public class StrIntri {
    public static void main(String[] args) {
        Exception lastException = null;
        for (int i = 0; i < 1_000_000; i++) {
            try {
                jdk.internal.access.SharedSecrets.getJavaLangAccess().countPositives(new byte[]{1,2,3}, 2, 5);
            } catch (Exception exception) {
                lastException = exception;
            }
        }
        if (lastException != null) {
            lastException.printStackTrace();
        } else {
            System.out.println("completed");
        }
    }
}

Compiled the JDK and run the test:

$ bash jib.sh configure -p linux-x64-slowdebug
$ CONF=linux-x64-slowdebug make jdk
$ ./build/linux-x64-slowdebug/jdk/bin/java -XX:+VerifyIntrinsicChecks --add-exports java.base/jdk.internal.access=ALL-UNNAMED StrIntri.java
java.lang.ArrayIndexOutOfBoundsException: Range [2, 2 + 5) out of bounds for length 3

Received AIOOBE as expected.

Removed all checks in StringCodec.java, and re-compiled the JDK
Set the countPositives(...) arguments in the program to (null, 1, 1), run it, and observed the VM crash with unexpected null in intrinsic.
Set the countPositives(...) arguments in the program to (new byte[]{1,2,3}, 2, 5), run it, and observed the VM crash with unexpected guard failure in intrinsic.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8361842: Move input validation checks to Java for java.lang.StringCoding intrinsics (Sub-task - P4)

Reviewers

Damon Fenacci (@dafedafe - Committer) Review applies to db1ed388
Roger Riggs (@RogerRiggs - Reviewer)
Tobias Hartmann (@TobiHartmann - Reviewer) Review applies to db1ed388
Claes Redestad (@cl4es - Reviewer) Review applies to d5aabf0c
John R Rose (@rose00 - Reviewer) Review applies to d5aabf0c
Chen Liang (@liach - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/25998/head:pull/25998
$ git checkout pull/25998

Update a local copy of the PR:
$ git checkout pull/25998
$ git pull https://git.openjdk.org/jdk.git pull/25998/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 25998

View PR using the GUI difftool:
$ git pr show -t 25998

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/25998.diff

Using Webrev

Link to Webrev Comment

bridgekeeper · 2025-06-26T10:49:31Z

👋 Welcome back vyazici! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-06-26T10:49:35Z

@vy This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8361842: Move input validation checks to Java for java.lang.StringCoding intrinsics

Reviewed-by: rriggs, liach, dfenacci, thartmann, redestad, jrose

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 263 new commits pushed to the master branch:

285adff: 8362448: Make use of the Double.toString(double) algorithm in java.text.DecimalFormat
c9ecedd: 8365594: Strengthen Universe klasses asserts to catch bootstrapping errors earlier
2a16cc8: 8365550: JFR: The active-settings view should not use LAST_BATCH
... and 260 more: https://git.openjdk.org/jdk/compare/52155dbbb0107c5077a6be7edfd91d4311411fc3...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-06-26T10:50:11Z

@vy The following labels will be automatically applied to this pull request:

core-libs
graal
hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

src/java.base/share/classes/java/lang/StringCoding.java

…r our cases This reverts commit 196fc5d.

vy · 2025-07-10T12:18:25Z

src/java.base/share/classes/java/lang/StringCoding.java

+     * </p>
+     *
+     * @param sa the source byte array containing characters encoded in UTF-16
+     * @param sp the index of the <em>byte (not character!)</em> from the source array to start reading from


Note the byte (not character!) emphasis here and below.

I think this is incorrect.
This is the index of a character (two bytes).
As it is used in encodeISOArray0(), it is incremented by 1 and passed to StringUTF16.getChar(), where it is multiplied by 2 to obtain the real byte[] index.

vy · 2025-07-10T12:20:45Z

src/java.base/share/classes/java/lang/StringCoding.java

+     *         {@linkplain Preconditions#checkFromIndexSize(int, int, int, BiFunction) out of bounds}
+     */
+    static int encodeISOArray(byte[] sa, int sp, byte[] da, int dp, int len) {
+        checkFromIndexSize(sp, len << 1, requireNonNull(sa, "sa").length, AIOOBE_FORMATTER);


sa contains 2-byte chars, and sp points to an index of this inflated array. Though, len denotes the codepoint count, hence the len << 1 while checking sp and len bounds.

The reference of sa.length is likely wrong also, as it is the source length in bytes but for the index check should be checking the source length in chars.
It might be worth trying to find or create a test for the accidental incorrect interpretation of length in bytes vs chars..

src/hotspot/share/opto/library_call.cpp

mlbridge · 2025-07-10T12:58:53Z

Webrevs

rose00 · 2025-07-10T20:54:24Z

I disagree with a small part of the statement of goals:

Always validate all input at the intrinsic (but preferably behind a VM flag)

As formulated above, this is a violation of DRY and if embraced the wrong way will lead to code that is harder to review and prove bug-free. Performing 100% accurate range/null/validation checks is deeply impractical for an assembly-based or IR-based intrinsic. It’s too hard to verify by code review, and coverage testing is suspect.

We must frankly put all the weight of verification on Java code, including Java bytecode intrinsic behaviors. Java code is high-level and can be read mostly as a declarative spec, if clearly written (as straight-line code, then the intrinsic call). Also, such simple Java code shapes (and their underlying bytecodes) are tested many orders of magnitude more than any given intrinsic.

I see two bits of evidence that you agree with me on this: 1. The intrinsic-local validation (IR or assembly) is allowed to Halt instead of throw, and 2. the intrinsic-local validation is optional, turned on only by a stress test mode. This tells me that the extra optional testing is also not required to be 100%.

Thus, I think the above goal would be better stated this way:

Validate input in the IR or assembly code of the intrinsic in an ad hoc manner to catch bugs in the Java validation.

Note: IR or assembly based validation code should not obscure the code or add large maintenance costs, and under a VM diagnostic flag (or debug flag), and causing a VM halt instead of a Java throw.

I think I'm agreeing with you on the material points. It is important to summarize our intentions accurately at the top, for those readers that are reading only the top as a summary.

vy · 2025-07-11T08:22:49Z

@rose00, thanks so much for the feedback. I agree with your remarks and get your points on "Always validate all input at the intrinsic" is a violation of DRY and an impractical goal.

I incorporated your suggestions as follows:

Renamed the ticket to Move input validation checks to Java for String-related intrinsics (to better reflect the goal)
Replaced Always validate all input at the intrinsic... with your suggestion

dafedafe

Thanks a lot for looking into this Volkan!
I left a couple of minor comments.
I also noticed that you haven't yet added the benchmark results to the description: do you want to run them again after the reviews?

src/hotspot/share/opto/c2_globals.hpp

src/hotspot/cpu/x86/macroAssembler_x86.cpp

src/hotspot/share/classfile/vmIntrinsics.hpp

vy · 2025-07-15T19:31:45Z

I left a couple of minor comments. I also noticed that you haven't yet added the benchmark results to the description: do you want to run them again after the reviews?

@dafedafe, thanks so much for the review! I've implemented the changes you requested, and shared some benchmark figures in the associated ticket. I am still actively working on evaluating the performance impact.

src/java.base/share/classes/java/lang/StringCoding.java

src/java.base/share/classes/sun/nio/cs/ISO_8859_1.java

src/hotspot/share/opto/library_call.cpp

Those who are touching to these methods should well be aware of the details elaborated in the `@apiNote`, no need to put it on a display.

cl4es

I've done some testing on linux-amd64 and verified that on microbenchmarks that exercise for example StringCoding.hasNegatives (a front door of one of the intrinsics this PR changes) the generated assembly is identical under ideal conditions. Spurious regressions seen in some setups could be inlining related: moving from a simple range check emitted by the intrinsic to a call to Preconditions.checkFromIndexSize may push us over some inlining threshold in some cases. I'll try to get my hands on a linux-aarch64 machine to do some diagnostic runs on.

An idea for future investigation could be to make Preconditions.checkFromIndexSize an intrinsic similar to Preconditions.checkIndex - to help the compiler do the right thing with more ease and perhaps slightly faster.

rose00 · 2025-08-14T20:30:08Z

… to help the compiler do the right thing with more ease and perhaps slightly faster

If the Java code is good enough, then the precondition method can simply be marked @ForceInline.

rose00 · 2025-08-14T20:35:35Z

Some more parts to the precondition intrinsic story, FTR:

Apart from inlining, one of the goals of the intrinsic preconditions is to allow them to more reliably interact with elimination of range checks. The JVM knows its own intrinsic checks in the aaload bytecode (and its brothers), and it also knows its own intrinsic precondition method. Both kinds of checks are fodder for RCE (range check elimination) and specifically the iteration range splitting performed when a loop is factored into pre/main/post loops. The main part is statically proven to traverse an index range which is incapable of failing any of the checks (within the loop body). This RCE in turn unlocks power moves like vectorization.

If the precondition check in question works OK for these use cases, it can be marked force-inline, but if there is also evidence that it would unlock more loop optimizations, then it should be made an intrinsic (or else built on top of another intrinsics, with force-inline).

rose00 · 2025-08-14T21:22:52Z

Another comment on the precondition in question: It does not appear to be inside a loop, but rather a precursor to a bulk operation (which searches the sign bits of a byte array slice). It's hard to imagine the JIT doing a better job with that as an intrinsic, since it probably won't be RCE-ed within an enclosing hot loop. So, yes, force-inline it.

Volkan found a pre-existing RFE about that precondition check, and I added a lengthy comment to it, FTR:

https://bugs.openjdk.org/browse/JDK-8361837#comment-14809088

src/java.base/share/classes/sun/nio/cs/ISO_8859_1.java

test/hotspot/jtreg/compiler/intrinsics/string/TestCountPositives.java

test/hotspot/jtreg/compiler/intrinsics/TestVerifyIntrinsicChecks.java

src/hotspot/share/opto/library_call.cpp

rose00

This is lovely work. I've left a few suggestions which you may wish to take action on.

…range conditions

vy · 2025-08-18T08:21:23Z

JDK-8361842, addressed by this PR, is the first step in a series of similar improvements under the JDK-8156534 umbrella issue. I wanted to get this one in perfect shape to serve as a guide for the subsequent PRs, hence the meticulous effort. Thanks so much to everyone helped with reviewing this work. 🙇 😍

I've verified that tier1,tier2,tier3,tier4,tier5,hs-comp-stress,hs-precheckin-comp passes for 2ba4ba6 on several platforms.

/integrate

openjdk · 2025-08-18T08:23:56Z

@vy This pull request has not yet been marked as ready for integration.

liach

Looks good in principle; didn't check in the details for compiler code, which I don't necessarily understand.

RogerRiggs

looks good.

vy · 2025-08-19T05:05:19Z

/integrate

openjdk · 2025-08-19T05:06:51Z

Going to push as commit 655dc51.
Since your change was applied there have been 268 commits pushed to the master branch:

f2f7a49: 8365071: ARM32: JFR intrinsic jvm_commit triggers C2 regalloc assert
e04a310: 8364806: Test sun/security/krb5/config/IncludeRandom.java times out on Windows
ec7361e: 8365660: test/jdk/sun/security/pkcs11/KeyAgreement/ tests skipped without SkipExceprion
... and 265 more: https://git.openjdk.org/jdk/compare/52155dbbb0107c5077a6be7edfd91d4311411fc3...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-08-19T05:07:05Z

@vy Pushed as commit 655dc51.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Move StringCoding::countPositives checks from C++ to Java

ac5df9f

openjdk bot added graal [email protected] hotspot [email protected] core-libs [email protected] labels Jun 26, 2025

Apply review feedback

1498824

liach reviewed Jun 26, 2025

View reviewed changes

src/java.base/share/classes/java/lang/StringCoding.java Outdated Show resolved Hide resolved

vy added 5 commits July 4, 2025 16:34

Add StringCodingCountPositives benchmark

196fc5d

Improve intrinsics in StringCoding

9932dd3

Remove StringCodingCountPositives, String{En,De}code already cove…

14275e5

…r our cases This reverts commit 196fc5d.

Fix EUC_JP.java.template broken due to encodeASCII rename

b9a6adf

Merge remote-tracking branch 'upstream/master' into strIntrinCheck

6af9864

vy changed the title ~~8156534: Check if range checks can be moved into Java wrapper for intrinsics~~ 8361842: Validate input in both Java and C++ for java.lang.StringCoding intrinsics Jul 10, 2025

vy commented Jul 10, 2025

View reviewed changes

src/hotspot/share/opto/library_call.cpp Outdated Show resolved Hide resolved

vy marked this pull request as ready for review July 10, 2025 12:55

openjdk bot added the rfr Pull request is ready for review label Jul 10, 2025

vy changed the title ~~8361842: Validate input in both Java and C++ for java.lang.StringCoding intrinsics~~ 8361842: Move input validation checks to Java for String-related intrinsics Jul 11, 2025

dafedafe reviewed Jul 14, 2025

View reviewed changes

src/hotspot/share/opto/c2_globals.hpp Outdated Show resolved Hide resolved

src/hotspot/cpu/x86/macroAssembler_x86.cpp Outdated Show resolved Hide resolved

src/hotspot/share/classfile/vmIntrinsics.hpp Outdated Show resolved Hide resolved

vy added 3 commits July 15, 2025 21:10

Improve wording of the VerifyIntrinsicChecks flag

c331fbf

Remove Markdown-styling in comments

b60ff45

Minimize the number of touched lines in vmIntrinsics.hpp

7c042b3

RogerRiggs reviewed Jul 15, 2025

View reviewed changes

cl4es reviewed Aug 11, 2025

View reviewed changes

src/hotspot/share/opto/library_call.cpp Outdated Show resolved Hide resolved

vy added 3 commits August 11, 2025 15:28

Move ->asRegion() after if (halt)

4d2a7a3

Rename generate_string_range_check::halt to halt_on_oob

9b721bb

Remove @apiNote in encodeISOArray() Javadoc

d5aabf0

Those who are touching to these methods should well be aware of the details elaborated in the `@apiNote`, no need to put it on a display.

cl4es approved these changes Aug 13, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 13, 2025

rose00 reviewed Aug 14, 2025

View reviewed changes

src/java.base/share/classes/sun/nio/cs/ISO_8859_1.java Show resolved Hide resolved

rose00 reviewed Aug 14, 2025

View reviewed changes

test/hotspot/jtreg/compiler/intrinsics/string/TestCountPositives.java Show resolved Hide resolved

rose00 reviewed Aug 14, 2025

View reviewed changes

test/hotspot/jtreg/compiler/intrinsics/TestVerifyIntrinsicChecks.java Show resolved Hide resolved

rose00 reviewed Aug 14, 2025

View reviewed changes

src/hotspot/share/opto/library_call.cpp Show resolved Hide resolved

rose00 approved these changes Aug 14, 2025

View reviewed changes

vy added 2 commits August 15, 2025 09:47

Document why Preconditions is not used

6235034

Document tests using -XX:+VerifyIntrinsicChecks don't check out-of-…

2ba4ba6

…range conditions

openjdk bot removed the ready Pull request is ready to be integrated label Aug 15, 2025

liach approved these changes Aug 18, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 18, 2025

RogerRiggs approved these changes Aug 18, 2025

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Aug 19, 2025

openjdk bot closed this Aug 19, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Aug 19, 2025

vy deleted the strIntrinCheck branch August 19, 2025 05:07

8361842: Move input validation checks to Java for java.lang.StringCoding intrinsics #25998

8361842: Move input validation checks to Java for java.lang.StringCoding intrinsics #25998

Conversation

vy commented Jun 26, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation notes

Functional and performance tests

Verification of the VM crash

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Jun 26, 2025

Uh oh!

openjdk bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Jun 26, 2025

Uh oh!

Uh oh!

vy Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

rgiulietti Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

vy Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

RogerRiggs Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mlbridge bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

rose00 commented Jul 10, 2025

Uh oh!

vy commented Jul 11, 2025

Uh oh!

dafedafe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vy commented Jul 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cl4es left a comment

Choose a reason for hiding this comment

Uh oh!

rose00 commented Aug 14, 2025

Uh oh!

rose00 commented Aug 14, 2025

Uh oh!

rose00 commented Aug 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rose00 left a comment

Choose a reason for hiding this comment

Uh oh!

vy commented Aug 18, 2025

Uh oh!

openjdk bot commented Aug 18, 2025

Uh oh!

liach left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

vy commented Jun 26, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Jun 26, 2025 •

edited

Loading

mlbridge bot commented Jul 10, 2025 •

edited

Loading

liach left a comment •

edited

Loading