8386116: vm/jvmti/StopThread/stop001/stop00103/stop00103a.html trips assert(!java_lang_Thread::is_in_vthread_transition(vthread)) failed

[pchilano]

An asynchronous exception handshake can be installed by JVMTI StopThread on a virtual thread blocked in MountUnmountDisabler::start_transition while trying to start an unmount transition in yieldContinuation. In that case, the handshake will be processed and the exception thrown outside the try block in yieldContinuation. Since the transition bits remain set, the virtual thread will hit the reported assert at the next unmount attempt.

The fix is to move startTransition inside the try block in yieldContinuation so that endTransition is executed in case of an exception.

Note that calls to MountUnmountDisabler::start_transition for the mount case are not an issue because StopThread does not work on an unmounted virtual thread.

I added a test that reproduces the crash without this fix. I also tested the patch in mach5 tiers1-6.

Thanks,
Patricio

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

• JDK-8386116: vm/jvmti/StopThread/stop001/stop00103/stop00103a.html trips assert(!java_lang_Thread::is_in_vthread_transition(vthread)) failed (Bug - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/31519/head:pull/31519
$ git checkout pull/31519

Update a local copy of the PR:
$ git checkout pull/31519
$ git pull https://git.openjdk.org/jdk.git pull/31519/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 31519

View PR using the GUI difftool:
$ git pr show -t 31519

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/31519.diff

[bridgekeeper]

👋 Welcome back pchilanomate! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@pchilano The following labels will be automatically applied to this pull request:

• core-libs
• hotspot
• serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[openjdk]

The total number of required reviews for this PR has been set to 2 based on the presence of this label: hotspot. This can be overridden with the /reviewers command.

[pchilano]

/label remove core-libs

[openjdk]

@pchilano
The core-libs label was successfully removed.

[JnRouvignac]

been -> being ?