Skip to content

Update ONNX Gen AI jextract example#304

Open
ammbra wants to merge 1 commit into
openjdk:masterfrom
ammbra:upgrade-onnx-genai-demo
Open

Update ONNX Gen AI jextract example#304
ammbra wants to merge 1 commit into
openjdk:masterfrom
ammbra:upgrade-onnx-genai-demo

Conversation

@ammbra

@ammbra ammbra commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

1. Status Quo
The current ONNX Gen AI example requires end users to manually build the native library. A lot of files are needed only to build the native runtime, but not serving a need later on.

2. Problem

The native build instructions are subject to change when comes to prerequisites and the build from latest sources can also come with unresolved issues on the native side.

3. Fix

This pull request introduces a few changes in the prerequisites:

  • Pre-download compatible releases of onnxruntime and onnxruntime-genai (the onnxruntime-genai is a wrapper over the native onnxruntime). Release archives have the structure:
onnxruntime-genai-0.14.0-osx-arm64/
├── LICENSE
├── README.md
├── SECURITY.md
├── ThirdPartyNotices.txt
├── include
│   ├── ort_genai.h
│   └── ort_genai_c.h
└── lib
   └── libonnxruntime-genai.dylib
onnxruntime-osx-arm64-1.26.0/
├── GIT_COMMIT_ID
├── LICENSE
├── Privacy.md
├── README.md
├── ThirdPartyNotices.txt
├── VERSION_NUMBER
├── include
│   ├── core
│   │   └── providers
│   │       ├── custom_op_context.h
│   │       └── resource.h
│   ├── coreml_provider_factory.h
│   ├── cpu_provider_factory.h
│   ├── onnxruntime_c_api.h
│   ├── onnxruntime_cxx_api.h
│   ├── onnxruntime_cxx_inline.h
│   ├── onnxruntime_env_config_keys.h
│   ├── onnxruntime_ep_c_api.h
│   ├── onnxruntime_ep_device_ep_metadata_keys.h
│   ├── onnxruntime_float16.h
│   ├── onnxruntime_lite_custom_op.h
│   ├── onnxruntime_run_options_config_keys.h
│   ├── onnxruntime_session_options_config_keys.h
│   ├── provider_options.h
│   └── webgpu_provider_factory.h
├── lib
│   ├── cmake
│   │   └── onnxruntime
│   │       ├── onnxruntimeConfig.cmake
│   │       ├── onnxruntimeConfigVersion.cmake
│   │       ├── onnxruntimeTargets-release.cmake
│   │       └── onnxruntimeTargets.cmake
│   ├── libonnxruntime.1.26.0.dylib
│   ├── libonnxruntime.1.26.0.dylib.dSYM
│   │   └── Contents
│   │       ├── Info.plist
│   │       └── Resources
│   │           ├── DWARF
│   │           │   └── libonnxruntime.1.26.0.dylib
│   │           └── Relocations
│   │               └── aarch64
│   │                   └── libonnxruntime.1.26.0.dylib.yml
│   ├── libonnxruntime.1.dylib -> libonnxruntime.1.26.0.dylib
│   ├── libonnxruntime.dylib
│   └── pkgconfig
│       └── libonnxruntime.pc
└── testdata
   └── libcustom_op_library.dylib

The user should manually move the libonnxruntime<release>.dylib in the onnxruntime-genai lib folder.

  • Generate Java bindings for the downloaded onnxruntime-genai release using the
  • Removes unnecessary variables in prompt method code and add a way to exit the from LLM conversation in call method.

I run tests through Github actions enabled in my fork.


Progress

  • Change must not contain extraneous whitespace
  • Change must be properly reviewed (no review required)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jextract.git pull/304/head:pull/304
$ git checkout pull/304

Update a local copy of the PR:
$ git checkout pull/304
$ git pull https://git.openjdk.org/jextract.git pull/304/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 304

View PR using the GUI difftool:
$ git pr show -t 304

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jextract/pull/304.diff

Using Webrev

Link to Webrev Comment

Signed-off-by: Ana-Maria Mihalceanu <[email protected]>
@bridgekeeper

bridgekeeper Bot commented Jun 19, 2026

Copy link
Copy Markdown

👋 Welcome back amihalceanu! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk

openjdk Bot commented Jun 19, 2026

Copy link
Copy Markdown

@ammbra This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

Update ONNX Gen AI jextract example

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk Bot added ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jun 19, 2026
@mlbridge

mlbridge Bot commented Jun 19, 2026

Copy link
Copy Markdown

Webrevs

@ammbra ammbra mentioned this pull request Jun 19, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready Pull request is ready to be integrated rfr Pull request is ready for review

Development

Successfully merging this pull request may close these issues.

1 participant