gemma3

Gemma 3 Demo

Interactive chat with Gemma 3 270M using Torq VMFB models.

Setup

See repo README.md for installing the virtual environment and base dependencies.

Enter the demo directory. Install its dependencies. Jump back to the repo root.

cd gemma3
pip install -r requirements.txt
cd ..

From the repo root, run:

python setup_demos.py gemma3

This downloads the default instruct model files to: models/Synaptics/gemma-3-270m-it-torq/

Running

Run the demo from the gemma3 directory:

cd gemma3
python src/infer.py -m ../models/Synaptics/gemma-3-270m-it-torq/model.vmfb.trim --instruct-model

Note

The demo defaults to the DMA/dmabuf allocator with device I/O enabled. Use --tda cpu to run with the CPU allocator, or --no-device-io to pass user inputs as NumPy arrays.

Tip

If the downloaded repo only contains model.vmfb, use that path instead.

Type exit or quit to stop the chat session. While an answer is being generated, press Ctrl + C or Ctrl + D to interrupt it and return to the prompt.

Run python src/infer.py -h to see all available inference options.

Validation

Gemma 3 includes a validation script for text translation datasets. For example:

cd gemma3
python src/validate.py -m ../models/Synaptics/gemma-3-270m-it-torq/model.vmfb.trim --instruct-model --max-samples 10

Run python src/validate.py -h to see all available validation options.