The 230M model (/model lfm-nano, LiquidAI/LFM2.5-230M-ONNX) is selectable but can't load, so it silently degrades to the decline string. The default has been reverted to LFM2.5 350M (lfm-tiny), which works; this tracks getting 230M actually running.
Root cause
The 230M tokenizer's chat_template contains {% generation %} / {% endgeneration %} blocks — training-time assistant-token-masking markers. The Jinja parser bundled with transformers.js (via @nevescloud/pip) doesn't recognize the generation statement and throws at template-apply time:
SyntaxError: Unknown statement type: generation
renderer.generate() throws, the offline path catches it, and the visitor gets the decline line. The 350M and 1.2B variants use add_generation_prompt (a plain variable) instead and are unaffected.
Notes from investigation
- The
{% generation %} blocks are inert for inference — stripping just the tag keywords leaves the rendered prompt identical.
- A naive in-page
fetch shim that rewrites the tokenizer_config response in flight tripped a secondary parse error inside transformers.js's cache layer (Unexpected non-whitespace character after JSON), even though the rewritten JSON parsed fine standalone. So in-flight rewriting is fragile.
Options
- Self-host a sanitized
tokenizer_config.json in the repo (generation tags stripped) and point the 230M loader at it. More robust than the in-flight shim — a static file with normal headers sidesteps the cache-layer issue.
- Bump the bundled transformers.js /
@huggingface/jinja in @nevescloud/pip to a version that parses {% generation %}, if one exists.
- Drop the 230M option if the size win (~150MB vs ~250MB) isn't worth the maintenance.
Acceptance
/model lfm-nano loads and answers (no decline fallback) for a greeting and a project question.- Decide whether 230M becomes the default again or stays opt-in.
Tuning the offline prompt is easy via the window.neevsChat debug bridge in index.html (read/override the system prompt, run the model from the console).
The 230M model (
/model lfm-nano,LiquidAI/LFM2.5-230M-ONNX) is selectable but can't load, so it silently degrades to the decline string. The default has been reverted to LFM2.5 350M (lfm-tiny), which works; this tracks getting 230M actually running.Root cause
The 230M tokenizer's
chat_templatecontains{% generation %}/{% endgeneration %}blocks — training-time assistant-token-masking markers. The Jinja parser bundled with transformers.js (via@nevescloud/pip) doesn't recognize thegenerationstatement and throws at template-apply time:renderer.generate()throws, the offline path catches it, and the visitor gets the decline line. The 350M and 1.2B variants useadd_generation_prompt(a plain variable) instead and are unaffected.Notes from investigation
{% generation %}blocks are inert for inference — stripping just the tag keywords leaves the rendered prompt identical.fetchshim that rewrites the tokenizer_config response in flight tripped a secondary parse error inside transformers.js's cache layer (Unexpected non-whitespace character after JSON), even though the rewritten JSON parsed fine standalone. So in-flight rewriting is fragile.Options
tokenizer_config.jsonin the repo (generation tags stripped) and point the 230M loader at it. More robust than the in-flight shim — a static file with normal headers sidesteps the cache-layer issue.@huggingface/jinjain@nevescloud/pipto a version that parses{% generation %}, if one exists.Acceptance
/model lfm-nanoloads and answers (no decline fallback) for a greeting and a project question.Tuning the offline prompt is easy via the
window.neevsChatdebug bridge inindex.html(read/override the system prompt, run the model from the console).