Problems with Ollama API

[alexkh]

Greetings!

I'm writing Inforno - a Rust GUI for chatting LLM's. Was having issues running Ornith-1.0 on Ollama in Linux, so I decided to test this Fox app in Linux, since it claims to be a drop-in replacement. The CLI interface seems to works perfectly.

Now, since my app already uses Ollama API, I decided to connect it directly by creating Fox config file and setting port to 11434.
I installed ornith-1.0-35b-Q4_K_M which should be runnable with moe_cpu = true so my whole config.toml is this:

port = 11435
moe_cpu = true
type_kv = "turbo3"

Now, the pipeline seems to work, but I'm getting partial or empty replies.

My prompt is simple: "is 189 divisible by 9?"
I'm getting this reply:

Agent1:
<|im_end｜><｜fim begin｜>189  <response_group index="0">
<|mask_start|>**Yes**, let's check using both methods:



### Method
<response_group index="0">

*   **Direct Division**: 
$$\r

or this reply:

Agent1:
To determine if 189 is divisible by 9, you can use the divisibility rule for <response>
<response_group index="0">
<|mask_start|>**

*   **Divisibility Rule for 9**: A number is divisible by 9 if the sum of its digits is divisible by 9.

**Step-by-Step Check:**

1.
 Add the digits of <response>
<|mask_end|><｜fim begin｜>**Yes**, let's check using both methods:



### Method

Two obvious problems: incomplete responses and weird html tags which are not present when using cli api.

I also tried turning off reasoning, turning off streaming - did not help much.

Best regards,

-Alexandre