Skip to content

fix: Gemini system messages, tool result formatting, and argument key symbolization#6

Merged
ejwhite7 merged 1 commit into
mainfrom
fix/gemini-system-and-tool-messages
Apr 30, 2026
Merged

fix: Gemini system messages, tool result formatting, and argument key symbolization#6
ejwhite7 merged 1 commit into
mainfrom
fix/gemini-system-and-tool-messages

Conversation

@ejwhite7

Copy link
Copy Markdown
Owner

Problem

Three issues in ruby-pi cause the agent loop to fail with HTTP 400 errors when tools are involved, which is the root cause of the ongoing olli-social avatar chat failures (see ejwhite7/olli-social-app PR #78 discussion).

Issue 1: Gemini rejects system messages in contents

The Loop#build_llm_messages prepends a { role: :system } message. Gemini's API does not accept system as a role in the contents array — it requires system instructions to be sent via the dedicated systemInstruction field. This causes an immediate HTTP 400 on every Gemini call when a system prompt is set.

Issue 2: Gemini rejects role: "tool" in contents

When the agent loop executes tool calls, it adds results as role: :tool messages to state. On the next think() call, format_message maps these to { role: "tool", parts: [{ text: ... }] } — but Gemini only accepts user and model roles. Tool results must be sent as functionResponse parts. This causes HTTP 400 on the second LLM call in any tool-using conversation.

Issue 3: String-keyed arguments break tool implementations

LLM providers return tool arguments as JSON with string keys ({"query" => "..."}) but Ruby tool implementations expect symbol keys (args[:query]). This causes tools to silently receive nil for all parameters, leading to incorrect results or errors.

Fix

lib/ruby_pi/llm/gemini.rb

  • build_request_body: Extracts role: :system messages from the messages array and sets body[:systemInstruction] with their content. Only non-system messages are included in contents.
  • format_message: Detects role: "tool" messages and converts them to Gemini's functionResponse format ({ role: "user", parts: [{ functionResponse: { name: ..., response: { result: ... } } }] }). Falls back to plain text with role: "user" when tool name is missing.

lib/ruby_pi/tools/executor.rb

  • execute_single: Calls deep_symbolize_keys on arguments before passing them to the tool block.
  • deep_symbolize_keys (new private method): Recursively converts all string keys to symbols in hashes, including nested hashes and arrays.

Tests

Added specs for all three fixes:

  • Gemini: verifies systemInstruction is set and system messages excluded from contents
  • Gemini: verifies tool messages are formatted as functionResponse
  • Executor: verifies string keys are symbolized (flat, nested, array cases)
  • Executor: verifies already-symbolized keys pass through without error

All 334 specs pass.

…olize tool arguments

Three fixes:

1. Gemini build_request_body: Extract role:system messages from the
   messages array and send them via the `systemInstruction` field
   instead of as `contents` entries. Gemini API rejects system
   messages in the contents array with HTTP 400.

2. Gemini format_message: Convert role:tool messages to Gemini's
   `functionResponse` format instead of passing them as plain text
   with an invalid 'tool' role. When the agent loop adds tool
   results to state and calls the LLM again, the messages must use
   Gemini's native function response format.

3. Executor deep_symbolize_keys: LLM providers return tool arguments
   as JSON with string keys, but Ruby tool implementations typically
   access args with symbol keys (args[:field]). Recursively symbolize
   argument keys before passing to tool blocks.

All three issues cause the agent loop to fail with HTTP 400 when
tools are involved, triggering the Fallback provider to retry with
Anthropic (which may also fail due to mismatched message formats).

Includes specs for all three fixes.
@ejwhite7 ejwhite7 merged commit 5d5aacd into main Apr 30, 2026
2 checks passed
@ejwhite7 ejwhite7 deleted the fix/gemini-system-and-tool-messages branch April 30, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant