Track LiteLLM OCI provider fix for GPT-5/O-series token limit translation

[gotsysdba]

Observed broken baseline: LiteLLM 1.87.0.

Summary

We added a local workaround in AI Optimizer for OCI-hosted OpenAI GPT-5/O-series models because LiteLLM's OCI provider mishandles token limit parameters for newer OCI OpenAI model IDs. (#513 #514)

For affected models such as:

• oci/openai.gpt-5*
• oci/openai.o*

OCI rejects the legacy max_tokens field. These models require OCI's maxCompletionTokens request field, but LiteLLM can fail to translate max_completion_tokens correctly when the model is not marked as reasoning-capable in LiteLLM's local model catalog.

As a result, passing either max_tokens or max_completion_tokens through LiteLLM can produce an invalid OCI request.

Local workaround

AI Optimizer currently drops both fields before calling LiteLLM for affected OCI OpenAI models:

• max_tokens
• max_completion_tokens

Code:

• src/server/app/runtime/langgraph/adapters/litellm.py
• helper: _drop_oci_openai_unsupported_token_limits(...)
• model detection: _is_oci_openai_max_completion_model(...)

The openai.o* prefix is intentionally broad and conservative; it may also suppress token limits for a future OCI OpenAI openai.o* model that does not require completion-token semantics.

Covered by tests

File:

• src/server/tests/runtime/langgraph/adapters/test_litellm.py

Tests:

• test_oci_openai_gpt5_drops_max_tokens
• test_oci_openai_o_series_drops_max_tokens
• test_oci_openai_non_reasoning_keeps_max_tokens
• test_oci_openai_gpt5_drops_explicit_fallback_token_kwargs

Desired upstream fix

Open an upstream LiteLLM issue/PR so the OCI provider correctly handles GPT-5/O-series OpenAI models hosted on OCI by either:

1. recognizing these OCI OpenAI model IDs as requiring completion-token semantics, or
2. translating max_completion_tokens to OCI maxCompletionTokens independently of the local reasoning-capable model catalog.

The provider should not emit OCI maxTokens for these models.

Removal criteria

Remove the AI Optimizer workaround once a released LiteLLM version correctly handles token limit translation for OCI OpenAI GPT-5/O-series models.

Before removing the workaround, verify:

• oci/openai.gpt-5* calls do not send maxTokens
• oci/openai.o* calls do not send maxTokens
• max_completion_tokens is translated to OCI maxCompletionTokens
• non-target OCI OpenAI models, for example oci/openai.gpt-4.1, keep existing behavior

Notes

While the shim is active, configured max_tokens values are silently ignored for these models, so generations run to the OCI provider default.

This issue tracks deleting the local compatibility shim after the upstream LiteLLM OCI provider behavior is fixed and the dependency is upgraded.