Skip to content

Commit ad65ae9

Browse files
adambaloghclaude
andauthored
Expose native web search and image generation in the SDK (#294)
The TEE gateway recently added an opt-in `web_search` flag and native Gemini image generation. This surfaces both through the LLM client. Web search: - `LLM.chat` and `LLM.completion` gain a `web_search: bool = False` arg that forwards `web_search` to the gateway when enabled. Supported by OpenAI, Anthropic, Google, and xAI models; billed per search on top of tokens. - LangChain adapter forwards `web_search` (constructor + per-call kwarg). - CLI `chat`/`completion` gain a `--web-search` flag. Image generation: - Add `GEMINI_2_5_FLASH_IMAGE` and `GEMINI_3_1_FLASH_IMAGE` ("nano banana") plus `GEMINI_3_5_FLASH` to the `TEE_LLM` enum. - Surface generated images on `TextGenerationOutput.images` (non-streaming) and `StreamChunk.images` (final stream chunk) as `data:` URIs. - CLI saves generated images to disk (`--image-output-dir`) instead of dumping base64. Adds examples (`llm_web_search.py`, `llm_image_generation.py`), README and docstring updates, and unit tests for payload forwarding and image surfacing. Co-authored-by: Claude <[email protected]>
1 parent ee6dbb4 commit ad65ae9

10 files changed

Lines changed: 393 additions & 12 deletions

File tree

README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ OpenGradient enables developers to build AI applications with verifiable executi
4646

4747
- **Verifiable LLM Inference**: Drop-in replacement for OpenAI and Anthropic APIs with cryptographic attestation
4848
- **Multi-Provider Support**: Access models from OpenAI, Anthropic, Google, and xAI through a unified interface
49+
- **Native Web Search**: Opt-in `web_search` flag enables each provider's built-in web search, billed per search
50+
- **Image Generation**: Native image-output models ("nano banana") return generated images directly on the response
4951
- **TEE Execution**: Trusted Execution Environment inference with cryptographic verification
5052
- **Model Hub Integration**: Registry for model discovery, versioning, and deployment
5153
- **Consensus-Based Verification**: End-to-end verified AI execution through the OpenGradient network
@@ -168,6 +170,37 @@ async for chunk in stream:
168170
print(chunk.choices[0].delta.content, end="")
169171
```
170172

173+
### Native Web Search
174+
175+
Set `web_search=True` to let the model search the web while answering. Each search is billed per search on top of token usage, at the provider's list price. Supported by OpenAI, Anthropic, Google, and xAI models; other providers ignore the flag.
176+
```python
177+
completion = await llm.chat(
178+
model=og.TEE_LLM.CLAUDE_SONNET_4_6,
179+
messages=[{"role": "user", "content": "What are today's top tech headlines?"}],
180+
max_tokens=500,
181+
web_search=True,
182+
)
183+
print(completion.chat_output["content"])
184+
```
185+
186+
### Image Generation
187+
188+
Native image-output models ("nano banana") return generated images on the response. The generated images are available in `result.images` as `data:` URIs, while any text caption is in `chat_output["content"]`. Images travel out-of-band and are not part of the signed output hash.
189+
```python
190+
import base64
191+
192+
result = await llm.chat(
193+
model=og.TEE_LLM.GEMINI_3_1_FLASH_IMAGE,
194+
messages=[{"role": "user", "content": "A friendly robot reading under a tree"}],
195+
max_tokens=1024,
196+
)
197+
198+
for i, image in enumerate(result.images or []):
199+
payload = image.split(",", 1)[1] # strip the "data:image/png;base64," prefix
200+
with open(f"image_{i}.png", "wb") as f:
201+
f.write(base64.b64decode(payload))
202+
```
203+
171204
### Verifiable LangChain Integration
172205

173206
Use OpenGradient as a drop-in LLM provider for LangChain agents with network-verified execution:
@@ -219,6 +252,9 @@ The SDK provides access to models from multiple providers via the `og.TEE_LLM` e
219252
- Gemini 2.5 Flash Lite
220253
- Gemini 3 Pro
221254
- Gemini 3 Flash
255+
- Gemini 3.5 Flash
256+
- Gemini 2.5 Flash Image (native image generation, "nano banana")
257+
- Gemini 3.1 Flash Image (native image generation, "nano banana 2")
222258

223259
#### xAI
224260
- Grok 4

examples/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,30 @@ python examples/llm_tool_calling.py
8484
- The model decides when to invoke tools based on the user's query
8585
- Uses x402 protocol for payment processing
8686

87+
#### `llm_web_search.py`
88+
Runs an LLM chat completion with native web search enabled.
89+
90+
```bash
91+
python examples/llm_web_search.py
92+
```
93+
94+
**What it does:**
95+
- Sets `web_search=True` to enable the provider's built-in web search
96+
- The model searches the web to answer the query, citing live sources
97+
- Each search is billed per search on top of token usage (supported by OpenAI, Anthropic, Google, and xAI models)
98+
99+
#### `llm_image_generation.py`
100+
Generates an image using a native image-output model ("nano banana").
101+
102+
```bash
103+
python examples/llm_image_generation.py
104+
```
105+
106+
**What it does:**
107+
- Calls an image-output model (`og.TEE_LLM.GEMINI_3_1_FLASH_IMAGE`) with a text prompt
108+
- Reads the generated images from `result.images` (data URIs) and writes them to disk
109+
- Image output is billed as completion tokens; images travel out-of-band and are not part of the signed output hash
110+
87111
## Alpha Testnet Examples
88112

89113
Examples for features only available on the **Alpha Testnet** are located in the [`alpha/`](./alpha/) folder. These include:

examples/llm_image_generation.py

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import asyncio
2+
import base64
3+
import logging
4+
import os
5+
import re
6+
7+
import opengradient as og
8+
9+
logging.basicConfig()
10+
logging.getLogger("opengradient").setLevel(logging.DEBUG)
11+
12+
_DATA_URI_RE = re.compile(r"^data:(?P<mime>[^;,]+)?(?:;base64)?,(?P<data>.*)$", re.DOTALL)
13+
14+
15+
def save_data_uri(data_uri: str, path: str) -> None:
16+
"""Decode a ``data:image/...;base64,...`` URI and write it to ``path``."""
17+
match = _DATA_URI_RE.match(data_uri)
18+
payload = match.group("data") if match else data_uri
19+
with open(path, "wb") as f:
20+
f.write(base64.b64decode(payload))
21+
22+
23+
async def main():
24+
llm = og.LLM(private_key=os.environ.get("OG_PRIVATE_KEY"))
25+
llm.ensure_opg_approval(min_allowance=0.1)
26+
27+
messages = [
28+
{"role": "user", "content": "Generate an image of a friendly robot reading a book under a tree."},
29+
]
30+
31+
# Image-output models ("nano banana") return generated images on the response.
32+
# The text caption (if any) is in chat_output["content"]; the generated images
33+
# are in result.images as data: URIs. Images travel out-of-band and are not part
34+
# of the signed output hash.
35+
result = await llm.chat(
36+
model=og.TEE_LLM.GEMINI_3_1_FLASH_IMAGE,
37+
messages=messages,
38+
max_tokens=1024,
39+
)
40+
41+
if result.chat_output and result.chat_output.get("content"):
42+
print(result.chat_output["content"])
43+
44+
images = result.images or []
45+
print(f"Generated {len(images)} image(s)")
46+
for index, image in enumerate(images):
47+
path = f"generated_image_{index + 1}.png"
48+
save_data_uri(image, path)
49+
print(f"Saved {path}")
50+
51+
52+
asyncio.run(main())

examples/llm_web_search.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
import asyncio
2+
import logging
3+
import os
4+
5+
import opengradient as og
6+
7+
logging.basicConfig()
8+
logging.getLogger("opengradient").setLevel(logging.DEBUG)
9+
10+
11+
async def main():
12+
llm = og.LLM(private_key=os.environ.get("OG_PRIVATE_KEY"))
13+
llm.ensure_opg_approval(min_allowance=0.1)
14+
15+
messages = [
16+
{"role": "user", "content": "What are the top technology headlines today? Cite your sources."},
17+
]
18+
19+
# Enable the provider's native web search with web_search=True. Each search is
20+
# billed per search on top of token usage. Web search is supported by OpenAI,
21+
# Anthropic, Google, and xAI models; other providers ignore the flag.
22+
result = await llm.chat(
23+
model=og.TEE_LLM.CLAUDE_SONNET_4_6,
24+
messages=messages,
25+
max_tokens=500,
26+
web_search=True,
27+
)
28+
print(result.chat_output["content"])
29+
30+
31+
asyncio.run(main())

src/opengradient/agents/og_langchain.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,7 @@ class OpenGradientChatModel(BaseChatModel):
135135
model_cid: Union[TEE_LLM, str]
136136
max_tokens: int = 300
137137
temperature: float = 0.0
138+
web_search: bool = False
138139
x402_settlement_mode: x402SettlementMode = x402SettlementMode.BATCH_HASHED
139140

140141
_llm: LLM = PrivateAttr()
@@ -149,6 +150,7 @@ def __init__(
149150
model: Optional[Union[TEE_LLM, str]] = None,
150151
max_tokens: int = 300,
151152
temperature: float = 0.0,
153+
web_search: bool = False,
152154
x402_settlement_mode: x402SettlementMode = x402SettlementMode.BATCH_HASHED,
153155
client: Optional[LLM] = None,
154156
rpc_url: Optional[str] = None,
@@ -165,6 +167,7 @@ def __init__(
165167
model_cid=resolved_model_cid,
166168
max_tokens=max_tokens,
167169
temperature=temperature,
170+
web_search=web_search,
168171
x402_settlement_mode=x402_settlement_mode,
169172
**kwargs,
170173
)
@@ -307,6 +310,7 @@ def _build_chat_kwargs(
307310
"temperature": kwargs.get("temperature", self.temperature),
308311
"tools": kwargs.get("tools", self._tools),
309312
"tool_choice": kwargs.get("tool_choice", self._tool_choice),
313+
"web_search": kwargs.get("web_search", self.web_search),
310314
"x402_settlement_mode": x402_settlement_mode,
311315
"stream": stream,
312316
}

0 commit comments

Comments
 (0)