Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0cba2f2
feat(chat): Add tool call support to mo.ui.chat
adamwdraper Dec 10, 2025
9dc4985
fix(chat): Move tool calls outside message bubble and fix OpenAI hist…
adamwdraper Dec 10, 2025
2b9f2a9
chore: Remove debug print statements from chat.py
adamwdraper Dec 10, 2025
343a88b
fix(chat): Correct OpenAI message format for tool call history
adamwdraper Dec 10, 2025
19656f1
fix(chat): Handle dataclass parts when converting to OpenAI format
adamwdraper Dec 10, 2025
01c607f
feat(chat): Add mo.ai.llm.pydantic_ai for simplified tool handling
adamwdraper Dec 10, 2025
e3bb5b7
Remove manual tool examples in favor of pydantic_ai
adamwdraper Dec 10, 2025
2e8fd8d
fix(example): Add missing chatbot display line in pydantic_ai_with_to…
adamwdraper Dec 10, 2025
73d1590
feat(chat): add thinking/reasoning support to pydantic_ai LLM class
adamwdraper Dec 10, 2025
5495225
feat(chat): stream thinking/reasoning in real-time before text
adamwdraper Dec 10, 2025
1c7f497
examples: combine thinking and tools into single pydantic_ai example
adamwdraper Dec 11, 2025
14fb9ab
feat(pydantic_ai): fix message history for multi-turn conversations w…
adamwdraper Dec 11, 2025
d026182
test(pydantic_ai): add tests for thinking, history storage, and unkno…
adamwdraper Dec 11, 2025
f213ba1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 11, 2025
fa8be9c
docs(chat): add pydantic_ai documentation with tools and thinking
adamwdraper Dec 11, 2025
8825cd2
feat(ai): add W&B Inference support to pydantic_ai
adamwdraper Dec 16, 2025
de963f5
refactor(pydantic_ai): use Provider pattern for all model creation
adamwdraper Dec 16, 2025
66efb65
test(pydantic_ai): add test for base_url without thinking enabled
adamwdraper Dec 16, 2025
9ced6a1
docs(chat): add W&B Inference and base_url documentation for pydantic_ai
adamwdraper Dec 16, 2025
882e883
Merge upstream/main into feature/chat-ui-tool-calls
adamwdraper Dec 16, 2025
1aee3ff
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 16, 2025
5cef086
fix: resolve ruff lint errors (PERF402, E402)
adamwdraper Dec 16, 2025
4f250fe
refactor(pydantic_ai): accept Agent directly instead of model string
adamwdraper Dec 16, 2025
5e3e8b3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 16, 2025
d212b38
refactor(pydantic_ai): support both model string and Agent directly
adamwdraper Dec 17, 2025
222e8f9
refactor(pydantic_ai): accept Agent only, simplify streaming
adamwdraper Dec 19, 2025
e2a90cd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 19, 2025
afcb4e5
fix(chat): rename reasoning accordion label to 'Thinking'
adamwdraper Dec 19, 2025
7e352f8
feat(chat): animate ellipsis while thinking, show char count when done
adamwdraper Dec 19, 2025
9bc13cb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 19, 2025
c482868
feat(chat): animate thinking ellipsis, keep accordion collapsed
adamwdraper Dec 19, 2025
e2c6550
fix: address pre-commit lint errors
adamwdraper Dec 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 129 additions & 7 deletions docs/api/inputs/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ mo.ui.chat(

## Streaming Responses

Chatbots can stream responses in real-time, creating a more interactive experience
Chatbots can stream responses in real-time, creating a more interactive experience
similar to ChatGPT where you see the response appear word-by-word as it's generated.

Responses from built-in models (OpenAI, Anthropic, Google, Groq, Bedrock) are streamed by default.
Expand All @@ -157,7 +157,7 @@ def streaming_model(messages, config):
"""Stream responses word by word."""
response = "This response will appear word by word!"
words = response.split()

for word in words:
yield word + " " # Yield delta chunks
time.sleep(0.1) # Simulate processing delay
Expand All @@ -176,7 +176,7 @@ async def async_streaming_model(messages, config):
"""Stream responses word by word asynchronously."""
response = "This response will appear word by word!"
words = response.split()

for word in words:
yield word + " " # Yield delta chunks
await asyncio.sleep(0.1) # Async processing delay
Expand All @@ -190,30 +190,32 @@ the progressively building response in real-time.

!!! tip "Delta vs Accumulated"
**Yield deltas, not accumulated text.** Each yield should be **new content only**:

✅ **Correct (delta mode):**
```python
yield "Hello"
yield " "
yield "world"
# Result: "Hello world"
```

❌ **Incorrect (accumulated mode, deprecated):**
```python
yield "Hello"
yield "Hello "
yield "Hello world"
# Inefficient: sends duplicate content
```

Delta mode is more efficient (reduces bandwidth by ~99% for long responses) and aligns with standard streaming APIs.

!!! tip "See streaming examples"
For complete working examples, check out:

- [`openai_example.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/openai_example.py) - OpenAI chatbot with streaming (default)
- [`streaming_custom.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/streaming_custom.py) - Custom streaming chatbot
- [`pydantic_ai_with_thinking_and_tools.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/pydantic_ai_with_thinking_and_tools.py) - Chatbot with thinking and tools
- [`wandb_inference_example.py`](https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/wandb_inference_example.py) - W&B Inference with reasoning models

## Built-in Models

Expand Down Expand Up @@ -289,6 +291,126 @@ mo.ui.chat(

::: marimo.ai.llm.groq

### Pydantic AI (with Tools and Thinking)

The `pydantic_ai` model provides advanced features like **tool calling** and
**thinking/reasoning** support using [pydantic-ai](https://ai.pydantic.dev/).

Create a [Pydantic AI Agent](https://ai.pydantic.dev/agents/) with your desired
configuration (model, tools, instructions, etc.) and pass it to `mo.ai.llm.pydantic_ai()`.

#### Basic Usage with Tools

```python
import marimo as mo
from pydantic_ai import Agent

def get_weather(location: str) -> dict:
"""Get weather for a location."""
return {"temperature": 72, "conditions": "sunny"}

def calculate(expression: str) -> float:
"""Evaluate a math expression."""
return eval(expression)

# Create a Pydantic AI Agent
agent = Agent(
"anthropic:claude-sonnet-4-5", # or "openai:gpt-4.1", etc.
tools=[get_weather, calculate],
instructions="You are a helpful assistant.",
)

chat = mo.ui.chat(mo.ai.llm.pydantic_ai(agent))
chat
```

Tool calls are displayed as collapsible accordions in the chat UI, showing
the tool name, inputs, and outputs.

#### With Thinking/Reasoning

Enable thinking to see the LLM's step-by-step reasoning process:

```python
from pydantic_ai import Agent
from pydantic_ai.models.anthropic import AnthropicModelSettings

agent = Agent(
"anthropic:claude-sonnet-4-5",
tools=[get_weather],
instructions="Think step-by-step.",
model_settings=AnthropicModelSettings(
max_tokens=8000,
anthropic_thinking={"type": "enabled", "budget_tokens": 4000},
),
)

chat = mo.ui.chat(mo.ai.llm.pydantic_ai(agent))
```

When enabled, a "Thinking" accordion appears before the response,
showing the LLM's reasoning process in real-time.

#### Full Agent Configuration

The Agent API gives you full control over Pydantic AI features:

```python
from pydantic_ai import Agent

agent = Agent(
"anthropic:claude-sonnet-4-5",
tools=[get_weather],
deps_type=MyDeps, # Dependency injection for tools
output_type=MyOutput, # Structured output schema
retries=3, # Retry configuration
# ... any other Agent options
)

chat = mo.ui.chat(mo.ai.llm.pydantic_ai(agent))
```

#### OpenAI-Compatible Providers (W&B Inference, DeepSeek, etc.)

For custom endpoints, configure a Model object with provider and profile:

```python
import marimo as mo
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.profiles.openai import OpenAIModelProfile

# Configure model with W&B Inference endpoint
model = OpenAIChatModel(
model_name="deepseek-ai/DeepSeek-R1-0528",
provider=OpenAIProvider(
api_key="your-wandb-key",
base_url="https://api.inference.wandb.ai/v1",
),
profile=OpenAIModelProfile(
openai_chat_thinking_field="reasoning_content",
),
)

# Create an Agent with the custom model
agent = Agent(model, instructions="Think step-by-step.")

chat = mo.ui.chat(mo.ai.llm.pydantic_ai(agent))
```

This works with any OpenAI-compatible endpoint:

- [W&B Inference](https://docs.wandb.ai/inference/) - Access open-source models with reasoning support
- [DeepSeek](https://platform.deepseek.com/) - DeepSeek models
- [Together AI](https://www.together.ai/) - Various open-source models
- Any other OpenAI-compatible API

See the [pydantic-ai documentation](https://ai.pydantic.dev/) for full
Agent and Model configuration options.

::: marimo.ai.llm.pydantic_ai

## Types

Chatbots can be implemented with a function that receives a list of
Expand Down
Loading
Loading