Skip to content

Commit 99a1a53

Browse files
Document chat processing modes and typing indicators for AIChatAgent
This update documents new features from PR #685 (cloudflare/agents): - Chat processing modes (sequential vs batch) - Typing indicators for smart message batching - New AIChatAgent properties: chatProcessingMode, chatIdleTimeout, chatTypingTimeout - New useAgentChat returns: onInputChange, inputProps, sendTypingIndicator The documentation includes: - API reference updates for AIChatAgent class properties - Examples showing sequential and batch processing modes - React examples demonstrating typing indicator integration - Use cases and best practices for each mode These features enable better handling of conversational chat patterns where users send multiple rapid messages. Related PR: cloudflare/agents#685 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 5344722 commit 99a1a53

File tree

1 file changed

+176
-0
lines changed

1 file changed

+176
-0
lines changed

src/content/docs/agents/api-reference/agents-api.mdx

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -809,6 +809,22 @@ class AIChatAgent<Env = unknown, State = unknown> extends Agent<Env, State> {
809809

810810
// Persist messages within the Agent's local storage.
811811
async saveMessages(messages: Message[]): Promise<void>;
812+
813+
// Chat processing mode - controls how multiple incoming messages are handled
814+
// "sequential": Each message gets its own response (default)
815+
// "batch": Combine rapid messages into one response using debounce timing
816+
chatProcessingMode: "sequential" | "batch";
817+
818+
// Idle timeout in milliseconds (batch mode only)
819+
// How long to wait after a message is sent before processing
820+
// Default: 5000ms (5 seconds)
821+
chatIdleTimeout: number;
822+
823+
// Typing timeout in milliseconds (batch mode only)
824+
// How long to wait after user stops typing before processing
825+
// Requires client to send typing indicators via onInputChange
826+
// Default: 1500ms (1.5 seconds)
827+
chatTypingTimeout: number;
812828
}
813829
```
814830

@@ -865,6 +881,75 @@ class CustomerSupportAgent extends AIChatAgent<Env> {
865881

866882
</TypeScriptExample>
867883

884+
##### Chat Processing Modes
885+
886+
`AIChatAgent` supports two modes for handling multiple incoming messages:
887+
888+
**Sequential Mode (Default)**
889+
890+
In sequential mode, each message gets its own response. Messages are queued and processed one at a time, waiting for each response to fully complete before processing the next. No client changes required.
891+
892+
<TypeScriptExample>
893+
894+
```ts
895+
import { AIChatAgent } from "agents/ai-chat-agent";
896+
897+
class SequentialAgent extends AIChatAgent<Env> {
898+
// Sequential is the default - no configuration needed
899+
chatProcessingMode = "sequential";
900+
901+
async onChatMessage(onFinish) {
902+
// Each message will be processed independently
903+
// Users will get separate responses for each message
904+
return await this.generateResponse();
905+
}
906+
}
907+
```
908+
909+
</TypeScriptExample>
910+
911+
**Batch Mode**
912+
913+
In batch mode, multiple rapid messages are combined into one response. This is better for conversational UX where users send multiple short messages in quick succession.
914+
915+
There are two ways to use batch mode:
916+
917+
1. **Simple batching** - Set a fixed time window (no client changes required)
918+
2. **Smart batching** - Use typing indicators for more responsive batching (requires client changes)
919+
920+
<TypeScriptExample>
921+
922+
```ts
923+
import { AIChatAgent } from "agents/ai-chat-agent";
924+
925+
// Simple batching - 3 second window
926+
class SimpleBatchAgent extends AIChatAgent<Env> {
927+
chatProcessingMode = "batch";
928+
chatIdleTimeout = 3000; // 3 seconds to batch messages
929+
930+
async onChatMessage(onFinish) {
931+
// All messages within 3 seconds will be batched
932+
// this.messages contains all batched messages
933+
return await this.generateResponse();
934+
}
935+
}
936+
937+
// Smart batching - with typing indicators
938+
class SmartBatchAgent extends AIChatAgent<Env> {
939+
chatProcessingMode = "batch";
940+
chatIdleTimeout = 5000; // 5s max wait for user to start typing
941+
chatTypingTimeout = 1500; // 1.5s after typing stops
942+
943+
async onChatMessage(onFinish) {
944+
// Once user starts typing, switches to shorter 1.5s timeout
945+
// More responsive to user typing patterns
946+
return await this.generateResponse();
947+
}
948+
}
949+
```
950+
951+
</TypeScriptExample>
952+
868953
### Chat Agent React API
869954

870955
#### useAgentChat
@@ -923,6 +1008,14 @@ function useAgentChat(options: UseAgentChatOptions): {
9231008
addToolResult: ({ toolCallId, result }: { toolCallId: string; result: any }) => void;
9241009
// Current error if any
9251010
error: Error | undefined;
1011+
// Manually send a typing indicator to the agent (throttled)
1012+
sendTypingIndicator: () => void;
1013+
// Input change handler that automatically sends typing indicators
1014+
onInputChange: (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>) => void;
1015+
// Props to spread onto input element for automatic typing detection
1016+
inputProps: {
1017+
onChange: (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>) => void;
1018+
};
9261019
};
9271020
```
9281021

@@ -992,6 +1085,89 @@ function ChatInterface() {
9921085

9931086
</TypeScriptExample>
9941087

1088+
##### Typing Indicators for Smart Batching
1089+
1090+
When using batch mode with typing indicators on your agent, you can use the `onInputChange` handler or `inputProps` to automatically send typing signals. This allows the agent to intelligently batch messages while remaining responsive to user typing.
1091+
1092+
<TypeScriptExample>
1093+
1094+
```tsx
1095+
// Using onInputChange for manual control
1096+
import { useAgentChat } from "agents/ai-react";
1097+
import { useAgent } from "agents/react";
1098+
import { useState } from "react";
1099+
1100+
function SmartBatchingChat() {
1101+
const agent = useAgent({
1102+
agent: "smart-batch-agent",
1103+
name: "user-123"
1104+
});
1105+
1106+
const {
1107+
messages,
1108+
input,
1109+
setInput,
1110+
handleSubmit,
1111+
onInputChange // New: typing indicator handler
1112+
} = useAgentChat({ agent });
1113+
1114+
return (
1115+
<form onSubmit={handleSubmit}>
1116+
<input
1117+
value={input}
1118+
onChange={(e) => {
1119+
setInput(e.target.value); // Update input value
1120+
onInputChange(e); // Send typing indicator to agent
1121+
}}
1122+
placeholder="Type your message..."
1123+
/>
1124+
<button type="submit">Send</button>
1125+
</form>
1126+
);
1127+
}
1128+
1129+
// Using inputProps for automatic setup
1130+
function SimplerSmartBatchingChat() {
1131+
const agent = useAgent({
1132+
agent: "smart-batch-agent",
1133+
name: "user-123"
1134+
});
1135+
1136+
const {
1137+
messages,
1138+
input,
1139+
setInput,
1140+
handleSubmit,
1141+
inputProps // New: contains onChange with typing indicator
1142+
} = useAgentChat({ agent });
1143+
1144+
return (
1145+
<form onSubmit={handleSubmit}>
1146+
<input
1147+
value={input}
1148+
onChange={(e) => {
1149+
setInput(e.target.value);
1150+
inputProps.onChange(e); // Automatically sends typing indicators
1151+
}}
1152+
placeholder="Type your message..."
1153+
/>
1154+
<button type="submit">Send</button>
1155+
</form>
1156+
);
1157+
}
1158+
```
1159+
1160+
</TypeScriptExample>
1161+
1162+
**How it works:**
1163+
1164+
- **Without typing indicators**: Agent waits `chatIdleTimeout` (default 5s) after receiving a message
1165+
- **With typing indicators**: Agent switches to `chatTypingTimeout` (default 1.5s) once typing is detected
1166+
- **Throttling**: Typing indicators are throttled to 500ms to avoid excessive network traffic
1167+
- **Use cases**:
1168+
- Users sending multiple quick messages ("Hey", "Can you help", "With this issue?")
1169+
- Conversational interfaces where rapid-fire messaging is common
1170+
- Mobile chat interfaces with auto-correct/suggestions
9951171

9961172
### Next steps
9971173

0 commit comments

Comments
 (0)