Skip to content

Commit 84eef5f

Browse files
committed
add tools and human loop images
1 parent eee0530 commit 84eef5f

File tree

7 files changed

+7545
-42
lines changed

7 files changed

+7545
-42
lines changed

src/assets/images/agents/agent-workflow.svg

Lines changed: 36 additions & 35 deletions
Loading

src/assets/images/agents/human-in-the-loop.svg

Lines changed: 7367 additions & 0 deletions
Loading

src/content/docs/agents/concepts/calling-llms.mdx

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,30 @@
22
title: Calling LLMs
33
pcx_content_type: concept
44
sidebar:
5-
order: 5
5+
order: 6
66

77
---
88

99
import { Render } from "~/components";
1010

11-
calling LLMs
11+
### Understanding LLM Providers and Model Types
12+
13+
Different LLM providers offer models optimized for specific types of tasks. When building AI systems, choosing the right model is crucial for both performance and cost efficiency.
14+
15+
#### Reasoning Models
16+
Models like OpenAI's o1, Anthropic's Claude, and DeepSeek's R1 are particularly well-suited for complex reasoning tasks. These models excel at:
17+
- Breaking down problems into steps
18+
- Following complex instructions
19+
- Maintaining context across long conversations
20+
- Generating code and technical content
21+
22+
For example, when implementing a travel booking system, you might use a reasoning model to analyze travel requirements and generate appropriate booking strategies.
23+
24+
#### Instruction Models
25+
Models like GPT-4 and Claude Instant are optimized for following straightforward instructions efficiently. They work well for:
26+
- Content generation
27+
- Simple classification tasks
28+
- Basic question answering
29+
- Text transformation
30+
31+
These models are often more cost-effective for straightforward tasks that don't require complex reasoning.
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
---
2+
title: Human in the Loop
3+
pcx_content_type: concept
4+
sidebar:
5+
order: 5
6+
7+
---
8+
9+
import { Render, Note, Aside } from "~/components";
10+
11+
### What is Human-in-the-Loop?
12+
13+
Human-in-the-Loop (HITL) workflows integrate human judgment and oversight into automated processes. These workflows pause at critical points for human review, validation, or decision-making before proceeding. This approach combines the efficiency of automation with human expertise and oversight where it matters most.
14+
15+
![A human-in-the-loop diagram](~/assets/images/agents/human-in-the-loop.svg)
16+
17+
#### Understanding Human-in-the-Loop Workflows
18+
19+
In a Human-in-the-Loop workflow, processes aren't fully automated. Instead, they include designated checkpoints where human intervention is required. For example, in a travel booking system, a human may want to confirm the travel before an agent follows through with a transaction. The workflow manages this interaction, ensuring that:
20+
21+
1. The process pauses at appropriate review points
22+
2. Human reviewers receive necessary context
23+
3. The system maintains state during the review period
24+
4. Review decisions are properly incorporated
25+
5. The process continues once approval is receive
26+
27+
### Best Practices for Human-in-the-Loop Workflows
28+
29+
#### Long-Term State Persistence
30+
31+
Human review processes don't operate on predictable timelines. A reviewer might need days or weeks to make a decision, especially for complex cases requiring additional investigation or multiple approvals. Your system needs to maintain perfect state consistency throughout this period, including:
32+
- The original request and context
33+
- All intermediate decisions and actions
34+
- Any partial progress or temporary states
35+
- Review history and feedback
36+
37+
<Aside type="tip">
38+
[Durable Objects](https://developers.cloudflare.com/durable-objects/) provide an ideal solution for managing state in human-in-the-loop workflows, offering persistent compute instances that maintain state for hours, weeks, or months.
39+
</Aside>
40+
41+
#### Continuous Improvement Through Evals
42+
43+
Human reviewers play a crucial role in evaluating and improving LLM performance. Implement a systematic evaluation process where human feedback is collected not just on the final output, but on the LLM's decision-making process. This can include:
44+
45+
- Decision Quality Assessment: Have reviewers evaluate the LLM's reasoning process and decision points, not just the final output.
46+
- Edge Case Identification: Use human expertise to identify scenarios where the LLM's performance could be improved.
47+
- Feedback Collection: Gather structured feedback that can be used to fine-tune the LLM or adjust the workflow. [AI Gateway](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback/) can be a useful tool for setting up an LLM feedback loop.
48+
49+
#### Error Handling and Recovery
50+
51+
Robust error handling is essential for maintaining workflow integrity. Your system should gracefully handle various failure scenarios, including reviewer unavailability, system outages, or conflicting reviews. Implement clear escalation paths for handling exceptional cases that fall outside normal parameters.
52+
53+
The system should maintain stability during paused states, ensuring that no work is lost even during extended review periods. Consider implementing automatic checkpointing that allows workflows to be resumed from the last stable state after any interruption.
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
title: Tools
3+
pcx_content_type: concept
4+
sidebar:
5+
order: 4
6+
7+
---
8+
9+
### What are Tools?
10+
11+
Tools enable AI systems to interact with external services and perform actions. They provide a structured way for agents and workflows to invoke APIs, manipulate data, and integrate with external systems. Tools form the bridge between AI decision-making capabilities and real-world actions.
12+
13+
### Understanding Tools
14+
15+
In an AI system, tools are typically implemented as function calls that the AI can use to accomplish specific tasks. For example, a travel booking agent might have tools for:
16+
- Searching flight availability
17+
- Checking hotel rates
18+
- Processing payments
19+
- Sending confirmation emails
20+
21+
Each tool has a defined interface specifying its inputs, outputs, and expected behavior. This allows the AI system to understand when and how to use each tool appropriately.
22+
23+
### Common Tool Patterns
24+
25+
#### API Integration Tools
26+
27+
The most common type of tools are those that wrap external APIs. These tools handle the complexity of API authentication, request formatting, and response parsing, presenting a clean interface to the AI system.
28+
29+
#### Model Context Protocol (MCP)
30+
The (Model Context Protocol)[https://modelcontextprotocol.io/introduction] provides a standardized way to define and interact with tools. Think of it as an abstraction on top of APIs designed for LLMs to interact with external resources. MCP defines a consistent interface for:
31+
32+
- **Tool Discovery**: Systems can dynamically discover available tools
33+
- **Parameter Validation**: Tools specify their input requirements using JSON Schema
34+
- **Error Handling**: Standardized error reporting and recovery
35+
- **State Management**: Tools can maintain state across invocations
36+
37+
38+
#### Data Processing Tools
39+
Tools that handle data transformation and analysis are essential for many AI workflows. These might include:
40+
41+
- CSV parsing and analysis
42+
- Image processing
43+
- Text extraction
44+
- Data validation
45+

src/content/docs/agents/concepts/what-are-agents.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: What are Agents?
2+
title: Agents
33
pcx_content_type: concept
44
sidebar:
55
order: 2
@@ -8,7 +8,7 @@ sidebar:
88

99
import { Render } from "~/components";
1010

11-
### What is an agent?
11+
### What are agents?
1212
An agent is an AI system that can autonomously execute tasks by making decisions about tool usage and process flow. Unlike traditional automation that follows predefined paths, agents can dynamically adapt their approach based on context and intermediate results. Agents are also distinct from co-pilots (e.g. traditional chat applications) in that they can fully automate a task, as opposed to simply augmenting and extending human input.
1313

1414
- **Agents** → non-linear, non-deterministic (can change from run to run)

src/content/docs/agents/concepts/workflows.mdx

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,24 @@ sidebar:
88

99
import { Render } from "~/components";
1010

11-
TODO - what are agents?
11+
### What are workflows?
12+
A workflow is the orchestration layer that coordinates how an agent's components work together. It defines the structured paths through which tasks are processed, tools are called, and results are managed. While agents make dynamic decisions about what to do, workflows provide the underlying framework that governs how those decisions are executed.
13+
14+
#### Understanding Workflows in Agent Systems
15+
16+
Think of a workflow like the operating procedures of a company. The company (agent) can make various decisions, but how those decisions get implemented follows established processes (workflows). For example, when you book a flight through a travel agent, they might make different decisions about which flights to recommend, but the process of actually booking the flight follows a fixed sequence of steps.
17+
18+
Let's examine a basic agent workflow:
19+
20+
#### Core Components of a Workflow
21+
A workflow typically consists of several key elements:
22+
23+
1. **Input Processing**
24+
The workflow defines how inputs are received and validated before being processed by the agent. This includes standardizing formats, checking permissions, and ensuring all required information is present.
25+
2. **Tool Integration**
26+
Workflows manage how external tools and services are accessed. They handle authentication, rate limiting, error recovery, and ensuring tools are used in the correct sequence.
27+
3. **State Management**
28+
The workflow maintains the state of ongoing processes, tracking progress through multiple steps and ensuring consistency across operations.
29+
4. **Output Handling**
30+
Results from the agent's actions are processed according to defined rules, whether that means storing data, triggering notifications, or formatting responses.
1231

13-
- terminology
14-
- diagram: orchestrator / container / task / workflow

0 commit comments

Comments
 (0)