Skip to content

Commit 4c80a28

Browse files
committed
Complete Context Engineering course with all 15 notebooks and reference agent
- Added Section 2: System Context (3 notebooks) * System instructions and prompt engineering * Defining tools with clear schemas * Tool selection strategies (advanced) - Added Section 3: Memory (4 notebooks) * Working memory with extraction strategies * Long-term memory management * Memory integration patterns * Memory tools for LLM control (advanced) - Added Section 4: Optimizations (5 notebooks) * Context window management and token budgets * Retrieval strategies (RAG, summaries, hybrid) * Grounding with memory * Tool optimization and filtering (advanced) * Crafting data for LLMs (advanced) - Updated reference agent with reusable modules * tools.py - Tool definitions from Section 2 * optimization_helpers.py - Production patterns from Section 4 * memory_client.py - Simplified Agent Memory Server interface * examples/advanced_agent_example.py - Complete production example - Added comprehensive documentation * COURSE_SUMMARY.md - Complete course overview * MEMORY_ARCHITECTURE.md - Memory system design * Updated README with all sections - Fixed tests to pass with new structure * Updated imports to use MemoryClient * Added tests for new modules * All 10 tests passing
1 parent 8f53551 commit 4c80a28

37 files changed

+8727
-1898
lines changed
Lines changed: 286 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,286 @@
1+
# Context Engineering Course - Complete Summary
2+
3+
## Overview
4+
5+
This course teaches production-ready context engineering for AI agents using Redis and the Agent Memory Server. It covers everything from fundamentals to advanced optimization techniques.
6+
7+
## Course Structure
8+
9+
### Section 1: Introduction (3 notebooks)
10+
1. **What is Context Engineering?** - Core concepts and importance
11+
2. **Setting Up Your Environment** - Installation and configuration
12+
3. **Project Overview** - Understanding the reference agent
13+
14+
### Section 2: System Context (3 notebooks)
15+
1. **System Instructions** - Crafting effective system prompts
16+
2. **Defining Tools** - Giving agents capabilities
17+
3. **Tool Selection Strategies** (Advanced) - Improving tool choice
18+
19+
**Key Patterns:**
20+
- Progressive system prompt building
21+
- Tool schema design with examples
22+
- Clear naming conventions
23+
- Detailed descriptions with when/when-not guidance
24+
25+
### Section 3: Memory (4 notebooks)
26+
1. **Working Memory with Extraction Strategies** - Session-scoped context
27+
2. **Long-term Memory** - Cross-session knowledge
28+
3. **Memory Integration** - Combining working and long-term memory
29+
4. **Memory Tools** (Advanced) - LLM control over memory
30+
31+
**Key Patterns:**
32+
- Automatic memory extraction
33+
- Semantic search for retrieval
34+
- Memory type selection (semantic vs episodic)
35+
- Tool-based memory management
36+
37+
### Section 4: Optimizations (5 notebooks)
38+
1. **Context Window Management** - Handling token limits
39+
2. **Retrieval Strategies** - RAG, summaries, and hybrid approaches
40+
3. **Grounding with Memory** - Using memory to resolve references
41+
4. **Tool Optimization** (Advanced) - Selective tool exposure
42+
5. **Crafting Data for LLMs** (Advanced) - Creating structured views
43+
44+
**Key Patterns:**
45+
- Token budget estimation
46+
- Hybrid retrieval (summary + RAG)
47+
- Tool filtering by intent
48+
- Retrieve → Summarize → Stitch → Save pattern
49+
- Structured view creation
50+
51+
## Reference Agent Components
52+
53+
### Core Modules
54+
55+
**`course_manager.py`**
56+
- Course catalog management
57+
- Vector search for courses
58+
- Course data models
59+
60+
**`memory_client.py`**
61+
- Working memory operations
62+
- Long-term memory operations
63+
- Integration with Agent Memory Server
64+
65+
**`agent.py`**
66+
- Main agent implementation
67+
- LangGraph workflow
68+
- State management
69+
70+
### New Modules (From Course Content)
71+
72+
**`tools.py`** (Section 2)
73+
- `create_course_tools()` - Search, get details, check prerequisites
74+
- `create_memory_tools()` - Store and search memories
75+
- `select_tools_by_keywords()` - Simple tool filtering
76+
77+
**`optimization_helpers.py`** (Section 4)
78+
- `count_tokens()` - Token counting for any model
79+
- `estimate_token_budget()` - Budget breakdown
80+
- `hybrid_retrieval()` - Combine summary + search
81+
- `create_summary_view()` - Structured summaries
82+
- `create_user_profile_view()` - User profile generation
83+
- `filter_tools_by_intent()` - Keyword-based filtering
84+
- `classify_intent_with_llm()` - LLM-based classification
85+
- `extract_references()` - Find grounding needs
86+
- `format_context_for_llm()` - Combine context sources
87+
88+
### Examples
89+
90+
**`examples/advanced_agent_example.py`**
91+
- Complete agent using all patterns
92+
- Tool filtering enabled
93+
- Token budget tracking
94+
- Memory integration
95+
- Production-ready structure
96+
97+
## Key Concepts by Section
98+
99+
### Section 2: System Context
100+
- **System vs Retrieved Context**: Static instructions vs dynamic data
101+
- **Tool Schemas**: Name, description, parameters
102+
- **Tool Selection**: How LLMs choose tools
103+
- **Best Practices**: Clear names, detailed descriptions, examples
104+
105+
### Section 3: Memory
106+
- **Working Memory**: Session-scoped, conversation history
107+
- **Long-term Memory**: User-scoped, persistent facts
108+
- **Memory Types**: Semantic (facts), Episodic (events), Message (conversations)
109+
- **Automatic Extraction**: Agent Memory Server extracts important facts
110+
- **Memory Flow**: Load → Search → Process → Save → Extract
111+
112+
### Section 4: Optimizations
113+
- **Token Budgets**: Allocating context window space
114+
- **Retrieval Strategies**: Full context (bad), RAG (good), Summaries (compact), Hybrid (best)
115+
- **Grounding**: Resolving references (pronouns, descriptions, implicit)
116+
- **Tool Filtering**: Show only relevant tools based on intent
117+
- **Structured Views**: Pre-computed summaries for LLM consumption
118+
119+
## Production Patterns
120+
121+
### 1. Complete Memory Flow
122+
```python
123+
# Load working memory
124+
working_memory = await memory_client.get_working_memory(session_id, model_name)
125+
126+
# Search long-term memory
127+
memories = await memory_client.search_memories(query, limit=5)
128+
129+
# Build context
130+
system_prompt = build_prompt(instructions, memories)
131+
132+
# Process with LLM
133+
response = llm.invoke(messages)
134+
135+
# Save working memory (triggers extraction)
136+
await memory_client.save_working_memory(session_id, messages)
137+
```
138+
139+
### 2. Hybrid Retrieval
140+
```python
141+
# Pre-computed summary
142+
summary = load_catalog_summary()
143+
144+
# Targeted search
145+
specific_items = await search_courses(query, limit=3)
146+
147+
# Combine
148+
context = f"{summary}\n\nRelevant items:\n{specific_items}"
149+
```
150+
151+
### 3. Tool Filtering
152+
```python
153+
# Filter tools by intent
154+
relevant_tools = filter_tools_by_intent(query, tool_groups)
155+
156+
# Bind only relevant tools
157+
llm_with_tools = llm.bind_tools(relevant_tools)
158+
```
159+
160+
### 4. Token Budget Management
161+
```python
162+
# Estimate budget
163+
budget = estimate_token_budget(
164+
system_prompt=prompt,
165+
working_memory_messages=10,
166+
long_term_memories=5,
167+
retrieved_context_items=3
168+
)
169+
170+
# Check if within limits
171+
if budget['total_with_response'] > 128000:
172+
# Trigger summarization or reduce context
173+
```
174+
175+
### 5. Structured Views
176+
```python
177+
# Retrieve data
178+
items = await get_all_items()
179+
180+
# Summarize
181+
summary = await create_summary_view(items, group_by="category")
182+
183+
# Save for reuse
184+
redis_client.set("summary_view", summary)
185+
186+
# Use in prompts
187+
system_prompt = f"Overview:\n{summary}\n\nInstructions:..."
188+
```
189+
190+
## Usage in Notebooks
191+
192+
All patterns are demonstrated in notebooks with:
193+
- ✅ Conceptual explanations
194+
- ✅ Bad examples (what not to do)
195+
- ✅ Good examples (best practices)
196+
- ✅ Runnable code
197+
- ✅ Testing and verification
198+
- ✅ Exercises for practice
199+
200+
## Importing in Your Code
201+
202+
```python
203+
from redis_context_course import (
204+
# Core
205+
CourseManager,
206+
MemoryClient,
207+
208+
# Tools (Section 2)
209+
create_course_tools,
210+
create_memory_tools,
211+
select_tools_by_keywords,
212+
213+
# Optimizations (Section 4)
214+
count_tokens,
215+
estimate_token_budget,
216+
hybrid_retrieval,
217+
create_summary_view,
218+
create_user_profile_view,
219+
filter_tools_by_intent,
220+
classify_intent_with_llm,
221+
extract_references,
222+
format_context_for_llm,
223+
)
224+
```
225+
226+
## Learning Path
227+
228+
1. **Start with Section 1** - Understand fundamentals
229+
2. **Work through Section 2** - Build system context and tools
230+
3. **Master Section 3** - Implement memory management
231+
4. **Optimize with Section 4** - Apply production patterns
232+
5. **Study advanced_agent_example.py** - See it all together
233+
6. **Build your own agent** - Apply to your use case
234+
235+
## Key Takeaways
236+
237+
### What Makes a Production-Ready Agent?
238+
239+
1. **Clear System Instructions** - Tell the agent what to do
240+
2. **Well-Designed Tools** - Give it capabilities with clear descriptions
241+
3. **Memory Integration** - Remember context across sessions
242+
4. **Token Management** - Stay within limits efficiently
243+
5. **Smart Retrieval** - Hybrid approach (summary + RAG)
244+
6. **Tool Filtering** - Show only relevant tools
245+
7. **Structured Views** - Pre-compute summaries for efficiency
246+
247+
### Common Pitfalls to Avoid
248+
249+
**Don't:**
250+
- Include all tools on every request
251+
- Use vague tool descriptions
252+
- Ignore token budgets
253+
- Use only full context or only RAG
254+
- Forget to save working memory
255+
- Store everything in long-term memory
256+
257+
**Do:**
258+
- Filter tools by intent
259+
- Write detailed tool descriptions with examples
260+
- Estimate and monitor token usage
261+
- Use hybrid retrieval (summary + targeted search)
262+
- Save working memory to trigger extraction
263+
- Store only important facts in long-term memory
264+
265+
## Next Steps
266+
267+
After completing this course, you can:
268+
269+
1. **Extend the reference agent** - Add new tools and capabilities
270+
2. **Apply to your domain** - Adapt patterns to your use case
271+
3. **Optimize further** - Experiment with different strategies
272+
4. **Share your learnings** - Contribute back to the community
273+
274+
## Resources
275+
276+
- **Agent Memory Server Docs**: [Link to docs]
277+
- **Redis Documentation**: https://redis.io/docs
278+
- **LangChain Documentation**: https://python.langchain.com
279+
- **Course Repository**: [Link to repo]
280+
281+
---
282+
283+
**Course Version**: 1.0
284+
**Last Updated**: 2024-09-30
285+
**Total Notebooks**: 15 (3 intro + 3 system + 4 memory + 5 optimizations)
286+

0 commit comments

Comments
 (0)