|
| 1 | +# Context Engineering Course - Complete Summary |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This course teaches production-ready context engineering for AI agents using Redis and the Agent Memory Server. It covers everything from fundamentals to advanced optimization techniques. |
| 6 | + |
| 7 | +## Course Structure |
| 8 | + |
| 9 | +### Section 1: Introduction (3 notebooks) |
| 10 | +1. **What is Context Engineering?** - Core concepts and importance |
| 11 | +2. **Setting Up Your Environment** - Installation and configuration |
| 12 | +3. **Project Overview** - Understanding the reference agent |
| 13 | + |
| 14 | +### Section 2: System Context (3 notebooks) |
| 15 | +1. **System Instructions** - Crafting effective system prompts |
| 16 | +2. **Defining Tools** - Giving agents capabilities |
| 17 | +3. **Tool Selection Strategies** (Advanced) - Improving tool choice |
| 18 | + |
| 19 | +**Key Patterns:** |
| 20 | +- Progressive system prompt building |
| 21 | +- Tool schema design with examples |
| 22 | +- Clear naming conventions |
| 23 | +- Detailed descriptions with when/when-not guidance |
| 24 | + |
| 25 | +### Section 3: Memory (4 notebooks) |
| 26 | +1. **Working Memory with Extraction Strategies** - Session-scoped context |
| 27 | +2. **Long-term Memory** - Cross-session knowledge |
| 28 | +3. **Memory Integration** - Combining working and long-term memory |
| 29 | +4. **Memory Tools** (Advanced) - LLM control over memory |
| 30 | + |
| 31 | +**Key Patterns:** |
| 32 | +- Automatic memory extraction |
| 33 | +- Semantic search for retrieval |
| 34 | +- Memory type selection (semantic vs episodic) |
| 35 | +- Tool-based memory management |
| 36 | + |
| 37 | +### Section 4: Optimizations (5 notebooks) |
| 38 | +1. **Context Window Management** - Handling token limits |
| 39 | +2. **Retrieval Strategies** - RAG, summaries, and hybrid approaches |
| 40 | +3. **Grounding with Memory** - Using memory to resolve references |
| 41 | +4. **Tool Optimization** (Advanced) - Selective tool exposure |
| 42 | +5. **Crafting Data for LLMs** (Advanced) - Creating structured views |
| 43 | + |
| 44 | +**Key Patterns:** |
| 45 | +- Token budget estimation |
| 46 | +- Hybrid retrieval (summary + RAG) |
| 47 | +- Tool filtering by intent |
| 48 | +- Retrieve → Summarize → Stitch → Save pattern |
| 49 | +- Structured view creation |
| 50 | + |
| 51 | +## Reference Agent Components |
| 52 | + |
| 53 | +### Core Modules |
| 54 | + |
| 55 | +**`course_manager.py`** |
| 56 | +- Course catalog management |
| 57 | +- Vector search for courses |
| 58 | +- Course data models |
| 59 | + |
| 60 | +**`memory_client.py`** |
| 61 | +- Working memory operations |
| 62 | +- Long-term memory operations |
| 63 | +- Integration with Agent Memory Server |
| 64 | + |
| 65 | +**`agent.py`** |
| 66 | +- Main agent implementation |
| 67 | +- LangGraph workflow |
| 68 | +- State management |
| 69 | + |
| 70 | +### New Modules (From Course Content) |
| 71 | + |
| 72 | +**`tools.py`** (Section 2) |
| 73 | +- `create_course_tools()` - Search, get details, check prerequisites |
| 74 | +- `create_memory_tools()` - Store and search memories |
| 75 | +- `select_tools_by_keywords()` - Simple tool filtering |
| 76 | + |
| 77 | +**`optimization_helpers.py`** (Section 4) |
| 78 | +- `count_tokens()` - Token counting for any model |
| 79 | +- `estimate_token_budget()` - Budget breakdown |
| 80 | +- `hybrid_retrieval()` - Combine summary + search |
| 81 | +- `create_summary_view()` - Structured summaries |
| 82 | +- `create_user_profile_view()` - User profile generation |
| 83 | +- `filter_tools_by_intent()` - Keyword-based filtering |
| 84 | +- `classify_intent_with_llm()` - LLM-based classification |
| 85 | +- `extract_references()` - Find grounding needs |
| 86 | +- `format_context_for_llm()` - Combine context sources |
| 87 | + |
| 88 | +### Examples |
| 89 | + |
| 90 | +**`examples/advanced_agent_example.py`** |
| 91 | +- Complete agent using all patterns |
| 92 | +- Tool filtering enabled |
| 93 | +- Token budget tracking |
| 94 | +- Memory integration |
| 95 | +- Production-ready structure |
| 96 | + |
| 97 | +## Key Concepts by Section |
| 98 | + |
| 99 | +### Section 2: System Context |
| 100 | +- **System vs Retrieved Context**: Static instructions vs dynamic data |
| 101 | +- **Tool Schemas**: Name, description, parameters |
| 102 | +- **Tool Selection**: How LLMs choose tools |
| 103 | +- **Best Practices**: Clear names, detailed descriptions, examples |
| 104 | + |
| 105 | +### Section 3: Memory |
| 106 | +- **Working Memory**: Session-scoped, conversation history |
| 107 | +- **Long-term Memory**: User-scoped, persistent facts |
| 108 | +- **Memory Types**: Semantic (facts), Episodic (events), Message (conversations) |
| 109 | +- **Automatic Extraction**: Agent Memory Server extracts important facts |
| 110 | +- **Memory Flow**: Load → Search → Process → Save → Extract |
| 111 | + |
| 112 | +### Section 4: Optimizations |
| 113 | +- **Token Budgets**: Allocating context window space |
| 114 | +- **Retrieval Strategies**: Full context (bad), RAG (good), Summaries (compact), Hybrid (best) |
| 115 | +- **Grounding**: Resolving references (pronouns, descriptions, implicit) |
| 116 | +- **Tool Filtering**: Show only relevant tools based on intent |
| 117 | +- **Structured Views**: Pre-computed summaries for LLM consumption |
| 118 | + |
| 119 | +## Production Patterns |
| 120 | + |
| 121 | +### 1. Complete Memory Flow |
| 122 | +```python |
| 123 | +# Load working memory |
| 124 | +working_memory = await memory_client.get_working_memory(session_id, model_name) |
| 125 | + |
| 126 | +# Search long-term memory |
| 127 | +memories = await memory_client.search_memories(query, limit=5) |
| 128 | + |
| 129 | +# Build context |
| 130 | +system_prompt = build_prompt(instructions, memories) |
| 131 | + |
| 132 | +# Process with LLM |
| 133 | +response = llm.invoke(messages) |
| 134 | + |
| 135 | +# Save working memory (triggers extraction) |
| 136 | +await memory_client.save_working_memory(session_id, messages) |
| 137 | +``` |
| 138 | + |
| 139 | +### 2. Hybrid Retrieval |
| 140 | +```python |
| 141 | +# Pre-computed summary |
| 142 | +summary = load_catalog_summary() |
| 143 | + |
| 144 | +# Targeted search |
| 145 | +specific_items = await search_courses(query, limit=3) |
| 146 | + |
| 147 | +# Combine |
| 148 | +context = f"{summary}\n\nRelevant items:\n{specific_items}" |
| 149 | +``` |
| 150 | + |
| 151 | +### 3. Tool Filtering |
| 152 | +```python |
| 153 | +# Filter tools by intent |
| 154 | +relevant_tools = filter_tools_by_intent(query, tool_groups) |
| 155 | + |
| 156 | +# Bind only relevant tools |
| 157 | +llm_with_tools = llm.bind_tools(relevant_tools) |
| 158 | +``` |
| 159 | + |
| 160 | +### 4. Token Budget Management |
| 161 | +```python |
| 162 | +# Estimate budget |
| 163 | +budget = estimate_token_budget( |
| 164 | + system_prompt=prompt, |
| 165 | + working_memory_messages=10, |
| 166 | + long_term_memories=5, |
| 167 | + retrieved_context_items=3 |
| 168 | +) |
| 169 | + |
| 170 | +# Check if within limits |
| 171 | +if budget['total_with_response'] > 128000: |
| 172 | + # Trigger summarization or reduce context |
| 173 | +``` |
| 174 | + |
| 175 | +### 5. Structured Views |
| 176 | +```python |
| 177 | +# Retrieve data |
| 178 | +items = await get_all_items() |
| 179 | + |
| 180 | +# Summarize |
| 181 | +summary = await create_summary_view(items, group_by="category") |
| 182 | + |
| 183 | +# Save for reuse |
| 184 | +redis_client.set("summary_view", summary) |
| 185 | + |
| 186 | +# Use in prompts |
| 187 | +system_prompt = f"Overview:\n{summary}\n\nInstructions:..." |
| 188 | +``` |
| 189 | + |
| 190 | +## Usage in Notebooks |
| 191 | + |
| 192 | +All patterns are demonstrated in notebooks with: |
| 193 | +- ✅ Conceptual explanations |
| 194 | +- ✅ Bad examples (what not to do) |
| 195 | +- ✅ Good examples (best practices) |
| 196 | +- ✅ Runnable code |
| 197 | +- ✅ Testing and verification |
| 198 | +- ✅ Exercises for practice |
| 199 | + |
| 200 | +## Importing in Your Code |
| 201 | + |
| 202 | +```python |
| 203 | +from redis_context_course import ( |
| 204 | + # Core |
| 205 | + CourseManager, |
| 206 | + MemoryClient, |
| 207 | + |
| 208 | + # Tools (Section 2) |
| 209 | + create_course_tools, |
| 210 | + create_memory_tools, |
| 211 | + select_tools_by_keywords, |
| 212 | + |
| 213 | + # Optimizations (Section 4) |
| 214 | + count_tokens, |
| 215 | + estimate_token_budget, |
| 216 | + hybrid_retrieval, |
| 217 | + create_summary_view, |
| 218 | + create_user_profile_view, |
| 219 | + filter_tools_by_intent, |
| 220 | + classify_intent_with_llm, |
| 221 | + extract_references, |
| 222 | + format_context_for_llm, |
| 223 | +) |
| 224 | +``` |
| 225 | + |
| 226 | +## Learning Path |
| 227 | + |
| 228 | +1. **Start with Section 1** - Understand fundamentals |
| 229 | +2. **Work through Section 2** - Build system context and tools |
| 230 | +3. **Master Section 3** - Implement memory management |
| 231 | +4. **Optimize with Section 4** - Apply production patterns |
| 232 | +5. **Study advanced_agent_example.py** - See it all together |
| 233 | +6. **Build your own agent** - Apply to your use case |
| 234 | + |
| 235 | +## Key Takeaways |
| 236 | + |
| 237 | +### What Makes a Production-Ready Agent? |
| 238 | + |
| 239 | +1. **Clear System Instructions** - Tell the agent what to do |
| 240 | +2. **Well-Designed Tools** - Give it capabilities with clear descriptions |
| 241 | +3. **Memory Integration** - Remember context across sessions |
| 242 | +4. **Token Management** - Stay within limits efficiently |
| 243 | +5. **Smart Retrieval** - Hybrid approach (summary + RAG) |
| 244 | +6. **Tool Filtering** - Show only relevant tools |
| 245 | +7. **Structured Views** - Pre-compute summaries for efficiency |
| 246 | + |
| 247 | +### Common Pitfalls to Avoid |
| 248 | + |
| 249 | +❌ **Don't:** |
| 250 | +- Include all tools on every request |
| 251 | +- Use vague tool descriptions |
| 252 | +- Ignore token budgets |
| 253 | +- Use only full context or only RAG |
| 254 | +- Forget to save working memory |
| 255 | +- Store everything in long-term memory |
| 256 | + |
| 257 | +✅ **Do:** |
| 258 | +- Filter tools by intent |
| 259 | +- Write detailed tool descriptions with examples |
| 260 | +- Estimate and monitor token usage |
| 261 | +- Use hybrid retrieval (summary + targeted search) |
| 262 | +- Save working memory to trigger extraction |
| 263 | +- Store only important facts in long-term memory |
| 264 | + |
| 265 | +## Next Steps |
| 266 | + |
| 267 | +After completing this course, you can: |
| 268 | + |
| 269 | +1. **Extend the reference agent** - Add new tools and capabilities |
| 270 | +2. **Apply to your domain** - Adapt patterns to your use case |
| 271 | +3. **Optimize further** - Experiment with different strategies |
| 272 | +4. **Share your learnings** - Contribute back to the community |
| 273 | + |
| 274 | +## Resources |
| 275 | + |
| 276 | +- **Agent Memory Server Docs**: [Link to docs] |
| 277 | +- **Redis Documentation**: https://redis.io/docs |
| 278 | +- **LangChain Documentation**: https://python.langchain.com |
| 279 | +- **Course Repository**: [Link to repo] |
| 280 | + |
| 281 | +--- |
| 282 | + |
| 283 | +**Course Version**: 1.0 |
| 284 | +**Last Updated**: 2024-09-30 |
| 285 | +**Total Notebooks**: 15 (3 intro + 3 system + 4 memory + 5 optimizations) |
| 286 | + |
0 commit comments