|
| 1 | +# JIT Template Rendering - Handoff Notes |
| 2 | + |
| 3 | +## Current State |
| 4 | +The JIT template rendering foundation is implemented and committed. The main components are: |
| 5 | + |
| 6 | +1. **TemplateMetadata** class in `src/orchestrator/core/template_metadata.py` - tracks template dependencies |
| 7 | +2. **Template analysis** in `YAMLCompiler._analyze_template()` - identifies which step IDs each template depends on |
| 8 | +3. **Task enhancement** - Task class now has `template_metadata` field to store analysis results |
| 9 | +4. **Control flow integration** - Fixed to preserve template metadata for conditional tasks |
| 10 | + |
| 11 | +## What's Working |
| 12 | +- Template dependency analysis correctly identifies dependencies |
| 13 | +- Basic pipelines (like `research_basic.yaml`) work correctly |
| 14 | +- The control system renders templates at runtime using the template manager |
| 15 | + |
| 16 | +## Current Issue |
| 17 | +The remaining issue is with conditional tasks that have both: |
| 18 | +1. Dependencies on other tasks |
| 19 | +2. A condition that uses results from those dependencies |
| 20 | + |
| 21 | +Example from `research_advanced_tools.yaml`: |
| 22 | +```yaml |
| 23 | +- id: extract_content |
| 24 | + dependencies: [search_topic, deep_search] |
| 25 | + condition: "{{ (search_topic.results | length > 0) or (deep_search.results | length > 0) }}" |
| 26 | +``` |
| 27 | +
|
| 28 | +The problem: We can't evaluate the condition until dependencies complete, but the current code tries to evaluate it too early. |
| 29 | +
|
| 30 | +## Solution Approach |
| 31 | +The solution is to ensure conditional tasks only evaluate their conditions AFTER their declared dependencies are satisfied. This requires modifying the execution flow in `orchestrator.py`. |
| 32 | + |
| 33 | +## Debugging Commands |
| 34 | +Test the implementation: |
| 35 | +```bash |
| 36 | +# Test basic pipeline (should work) |
| 37 | +python scripts/run_pipeline.py examples/research_basic.yaml -i topic="test" |
| 38 | +
|
| 39 | +# Test advanced pipeline (currently fails on conditional task) |
| 40 | +python scripts/run_pipeline.py examples/research_advanced_tools.yaml -i topic="test" |
| 41 | +``` |
| 42 | + |
| 43 | +## Key Files to Review |
| 44 | +1. `src/orchestrator/orchestrator.py` - Lines 483-503 where conditional task evaluation happens |
| 45 | +2. `src/orchestrator/control_flow/conditional.py` - The ConditionalTask.should_execute() method |
| 46 | +3. `src/orchestrator/core/control_system.py` - The _render_task_templates() method |
| 47 | + |
| 48 | +## Next Steps |
| 49 | +1. Ensure conditional tasks wait for dependencies before evaluating conditions |
| 50 | +2. Test all example pipelines to ensure they work |
| 51 | +3. Add unit tests for template analysis and rendering |
| 52 | +4. Update documentation |
| 53 | + |
| 54 | +The implementation is very close - just need to fix the timing of conditional task evaluation. |
0 commit comments