Migrate template system to Jinja2 #125

anticomputer · 2025-12-29T17:03:37Z

Replace custom regex-based template preprocessing with Jinja2.

Breaking Change

Version 1 YAML files are no longer supported. Bumped version to 0.1.0.

New syntax:

{{ globals.key }} instead of {{ GLOBALS_key }}
{{ inputs.key }} instead of {{ INPUTS_key }}
{{ result }} instead of {{ RESULT }}
{{ env('VAR') }} instead of {{ env VAR }}
{% include 'path' %} instead of {{ PROMPTS_path }}

Migration:

python scripts/migrate_to_jinja2.py /path/to/taskflows

See doc/MIGRATION.md for details.

New Capabilities

Jinja2 brings powerful templating features:

Conditionals:

user_prompt: |
  {% if result.severity == 'high' %}
  URGENT: Found critical vulnerability in {{ result.function }}
  {% else %}
  Review findings for {{ result.function }}
  {% endif %}

Filters:

user_prompt: |
  Analyze {{ result.code | length }} lines of code
  Function name: {{ result.name | upper }}
  Last modified: {{ result.timestamp | default('unknown') }}

Loops:

user_prompt: |
  Review these functions:
  {% for func in result.functions %}
  - {{ func.name }}: {{ func.complexity }} complexity
  {% endfor %}

Changes

New template_utils.py with Jinja2 integration
Migration script with dry-run support
Updated all YAML files to v2 syntax
Version validator rejects v1 files at load time
Fixed repeat_prompt rendering to skip initial render when result is undefined

Replace custom regex-based template preprocessing with Jinja2. - Add Jinja2 dependency - Create template_utils.py with loader, env function, and render function - Replace preprocess_prompt in __main__.py with render_template calls - Update repeat_prompt logic to use Jinja2 - Update swap_env in env_utils.py for Jinja2 New syntax (YAML files not yet migrated): - {{ GLOBALS_key }} → {{ globals.key }} - {{ INPUTS_key }} → {{ inputs.key }} - {{ RESULT }} → {{ result }} - {{ env VAR }} → {{ env('VAR') }} - {{ PROMPTS_path }} → {% include 'path' %}

- Create scripts/migrate_to_jinja2.py for automated migration - Migrate example taskflows and toolbox configs to new syntax - Script supports dry-run mode and recursive directory processing

- Update version validator to accept v1 (deprecated) and v2 - Bump all migrated YAML files to version 2 - Add deprecation warning for v1 files with migration instructions - Version 2 indicates Jinja2 templating syntax

- Fail fast: reject v1 files at load time with clear error message - Update README with breaking change notice - Create MIGRATION.md with comprehensive migration guide - Update GRAMMAR.md examples to use Jinja2 syntax

Skip initial template render when repeat_prompt is enabled since result variable is only available during iteration loop.

src/seclab_taskflow_agent/template_utils.py

+    env = jinja2.Environment(
+        loader=PromptLoader(available_tools),
+        # Use same delimiters as custom system
+        variable_start_string='{{',
+        variable_end_string='}}',
+        block_start_string='{%',
+        block_end_string='%}',
+        # Disable auto-escaping (YAML context doesn't need HTML escaping)
+        autoescape=False,
+        # Keep whitespace for prompt formatting
+        trim_blocks=True,
+        lstrip_blocks=True,
+        # Raise errors for undefined variables
+        undefined=jinja2.StrictUndefined,
+    )


tests/test_template_utils.py

+        """Test loading existing prompt."""
+        available_tools = AvailableTools()
+        loader = PromptLoader(available_tools)
+        env = jinja2.Environment(loader=loader)


tests/test_template_utils.py

+        """Test error on nonexistent prompt."""
+        available_tools = AvailableTools()
+        loader = PromptLoader(available_tools)
+        env = jinja2.Environment(loader=loader)


Fix remaining references to old {{ env VAR }} syntax in README and example comments. Remove unused pprint/pformat imports.

kevinbackhouse · 2026-01-05T10:51:42Z

I think this is a good idea. I'm wondering what we should do with version numbers though. Since we didn't publish our announcement blog post yet, I think it should still be ok to make backwards-incompatible changes. I think it'll be less confusing if the language version matches the release version. So we'll have language version 1 corresponding to releases starting with v1.0.0. If we need to upgrade the language version to 2 then we'll move the release to v2.0.0.

While we're at at, maybe we should also change the language version to a string? Then we could have language versions like "1.1". Like with the semver convention, the minor version would be for new features that don't break backwards-compatibility.

m-y-mo · 2026-01-05T11:37:17Z

@anticomputer Can you check the followings:

reusable prompts can contain Globals/inputs template parameters, could you make sure that they are rendered correctly after being included? i.e. I don't want something like {{ globals.key }} left in a prompt that's got included.
Make sure globals/input template parameters renders correctly in reusable taskflows (those that has a uses in the task)
Migrate everything here: https://github.com/GitHubSecurityLab/seclab-taskflows
Thanks

anticomputer · 2026-01-06T04:27:19Z

@anticomputer Can you check the followings:

reusable prompts can contain Globals/inputs template parameters, could you make sure that they are rendered correctly after being included? i.e. I don't want something like {{ globals.key }} left in a prompt that's got included.

Make sure globals/input template parameters renders correctly in reusable taskflows (those that has a uses in the task)

Migrate everything here: https://github.com/GitHubSecurityLab/seclab-taskflows
Thanks

Sure thing, I'll take another poke this weekend 👍

anticomputer · 2026-01-06T04:29:23Z

I think this is a good idea. I'm wondering what we should do with version numbers though. Since we didn't publish our announcement blog post yet, I think it should still be ok to make backwards-incompatible changes. I think it'll be less confusing if the language version matches the release version. So we'll have language version 1 corresponding to releases starting with v1.0.0. If we need to upgrade the language version to 2 then we'll move the release to v2.0.0.

While we're at at, maybe we should also change the language version to a string? Then we could have language versions like "1.1". Like with the semver convention, the minor version would be for new features that don't break backwards-compatibility.

Both of those make sense to me, I'll bump the versions back to v1 as part of addressing the asks @m-y-mo has 👍

anticomputer added 6 commits December 29, 2025 12:00

Add migration script and migrate YAML files to Jinja2 syntax

8188c35

- Create scripts/migrate_to_jinja2.py for automated migration - Migrate example taskflows and toolbox configs to new syntax - Script supports dry-run mode and recursive directory processing

Update YAML version scheme to v2 with deprecation warning

c00dc11

- Update version validator to accept v1 (deprecated) and v2 - Bump all migrated YAML files to version 2 - Add deprecation warning for v1 files with migration instructions - Version 2 indicates Jinja2 templating syntax

Reject v1 YAML files and add migration documentation

23eb693

- Fail fast: reject v1 files at load time with clear error message - Update README with breaking change notice - Create MIGRATION.md with comprehensive migration guide - Update GRAMMAR.md examples to use Jinja2 syntax

Update remaining YAML files to v2 and fix repeat_prompt rendering

98874d0

Skip initial template render when repeat_prompt is enabled since result variable is only available during iteration loop.

Bump version to 0.1.0 for breaking YAML API change

7807de4

anticomputer marked this pull request as ready for review December 29, 2025 17:19

anticomputer requested review from JarLob, Kwstubbs, kevinbackhouse, m-y-mo, p- and sylwia-budzynska as code owners December 29, 2025 17:19

anticomputer force-pushed the anticomputer/jinja-templating branch from 2d63570 to b9e1075 Compare December 29, 2025 19:15

github-advanced-security bot found potential problems Dec 30, 2025

View reviewed changes

anticomputer added 3 commits December 30, 2025 17:38

Update documentation to use new env template syntax

cff0449

Fix remaining references to old {{ env VAR }} syntax in README and example comments. Remove unused pprint/pformat imports.

Add test script and fix example_reusable_taskflows model

6be0c03

Detect rate limit errors in test script

57e49cc

anticomputer force-pushed the anticomputer/jinja-templating branch from b9e1075 to 57e49cc Compare December 30, 2025 22:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate template system to Jinja2 #125

Migrate template system to Jinja2 #125

Uh oh!

anticomputer commented Dec 29, 2025 •

edited

Loading

Uh oh!

Check warning

Check warning

Check warning

kevinbackhouse commented Jan 5, 2026

Uh oh!

m-y-mo commented Jan 5, 2026

Uh oh!

anticomputer commented Jan 6, 2026

Uh oh!

anticomputer commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Migrate template system to Jinja2 #125

Are you sure you want to change the base?

Migrate template system to Jinja2 #125

Uh oh!

Conversation

anticomputer commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Breaking Change

New Capabilities

Changes

Uh oh!

Check warning

Check warning

Check warning

kevinbackhouse commented Jan 5, 2026

Uh oh!

m-y-mo commented Jan 5, 2026

Uh oh!

anticomputer commented Jan 6, 2026

Uh oh!

anticomputer commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

anticomputer commented Dec 29, 2025 •

edited

Loading