Skip to content

Commit 172fc34

Browse files
dguidoclaude
andauthored
Add test to detect inline comments in Jinja2 expressions within YAML files (#14817)
* Add test to detect inline comments in Jinja2 expressions within YAML files This test would have caught the bug reported where inline comments (#) within Jinja2 expressions in YAML task files caused Ansible template errors. The test: - Extracts and validates all Jinja2 expressions from YAML files - Specifically detects inline comments within {{ }} and {% %} blocks - Includes regression test for the exact reported bug pattern - Avoids false positives (# in strings, escaped #, comments outside expressions) - Focuses on the critical inline comment issue The original bug was in roles/strongswan/tasks/openssl.yml where comments like "# Per-deployment UUID..." were placed inside a Jinja2 expression, causing "unexpected char '#'" errors during playbook execution. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Refactor test to use pytest framework and add comprehensive edge cases - Converted standalone script to proper pytest test functions - Replaced main() with individual test functions using pytest assertions - Added comprehensive edge case tests for inline comment detection: * Hash symbols in strings (should pass) * Escaped hashes (should pass) * Comments in control blocks (should fail) * Multi-line expressions with comments (should fail) * URL fragments and hex colors (should pass) - Test functions now properly integrate with pytest: * test_regression_openssl_inline_comments() - regression test * test_edge_cases_inline_comments() - comprehensive edge cases * test_yaml_files_no_inline_comments() - scan all YAML files * test_openssl_file_specifically() - test the originally buggy file This addresses the review feedback about pytest integration and adds the suggested test cases for better coverage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Fix linter issues in test_yaml_jinja2_expressions.py - Fixed trailing whitespace issues (W293) - Applied ruff formatting for consistent code style - All tests still pass after formatting changes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Add mutation testing guidance to CLAUDE.md Added a section on writing effective tests that emphasizes the importance of verifying that tests actually detect failure cases. This lightweight mutation testing approach ensures: - Tests catch the specific bugs they're designed to prevent - We avoid false confidence from tests that always pass - Test purposes are clear and documented - Both success and failure cases are validated The guidance includes a concrete example from our recent inline comment detection test, showing how to verify both the problematic pattern (should fail) and the fixed pattern (should pass). 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> --------- Co-authored-by: Claude <[email protected]>
1 parent 2ab57c3 commit 172fc34

File tree

2 files changed

+426
-0
lines changed

2 files changed

+426
-0
lines changed

CLAUDE.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,34 @@ shellcheck *.sh
141141
pwsh -Command "Invoke-ScriptAnalyzer -Path ./algo.ps1"
142142
```
143143

144+
### Writing Effective Tests - Mutation Testing Approach
145+
146+
When writing tests, **always verify that your test actually detects the failure case**. This is a form of lightweight mutation testing that ensures tests add real value:
147+
148+
1. **Write the test for the bug/issue you're preventing**
149+
2. **Temporarily introduce the bug** to verify the test fails
150+
3. **Fix the bug** and verify the test passes
151+
4. **Document what specific issue the test prevents**
152+
153+
Example from our codebase:
154+
```python
155+
def test_regression_openssl_inline_comments():
156+
"""Tests that we detect inline comments in Jinja2 expressions."""
157+
# This pattern SHOULD fail (has inline comments)
158+
problematic = "{{ ['DNS:' + id, # comment ] }}"
159+
assert not validate(problematic), "Should detect inline comments"
160+
161+
# This pattern SHOULD pass (no inline comments)
162+
fixed = "{{ ['DNS:' + id] }}"
163+
assert validate(fixed), "Should pass without comments"
164+
```
165+
166+
This practice ensures:
167+
- Tests aren't just checking happy paths
168+
- Tests will actually catch regressions
169+
- The test's purpose is clear to future maintainers
170+
- We avoid false confidence from tests that always pass
171+
144172
## Common Issues and Solutions
145173

146174
### 1. Ansible-lint "name[missing]" Warnings

0 commit comments

Comments
 (0)