Skip to content

Commit 39feccf

Browse files
Add Makefile target and agent guidance for updating template acceptance tests (#3762)
## Changes Adds a new `make test-update-templates` target for faster template acceptance test updates, and adds guidance to AGENTS.md on the proper workflow for updating acceptance tests. ## Tests Exercised with PRs like #3712. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <[email protected]>
1 parent 20dfac0 commit 39feccf

File tree

2 files changed

+51
-2
lines changed

2 files changed

+51
-2
lines changed

AGENTS.md

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,50 @@ test case in your responses. I am just interested in the tests.
152152
- Each test directory contains `databricks.yml`, `script`, and `output.txt`
153153
- Run with `go test ./acceptance -run TestAccept/bundle/<path>/<to>/<folder> -tail -test.v`
154154
- Use `-update` flag to regenerate expected output files
155-
- When you see the test fails because it has an old output, just run it one more time with an `-update` flag instead of changing the `output.txt` directly
155+
- When a test fails because it has an old output, just run it one more time with an `-update` flag instead of changing the `output.txt` directly
156+
157+
**When asked to update acceptance tests, follow this workflow**:
158+
159+
1. **Run the update command**:
160+
- For all acceptance tests: `make test-update`
161+
- When asked to update acceptance tests for templates specifically: `make test-update-templates`
162+
163+
2. **Verify code quality**:
164+
- Run `make fmt` and `make lint`
165+
- **Critical**: If these commands modify any files in `acceptance/`, this indicates an issue in the source files (e.g., in `libs/template/templates/` for template tests)!
166+
167+
3. **Fix the root cause**:
168+
- **Never manually edit files in `acceptance/`** - they are auto-generated outputs
169+
- Find and fix the corresponding source file that generated the problematic acceptance test output
170+
- For template tests: fix files in `libs/template/templates/`
171+
- Common issues: trailing whitespace, missing/extra newlines, formatting problems
172+
173+
4. **Regenerate after fixing**:
174+
- After fixing the source files, run the update command again (e.g., `make test-update-templates`)
175+
- This regenerates the acceptance test outputs from the corrected sources
176+
- Now `make fmt` and `make lint` should pass with no changes
177+
178+
**Example workflow**:
179+
```bash
180+
# Update acceptance tests
181+
make test-update # or make test-update-templates for templates only
182+
183+
# Check for issues - if these modify files in acceptance/, you have a source file problem
184+
make fmt
185+
make lint
186+
187+
# If there are modifications in acceptance/:
188+
# 1. Find the corresponding source file (e.g., in libs/template/templates/ for templates)
189+
# 2. Fix the issue there (e.g., whitespace, newlines)
190+
# 3. Regenerate from the fixed source
191+
make test-update # or make test-update-templates
192+
193+
# Verify everything is clean
194+
make fmt # Should show no changes now
195+
make lint # Should show no issues now
196+
```
197+
198+
**Key principle**: Files in `acceptance/` are outputs, not sources. Always fix the source files and regenerate.
156199

157200
# Logging
158201

Makefile

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,12 @@ test-update:
6363
@# at the moment second pass is required because some tests show diff against output of another test for easier review
6464
-go test ./acceptance -run '^TestAccept$$' -update -timeout=${LOCAL_TIMEOUT}
6565

66+
# Updates acceptance test output for template tests only
67+
test-update-templates:
68+
-go test ./acceptance -run '^TestAccept/bundle/templates' -update -timeout=${LOCAL_TIMEOUT}
69+
@# at the moment second pass is required because some tests show diff against output of another test for easier review
70+
-go test ./acceptance -run '^TestAccept/bundle/templates' -update -timeout=${LOCAL_TIMEOUT}
71+
6672
# Updates acceptance test output (integration tests, requires access)
6773
test-update-aws:
6874
deco env run -i -n aws-prod-ucws -- go test ./acceptance -run ^TestAccept$$ -update -timeout=1h -skiplocal -v
@@ -142,4 +148,4 @@ generate:
142148
$(GENKIT_BINARY) update-sdk
143149

144150

145-
.PHONY: lint lintfull tidy lintcheck fmt fmtfull test cover showcover build snapshot snapshot-release schema integration integration-short acc-cover acc-showcover docs ws links checks test-update test-update-aws test-update-all generate-validation
151+
.PHONY: lint lintfull tidy lintcheck fmt fmtfull test cover showcover build snapshot snapshot-release schema integration integration-short acc-cover acc-showcover docs ws links checks test-update test-update-templates test-update-aws test-update-all generate-validation

0 commit comments

Comments
 (0)