Skip to content

Commit 9eaf263

Browse files
authored
Merge pull request #130 from Hack23/copilot/analyze-agentic-failure-workflows
Fix agentic workflow permission test and analyze coverage
2 parents be52c47 + 0e4424b commit 9eaf263

File tree

4 files changed

+456
-12
lines changed

4 files changed

+456
-12
lines changed

.github/workflows/news-article-generator.md

Lines changed: 75 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,9 @@ on:
2020
default: en,sv
2121

2222
permissions:
23-
contents: read
23+
contents: write
24+
pull-requests: write
2425
issues: read
25-
pull-requests: read
2626

2727
timeout-minutes: 30
2828

@@ -291,7 +291,51 @@ Create/update `news/metadata/last-generation.json`:
291291
}
292292
```
293293

294-
### Step 8: Create Pull Request
294+
### Step 8: Validate Generated Articles
295+
296+
Before creating the PR, validate the quality of generated articles:
297+
298+
**HTML Validation:**
299+
```bash
300+
# Validate HTML structure
301+
npx htmlhint news/*.html --config .htmlhintrc
302+
303+
# Check for common issues:
304+
# - Missing alt attributes
305+
# - Duplicate IDs
306+
# - Invalid nesting
307+
# - Missing required meta tags
308+
```
309+
310+
**Metadata Validation:**
311+
```bash
312+
# Verify all required meta tags exist:
313+
# - og:title, og:description, og:image, og:url
314+
# - article:published_time, article:author
315+
# - twitter:card, twitter:title, twitter:description
316+
# - Schema.org NewsArticle structured data
317+
318+
# Verify YAML frontmatter (if used)
319+
# - title, date, author, type, language
320+
```
321+
322+
**Link Checking:**
323+
```bash
324+
# Check all links in generated articles
325+
# - Internal links to Riksdag documents
326+
# - External sources
327+
# - Verify no broken links
328+
```
329+
330+
**Quality Criteria:**
331+
- ✅ All HTML validates without errors
332+
- ✅ All required meta tags present
333+
- ✅ No broken links
334+
- ✅ Proper heading hierarchy (h1 → h2 → h3)
335+
- ✅ Alt text on all images
336+
- ✅ Source citations with document IDs
337+
338+
### Step 9: Create Pull Request
295339

296340
Use safe-outputs to create a PR with:
297341

@@ -317,17 +361,29 @@ This PR contains automatically generated news articles from riksdag-regering-mcp
317361
- **32 specialized tools**: Documents, MPs, votes, speeches, calendar events
318362

319363
### Quality Checks
320-
- [x] HTML structure valid
364+
- [x] HTML validation passed
365+
- [x] Metadata validation passed
366+
- [x] No broken links detected
321367
- [x] SEO metadata complete
322368
- [x] Source attribution included
323369
- [x] Multi-language support (EN/SV)
370+
- [x] News indexes regenerated
324371
- [x] Sitemap updated
325372
- [ ] Editorial review recommended
326373

374+
### Validation Results
375+
```
376+
HTML Validation: PASSED (0 errors)
377+
Link Check: PASSED (0 broken links)
378+
Metadata: COMPLETE
379+
SEO Score: {score}/100
380+
```
381+
327382
### References
328383
- MCP Server: riksdag-regering-mcp (npm)
329384
- Data: Swedish Riksdag Open Data API
330385
- Style Guide: The Economist
386+
- Workflow: {workflow_run_url}
331387
332388
---
333389
*This PR was automatically created by the News Article Generator agent*
@@ -403,7 +459,7 @@ This PR contains automatically generated news articles from riksdag-regering-mcp
403459
**If no significant updates:**
404460
1. Check last-generation.json timestamp
405461
2. If < 11 hours and not forced, skip gracefully
406-
3. Create comment explaining: "No significant updates found"
462+
3. Use `noop` safe-output to log: "No significant updates found in riksdag-regering-mcp data. Last generation: {timestamp}. Use force_generation=true to override."
407463
4. Do not create PR
408464
5. Exit with success
409465

@@ -412,7 +468,20 @@ This PR contains automatically generated news articles from riksdag-regering-mcp
412468
2. Continue with remaining articles
413469
3. Include partial results in PR
414470
4. Mark failed articles in metadata
415-
5. Provide debugging information
471+
5. Provide debugging information in PR body
472+
473+
**If validation fails:**
474+
1. Log validation errors (HTML errors, broken links, missing metadata)
475+
2. Include validation report in PR body
476+
3. Add `needs-fixes` label to PR
477+
4. Do not block PR creation (editorial review can fix issues)
478+
5. Provide clear remediation steps
479+
480+
**Critical failures that should stop workflow:**
481+
- MCP server completely unavailable (> 3 retries)
482+
- File system errors (cannot write files)
483+
- Git operations failing (cannot commit/push)
484+
- Safe-outputs failing (cannot create PR)
416485

417486
## Example Queries
418487

0 commit comments

Comments
 (0)