Skip to content

feat: expand digdag skill references and consolidate workflow-management#153

Open
szyn wants to merge 1 commit intomainfrom
feat/improve-workflow-skill-references
Open

feat: expand digdag skill references and consolidate workflow-management#153
szyn wants to merge 1 commit intomainfrom
feat/improve-workflow-skill-references

Conversation

@szyn
Copy link
Copy Markdown
Member

@szyn szyn commented Apr 9, 2026

Summary

  • Expand operators.md with additional operators, Secrets Reference table, and output variables
  • Add new reference files: scheduling.md and runtime.md
  • Consolidate workflow-management into digdag skill references and remove as separate skill
  • Replace docs.digdag.io references with docs.treasuredata.com

Background

workflow-management was removed as a separate skill because:

  1. Most code examples used sh> which is not available on TD platform
  2. CLI debugging steps duplicated content in tdx-skills/workflow
  3. Useful operational content (variable caveats, secret expansion rules, safe data refresh, system limits) belonged topically in digdag's reference files rather than a separate skill

Correct content was migrated into the appropriate reference files. Incorrect examples were replaced with correct patterns.

Test plan

  • Run trigger tests: ./tests/run-tests.sh
  • Verify digdag skill triggers correctly
  • Check all reference file links resolve

🤖 Generated with Claude Code

@github-actions github-actions bot added the tdx:core-skills Touches core skills: tdx-skills, sql-skills, workflow-skills label Apr 9, 2026
@szyn szyn force-pushed the feat/improve-workflow-skill-references branch from c5733b2 to 3a3e50b Compare April 9, 2026 07:23

```yaml
+delete_existing:
td_partial_delete>: daily_summary
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

td_partial_delete operator has been already deprecated. See also this release note

@szyn szyn force-pushed the feat/improve-workflow-skill-references branch from 3a3e50b to 4dd79b3 Compare April 9, 2026 07:27
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@szyn szyn force-pushed the feat/improve-workflow-skill-references branch from 4dd79b3 to aeb457d Compare April 9, 2026 08:09
@szyn szyn marked this pull request as ready for review April 13, 2026 05:59
@szyn szyn requested review from a team as code owners April 13, 2026 05:59
Copilot AI review requested due to automatic review settings April 13, 2026 05:59
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands and reorganizes the Digdag (Treasure Workflow) documentation skill by migrating operational “workflow-management” guidance into Digdag reference docs, adding new scheduling/runtime references, and updating doc links to Treasure Data’s docs site.

Changes:

  • Removed the standalone workflow-management skill and consolidated its content into Digdag references.
  • Expanded Digdag reference documentation (operators, ETL patterns, scheduling, runtime/limits) and updated official documentation links.
  • Updated skill registry/docs/tests to reflect the consolidated skill set.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
workflow-skills/workflow-management/SKILL.md Removes the standalone workflow-management skill content (migration/consolidation).
workflow-skills/digdag/SKILL.md Updates Digdag skill description, operator overview, built-in variables, retry notes, and adds links to new reference docs.
workflow-skills/digdag/references/scheduling.md Adds a dedicated scheduling reference (types, options, SLA, timezone notes).
workflow-skills/digdag/references/runtime.md Adds runtime behavior notes (variables, secrets, concurrency, system limits).
workflow-skills/digdag/references/patterns-etl.md Updates ETL patterns (idempotent refresh guidance, polling guidance).
workflow-skills/digdag/references/operators.md Major expansion of operator reference content and a new secrets reference table.
tests/trigger-tests.yml Removes the trigger test case for the removed workflow-management skill.
README.md Removes workflow-management from the documented skill list and examples.
CLAUDE.md Updates the repository tree to reflect the consolidated workflow skills structure.
aps-doc-skills/core/SKILL.md Updates the Digdag documentation resource link to Treasure Workflow docs.
.claude-plugin/marketplace.json Removes workflow-management from the workflow-skills plugin registration and updates description accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +11 to +27

# Daily at 9am
schedule:
daily>: "09:00:00"

# Weekly on Monday
schedule:
weekly>: Mon,09:00:00

# Monthly on the 1st
schedule:
monthly>: 1,09:00:00

# Cron (quote if starts with *)
schedule:
cron>: "*/15 * * * *"

WHERE TD_TIME_RANGE(time, '${session_time}')
```

For external systems, use `http>` with an API that returns `408` or `429` until conditions are met — `http>` retries on these status codes automatically (up to the 24h task limit).
@@ -260,111 +352,75 @@ Poll a table until it has enough records in the session time range.
| `interval:` | 30s | Polling frequency |
| `engine:` | — | `presto` or `hive` |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tdx:core-skills Touches core skills: tdx-skills, sql-skills, workflow-skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants