Skip to content

Commit 5d1e3a4

Browse files
Piotr1215Piotr1215
authored
feat(ai-step): new small composite action for structured AI output (#131)
* feat(ai-step): new small composite action for structured AI output Callers want an ai-in / JSON-out primitive: pass a prompt + input + JSON Schema, get back schema-conforming JSON on steps.*.outputs.result that downstream steps can branch on via fromJSON(). ai-pr-review owns the PR-review preset (comments, sticky summary, provenance) — the wrong shape for "classify this diff," "pick a reviewer," or "extract fields from a changelog." Generalizing ai-pr-review would bloat its input surface and couple unrelated concerns. ai-step is the small building block: wraps claude-code-action with --json-schema and codex-action with output-schema, unifies the outputs, never hard-fails. The caller decides how to react to empty or unexpected JSON — structured output is the contract, caller owns the response policy. Closes DEVOPS-834 * fix(ai-step): strip literal ${{ }} from input description GitHub Actions parses any ${{ ... }} it sees inside action.yml — including prose in a description field — as an expression. The placeholder example in the 'input' description triggered: Unrecognized named-value: 'steps' on every caller, breaking composite-action loading before it ran. Rewrite the description in plain prose; the usage examples in the README keep the templated form. * fix(ai-step): pass github.token to avoid OIDC requirement claude-code-action tries to exchange an OIDC token for a GitHub App token on startup unless github_token is explicitly provided. That fails with 'Could not fetch an OIDC token' in any caller workflow that doesn't set id-token: write — a surprising requirement for what is meant to be a pure text-to-JSON primitive with no PR interaction. Passing github.token (always available to composite actions) makes id-token: write unnecessary for ai-step callers. * refactor(ai-step): call provider APIs directly via python SDKs The claude-code-action wrapper was the wrong dependency for a generic text-to-JSON primitive: it ships a full PR-review pipeline (bun install, sandboxing, review-mode cleanup) that we don't need, which cost ~90s cold-start per call and exposed bugs unrelated to schema binding (tsconfig directory-mismatch; --json-schema hangs when combined with default output-format; OIDC dance even for pure API calls). Both providers ship first-class structured-output support on their chat APIs — Anthropic via output_config.format.schema on the Messages API, OpenAI via response_format.json_schema.schema on Chat Completions. Using the official python SDKs gives us native schema binding, proper error types, and a ~3-5s end-to-end call in place of ~90s, with none of the wrapper-specific failure modes. Losing tool-use/MCP support in v1 is intentional; callers who need it can reach for claude-code-action directly. * fix(ai-step): auto-set additionalProperties=false on object schemas Anthropic structured-output (and OpenAI strict mode) reject schemas where an object type doesn't explicitly set additionalProperties=false. First smoke call failed with 'For object type, additionalProperties must be explicitly set to false' — annoying for every caller to repeat on every nested object. Walk the parsed schema once before sending and set the flag where missing, leaving explicit values alone. * docs(ai-step): refresh for direct-SDK internals + drop unsupported schema fields Previous docs described a claude-code-action / codex-action wrapper and listed tool/mcp-config inputs we removed when we moved to direct SDK calls. Also relax the in-repo smoke schema to drop numeric min/max so the same schema works on Anthropic strict mode.
1 parent a889ee9 commit 5d1e3a4

10 files changed

Lines changed: 971 additions & 2 deletions

File tree

‎.github/actions/ai-step/README.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# AI step
2+
3+
Small reusable building block for CI: run an AI call with a
4+
caller-supplied prompt and input, bind the output to a JSON Schema,
5+
expose the schema-conforming JSON as a step output. Downstream steps
6+
parse with `fromJSON(steps.<id>.outputs.result)` and branch on typed
7+
fields.
8+
9+
The contract is the schema. Whatever the model returns, the action
10+
exposes it on `result` and sets `conclusion=success`. The action never
11+
emits `failed` — the caller knows what empty or unexpected output means
12+
for their pipeline, and decides whether to continue or `exit 1`.
13+
14+
## When to use this vs `ai-pr-review`
15+
16+
- **`ai-pr-review`** — job-shaped reusable workflow for reviewing PRs.
17+
Owns checkout, commenting, sticky summaries, provenance footer.
18+
- **`ai-step`** — step-shaped primitive for any AI-in / JSON-out flow.
19+
No PR awareness, no checkout, no write permissions. Classify a diff,
20+
extract fields from a changelog, pick a reviewer, summarize release
21+
notes — anywhere you want the model's answer as typed JSON a later
22+
step can branch on.
23+
24+
## Effort → model
25+
26+
| Effort | Anthropic | OpenAI |
27+
|--------|----------------------|-----------------|
28+
| low | `claude-haiku-4-5` | `gpt-5.4-mini` |
29+
| medium | `claude-sonnet-4-6` | `gpt-5.3-codex` |
30+
| high | `claude-opus-4-7` | `gpt-5.4` |
31+
32+
## Usage
33+
34+
```yaml
35+
jobs:
36+
classify-diff:
37+
runs-on: ubuntu-latest
38+
steps:
39+
- uses: actions/checkout@v4
40+
with:
41+
repository: loft-sh/github-actions
42+
ref: ai-step/v1
43+
sparse-checkout: .github/actions/ai-step
44+
persist-credentials: false
45+
46+
- id: diff
47+
run: |
48+
{
49+
echo 'text<<EOF'
50+
git diff origin/main...HEAD
51+
echo 'EOF'
52+
} >> "$GITHUB_OUTPUT"
53+
54+
- id: classify
55+
uses: ./.github/actions/ai-step
56+
with:
57+
provider: anthropic
58+
effort: low
59+
prompt: |
60+
Classify this diff. Return JSON matching the schema.
61+
input: ${{ steps.diff.outputs.text }}
62+
output-schema: |
63+
{
64+
"type": "object",
65+
"required": ["severity", "areas"],
66+
"properties": {
67+
"severity": { "type": "string", "enum": ["low","medium","high"] },
68+
"areas": { "type": "array", "items": { "type": "string" } }
69+
}
70+
}
71+
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
72+
73+
- if: steps.classify.outputs.result != '' && fromJSON(steps.classify.outputs.result).severity == 'high'
74+
run: echo "needs human review"
75+
```
76+
77+
## How it works
78+
79+
The action installs the `anthropic` or `openai` Python SDK on the
80+
runner, then calls the provider's chat API directly with its native
81+
structured-output binding:
82+
83+
- **Anthropic** → Messages API with `output_config.format.schema`
84+
- **OpenAI** → Chat Completions with `response_format.json_schema.schema`
85+
86+
No `claude-code-action`, no `codex-action`, no bun install. End-to-end
87+
call is ~15s including SDK install; the LLM call itself is 2–4s. The
88+
action never hard-fails: API errors, empty responses, and non-JSON
89+
content all degrade to `conclusion=failed` with the upstream body
90+
preserved in the CI log. Caller decides how to react.
91+
92+
### Schema compatibility
93+
94+
Strict structured-output modes on both providers reject some JSON
95+
Schema features:
96+
97+
- `minimum`, `maximum`, `minLength`, `maxLength`, `pattern` — rejected
98+
- recursive schemas, `$ref` across documents — rejected
99+
- objects: `additionalProperties` must be `false` (the action sets this
100+
automatically on any object node where it's missing, so you don't
101+
have to repeat it in every nested schema)
102+
103+
Structured output guarantees the **shape** of the result (fields
104+
present, types match, enums respected). It does NOT guarantee semantic
105+
correctness of the values — that's the model's reasoning. Validate
106+
ranges and business rules in your downstream step, not in the schema.
107+
108+
### Tool use / MCP
109+
110+
Not supported in v1. If you need Claude Code tools, MCP servers, or
111+
filesystem access during the reasoning step, reach for
112+
`anthropics/claude-code-action` directly — `ai-step` is the minimal
113+
text-to-JSON primitive.
114+
115+
## Inputs
116+
117+
<!-- AUTO-DOC-INPUT:START - Do not remove or modify this section -->
118+
119+
| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
120+
|-------------------|--------|----------|------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
121+
| anthropic-api-key | string | false | | Anthropic API key. Required when provider=anthropic. |
122+
| effort | string | false | `"medium"` | Effort level (low | medium | high) — maps to <br>a provider-specific model. |
123+
| input | string | false | | Optional data the model should act <br>on, appended to the prompt. Caller <br>sources it — a literal string, <br>a prior step output, or the <br>contents of a file read in <br>a prior step. |
124+
| openai-api-key | string | false | | OpenAI API key. Required when provider=openai. |
125+
| output-schema | string | true | | JSON Schema (string) the model output <br>must conform to. Required. Structured output <br>is the contract — without a <br>schema the action skips. For `anthropic` this <br>becomes `output_format.schema` on the Messages API; for <br>`openai` it becomes `response_format.json_schema.schema` |
126+
| | | | | on the Chat Completions <br>API. |
127+
| prompt | string | true | | Instructions for the model. Passed verbatim. |
128+
| provider | string | true | | AI provider: `anthropic` or `openai`. |
129+
130+
<!-- AUTO-DOC-INPUT:END -->
131+
132+
## Outputs
133+
134+
<!-- AUTO-DOC-OUTPUT:START - Do not remove or modify this section -->
135+
136+
| OUTPUT | TYPE | DESCRIPTION |
137+
|------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
138+
| conclusion | string | `success` when the AI step ran <br>and returned JSON; `skipped` when the <br>resolver vetoed the input; `failed` when <br>the provider errored or returned non-JSON. |
139+
| reason | string | One-line explanation when conclusion=skipped. |
140+
| result | string | Schema-conforming JSON string. Parse with `fromJSON(...)` <br>in downstream `if:` conditions. Empty when <br>`conclusion` is not `success`. |
141+
142+
<!-- AUTO-DOC-OUTPUT:END -->
143+
144+
## Testing
145+
146+
```bash
147+
make test-ai-step
148+
```
149+
150+
Runs the bats suite in `test/` against `src/resolve-config.sh`.

‎.github/actions/ai-step/action.yml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: AI step
2+
description: |
3+
Small reusable building block: run an AI call with a caller-supplied
4+
prompt and input, bind the output to a JSON Schema, expose the
5+
schema-conforming JSON as a step output. Downstream steps parse with
6+
fromJSON(steps.<id>.outputs.result) and branch on typed fields.
7+
8+
Calls the provider's chat API directly (Anthropic Messages API or
9+
OpenAI Chat Completions API) with native structured-output binding.
10+
No bun, no claude-code-action wrapper, no PR-review machinery.
11+
inputs:
12+
provider:
13+
description: 'AI provider: `anthropic` or `openai`.'
14+
required: true
15+
effort:
16+
description: 'Effort level (low | medium | high) — maps to a provider-specific model.'
17+
required: false
18+
default: 'medium'
19+
prompt:
20+
description: 'Instructions for the model. Passed verbatim.'
21+
required: true
22+
input:
23+
description: |
24+
Optional data the model should act on, appended to the prompt.
25+
Caller sources it — a literal string, a prior step output, or
26+
the contents of a file read in a prior step.
27+
required: false
28+
default: ''
29+
output-schema:
30+
description: |
31+
JSON Schema (string) the model output must conform to. Required.
32+
Structured output is the contract — without a schema the action
33+
skips. For `anthropic` this becomes `output_format.schema` on the
34+
Messages API; for `openai` it becomes `response_format.json_schema.schema`
35+
on the Chat Completions API.
36+
required: true
37+
anthropic-api-key:
38+
description: 'Anthropic API key. Required when provider=anthropic.'
39+
required: false
40+
openai-api-key:
41+
description: 'OpenAI API key. Required when provider=openai.'
42+
required: false
43+
44+
outputs:
45+
result:
46+
description: |
47+
Schema-conforming JSON string. Parse with `fromJSON(...)` in
48+
downstream `if:` conditions. Empty when `conclusion` is not `success`.
49+
value: ${{ steps.run.outputs.result }}
50+
conclusion:
51+
description: '`success` when the AI step ran and returned JSON; `skipped` when the resolver vetoed the input; `failed` when the provider errored or returned non-JSON.'
52+
value: ${{ steps.conclusion.outputs.conclusion }}
53+
reason:
54+
description: 'One-line explanation when conclusion=skipped.'
55+
value: ${{ steps.cfg.outputs.reason }}
56+
57+
runs:
58+
using: composite
59+
steps:
60+
- name: Resolve config
61+
id: cfg
62+
shell: bash
63+
env:
64+
INPUT_PROVIDER: ${{ inputs.provider }}
65+
INPUT_EFFORT: ${{ inputs.effort }}
66+
INPUT_OUTPUT_SCHEMA: ${{ inputs.output-schema }}
67+
run: ${{ github.action_path }}/src/resolve-config.sh
68+
69+
- name: Install provider SDK
70+
if: steps.cfg.outputs.proceed == 'true'
71+
shell: bash
72+
env:
73+
PROVIDER: ${{ inputs.provider }}
74+
run: |
75+
set -euo pipefail
76+
case "$PROVIDER" in
77+
anthropic) python3 -m pip install --quiet --disable-pip-version-check anthropic ;;
78+
openai) python3 -m pip install --quiet --disable-pip-version-check openai ;;
79+
*) echo "::error::unknown provider '$PROVIDER'"; exit 1 ;;
80+
esac
81+
82+
- name: Call provider
83+
id: run
84+
if: steps.cfg.outputs.proceed == 'true'
85+
shell: bash
86+
env:
87+
INPUT_PROVIDER: ${{ inputs.provider }}
88+
INPUT_MODEL: ${{ steps.cfg.outputs.model }}
89+
INPUT_PROMPT: ${{ inputs.prompt }}
90+
INPUT_INPUT: ${{ inputs.input }}
91+
INPUT_OUTPUT_SCHEMA: ${{ inputs.output-schema }}
92+
INPUT_ANTHROPIC_API_KEY: ${{ inputs.anthropic-api-key }}
93+
INPUT_OPENAI_API_KEY: ${{ inputs.openai-api-key }}
94+
run: python3 ${{ github.action_path }}/src/run.py
95+
96+
- name: Emit conclusion
97+
id: conclusion
98+
if: always()
99+
shell: bash
100+
env:
101+
PROCEED: ${{ steps.cfg.outputs.proceed }}
102+
RUN_CONCLUSION: ${{ steps.run.outputs.conclusion }}
103+
run: |
104+
if [ "$PROCEED" != "true" ]; then
105+
echo "conclusion=skipped" >> "$GITHUB_OUTPUT"
106+
elif [ -n "$RUN_CONCLUSION" ]; then
107+
echo "conclusion=$RUN_CONCLUSION" >> "$GITHUB_OUTPUT"
108+
else
109+
echo "conclusion=failed" >> "$GITHUB_OUTPUT"
110+
fi

‎.github/actions/ai-step/src/resolve-config.sh‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#!/usr/bin/env bash
2+
# Validate caller inputs and resolve them into the concrete values the
3+
# downstream AI step needs: the provider-specific model id. All
4+
# conditional logic for the whole action lives here — YAML only
5+
# dispatches on outputs.
6+
#
7+
# Required env: INPUT_PROVIDER, INPUT_EFFORT, INPUT_OUTPUT_SCHEMA
8+
# Writes to $GITHUB_OUTPUT:
9+
# proceed=true|false — whether the AI step should run
10+
# reason=<string> — one-line explanation (populated on skip)
11+
# model=<string> — provider-specific model identifier
12+
# Always exits 0 — invalid input degrades to a skip, never hard-fails.
13+
set -euo pipefail
14+
15+
: "${INPUT_PROVIDER:?INPUT_PROVIDER required}"
16+
: "${INPUT_EFFORT:?INPUT_EFFORT required}"
17+
: "${INPUT_OUTPUT_SCHEMA?INPUT_OUTPUT_SCHEMA required}"
18+
19+
emit() {
20+
local k="$1" v="$2"
21+
[ -n "${GITHUB_OUTPUT:-}" ] && printf '%s=%s\n' "$k" "$v" >> "$GITHUB_OUTPUT"
22+
printf '%s=%s\n' "$k" "$v"
23+
}
24+
25+
skip() {
26+
local reason="$1"
27+
echo "::notice::ai-step: $reason"
28+
emit proceed false
29+
emit reason "$reason"
30+
emit model ""
31+
exit 0
32+
}
33+
34+
# schema is the contract of the action — empty schema defeats the point
35+
if [ -z "${INPUT_OUTPUT_SCHEMA// }" ]; then
36+
skip "output-schema is required — structured output is the contract"
37+
fi
38+
39+
# provider + effort → model
40+
case "$INPUT_PROVIDER:$INPUT_EFFORT" in
41+
anthropic:low) model='claude-haiku-4-5' ;;
42+
anthropic:medium) model='claude-sonnet-4-6' ;;
43+
anthropic:high) model='claude-opus-4-7' ;;
44+
anthropic:*) skip "invalid effort '$INPUT_EFFORT' — valid: low, medium, high" ;;
45+
openai:low) model='gpt-5.4-mini' ;;
46+
openai:medium) model='gpt-5.3-codex' ;;
47+
openai:high) model='gpt-5.4' ;;
48+
openai:*) skip "invalid effort '$INPUT_EFFORT' — valid: low, medium, high" ;;
49+
*) skip "invalid provider '$INPUT_PROVIDER' — valid: anthropic, openai" ;;
50+
esac
51+
52+
emit proceed true
53+
emit reason ""
54+
emit model "$model"

0 commit comments

Comments
 (0)