You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -98,40 +96,6 @@ This will create a job summary with insights on failures, flakiness and test dur
98
96
<img src="images/insights.png" alt="GitHub Test Reporter" width="600">
99
97
</div>
100
98
101
-
## Continuous AI Usage
102
-
103
-
To get started with continuous AI analysis of test results, add the following to your workflow file:
104
-
105
-
```yaml
106
-
- name: Publish Test Report with AI Analysis
107
-
uses: ctrf-io/github-test-reporter@v1
108
-
with:
109
-
report-path: './ctrf/*.json'
110
-
ai-summary-report: true
111
-
on-fail-only: true
112
-
summary: false
113
-
pull-request: true
114
-
ai: |
115
-
{
116
-
"provider": "openai",
117
-
"model": "gpt-5"
118
-
}
119
-
if: always()
120
-
env:
121
-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
122
-
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
123
-
```
124
-
125
-
Add your AI API key to the repository secrets. Choose from a variety of AI providers and models. See [AI Configuration](#ai-configuration) for more information.
126
-
127
-
Make sure your GitHub Token has pull request write permission.
128
-
129
-
This will create a pull request comment with a summary of the AI analysis.
130
-
131
-
<div align="center">
132
-
<img src="images/ai-summary.png" alt="GitHub Test Reporter" width="600">
133
-
</div>
134
-
135
99
## Pull Request Comment Usage
136
100
137
101
To get started with pull request comments, add the following to your workflow file:
@@ -248,82 +212,12 @@ For more advanced usage, there are several inputs available.
248
212
group-by: 'filePath' # Specify grouping for applicable reports (e.g., suite or file path). Default is filePath
249
213
always-group-by: false # Force grouping by suite or file path for all reports. Default is false
250
214
report-order: 'summary-report,failed-report,flaky-report,skipped-report,test-report' # Comma-separated list of report types to specify the order in which reports should be displayed
251
-
ai: '{}' # JSON configuration for AI-powered test analysis. See AI Configuration section below
252
215
integrations-config: '{}' # JSON configuration for integrations with other developer tools
253
216
if: always()
254
217
```
255
218
256
219
Only `report-path` is required.
257
220
258
-
## Generating an AI Report
259
-
260
-
You can generate human-readable AI reports for your failed tests using models
261
-
from the leading AI providers. The GitHub Test Reporter now features dedicated
262
-
AI-first configuration for seamless integration with continuous AI workflows.
263
-
264
-
### AI Configuration
265
-
266
-
Use the `ai` input to configure AI-powered test analysis. Simply provide a JSON object
267
-
with the provider and any optional settings:
268
-
269
-
```yaml
270
-
- name: Publish Test Report with AI Analysis
271
-
uses: ctrf-io/github-test-reporter@v1
272
-
with:
273
-
report-path: './ctrf/*.json'
274
-
ai-summary-report: true
275
-
pull-request: true
276
-
ai: |
277
-
{
278
-
"provider": "openai",
279
-
"model": "gpt-4"
280
-
}
281
-
env:
282
-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
283
-
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
284
-
if: always()
285
-
```
286
-
287
-
### Supported AI Providers
288
-
289
-
The following AI providers are supported:
290
-
291
-
- **openai** - OpenAI (GPT-4, GPT-3.5, etc.)
292
-
- **claude** - Anthropic Claude
293
-
- **gemini** - Google Gemini
294
-
- **azure-openai** - Azure OpenAI Service
295
-
- **grok** - xAI Grok
296
-
- **deepseek** - DeepSeek
297
-
- **mistral** - Mistral AI
298
-
- **perplexity** - Perplexity AI
299
-
- **openrouter** - OpenRouter (access to multiple models)
You can add a pull request comment by using the `pull-request-report` input:
@@ -551,11 +445,17 @@ CTRF tooling offers seamless developer tool integration, allowing you to combine
551
445
|------------|-------------|------------|
552
446
| Slack Test Reporter | Send test results and notifications to Slack channels | [ctrf-io/slack-test-reporter](https://github.com/ctrf-io/slack-test-reporter) |
553
447
| Microsoft Teams Test Reporter | Post test results and alerts to Teams channels | [ctrf-io/teams-test-reporter](https://github.com/ctrf-io/teams-test-reporter) |
448
+
| AI Test Reporter | Intelligent test analysis using leading AI models | [ctrf-io/ai-test-reporter](https://github.com/ctrf-io/ai-test-reporter) |
554
449
555
450
For detailed information about configuring and using these integrations, see our [Integrations Documentation](docs/integrations.md).
556
451
557
452
Integrations are currently in beta. Please report any issues to the [GitHub Test Reporter repository](https://github.com/ctrf-io/github-test-reporter/issues).
558
453
454
+
## Generating an AI Report
455
+
456
+
You can generate human-readable AI report for your failed tests using models
457
+
from the leading AI providers by using the AI Test Reporter integration or the [AI Test Reporter](https://github.com/ctrf-io/ai-test-reporter) directly.
458
+
559
459
## Further Processing
560
460
561
461
You can further process the CTRF report by using the output `report` from the action or by using the `write-ctrf-to-file` input. For reports larger than 1MB, consider using the `write-ctrf-to-file` input.
@@ -589,4 +489,4 @@ analyzing test outcomes across multiple platforms becomes more straightforward.
589
489
## Support Us
590
490
591
491
If you find this project useful, consider giving it a GitHub star ⭐ It means a
@@ -145,25 +143,164 @@ Example accessing test data:
145
143
146
144
## GitHub Properties
147
145
148
-
GitHub properties are made available to use in your template. You can access
149
-
these properties by using the `github` property, for example `github.repoName`
146
+
GitHub properties are made available to use in your template. You can access these properties by using the `github` property, for example `github.repoName` or `github.actor`.
147
+
148
+
### Root Context Properties
149
+
150
+
Access workflow and execution information:
151
+
152
+
**Workflow Information:**
153
+
154
+
-`github.workflow` - Name of the workflow
155
+
-`github.action` / `github.action_name` - Current action name
156
+
-`github.job` / `github.job_id` / `github.jobName` - Current job identifier
157
+
-`github.runNumber` / `github.run_number` - Unique number for each workflow run
158
+
-`github.runId` / `github.run_id` / `github.workflowId` - Unique identifier for the workflow run
159
+
-`github.workflowName` - Workflow name (legacy property)
160
+
161
+
**Event Information:**
162
+
163
+
-`github.eventName` / `github.event_name` - Event that triggered the workflow (e.g., "push", "pull_request")
164
+
-`github.actor` / `github.actor_name` / `github.actorName` - User who triggered the workflow
165
+
-`github.sha` - Commit SHA that triggered the workflow
166
+
-`github.ref` - Git ref that triggered the workflow (e.g., "refs/heads/main")
167
+
-`github.branchName` - Current branch name
168
+
169
+
**Repository & URLs:**
170
+
171
+
-`github.repoName` - Repository name
172
+
-`github.buildUrl` / `github.build_url` - URL to the workflow run summary
Access information about the user who triggered the workflow:
272
+
273
+
-`github.sender.login` - Username
274
+
-`github.sender.id` - User ID
275
+
-`github.sender.nodeId` / `github.sender.node_id` - Node ID
276
+
-`github.sender.type` - User type (e.g., "User", "Bot")
277
+
-`github.sender.siteAdmin` / `github.sender.site_admin` - Whether user is a site admin (boolean)
278
+
-`github.sender.htmlUrl` / `github.sender.html_url` - User profile URL
279
+
-`github.sender.avatarUrl` / `github.sender.avatar_url` - User avatar URL
280
+
-`github.sender.gravatarId` / `github.sender.gravatar_id` - Gravatar ID
281
+
282
+
### Full Context Access
283
+
284
+
You can access the complete webhook event payload via `github.context`. This contains the raw event payload data sent by GitHub when the workflow was triggered (e.g., the full `pull_request` object, `repository` object, `issue` object, etc. depending on the event type).
285
+
286
+
This is useful for accessing event-specific data that isn't exposed in the structured properties above. For example:
287
+
288
+
-`github.context.pull_request.head.ref` - Head branch name
289
+
-`github.context.pull_request.base.ref` - Base branch name
290
+
-`github.context.issue` - Issue data (for issue events)
291
+
-`github.context.comment` - Comment data (for comment events)
292
+
293
+
See the [GitHub Actions events documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows) for available event types.
294
+
295
+
**Note:** Contexts, objects, and properties will vary significantly under different workflow run conditions. For example, pull request properties are only available when the workflow is triggered by a pull request event.
296
+
297
+
### Debugging Available Properties
298
+
299
+
To see all available properties for your specific workflow, print the context in your workflow logs:
0 commit comments