Skip to content

Commit c8567a7

Browse files
add ai input (#246)
* add ai input
1 parent 7b0efe6 commit c8567a7

36 files changed

+54081
-25613
lines changed

README.md

Lines changed: 103 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -10,34 +10,14 @@ analyses directly within your GitHub Actions CI/CD workflow and Pull Requests.
1010
Choose from a variety of pre-built reports or create custom reports tailored to
1111
your project's needs, ensuring that test results are always where you need them.
1212

13+
## Support Us
1314

14-
<div align="center">
15-
<h2>⭐ Support Us ⭐</h2>
16-
<h3>If you like what you see, drop us a star ⭐</h3>
17-
<h3>It helps more teams find the project and motivates us to keep building! 💚</h3>
18-
</div>
15+
**GitHub Test Reporter is community-built and open source.**
1916

17+
You can support the project by:
2018

21-
<div align="center">
22-
<div style="padding: 1.5rem; border-radius: 8px; margin: 1rem 0; border: 1px solid #30363d;">
23-
<span style="font-size: 23px;">💚</span>
24-
<h3 style="margin: 1rem 0;">CTRF tooling is open source and free to use</h3>
25-
26-
<div style="margin-top: 1.5rem;">
27-
<a href="https://github.com/ctrf-io/github-test-reporter">
28-
<img src="https://img.shields.io/github/stars/ctrf-io/github-test-reporter?style=for-the-badge&color=2ea043" alt="GitHub stars">
29-
</a>
30-
<a href="https://github.com/ctrf-io">
31-
<img src="https://img.shields.io/github/followers/ctrf-io?style=for-the-badge&color=2ea043" alt="GitHub followers">
32-
</a>
33-
</div>
34-
</div>
35-
36-
<p style="font-size: 14px; margin: 1rem 0;">
37-
Contributions are very welcome! <br/>
38-
Explore more <a href="https://www.ctrf.io/integrations">integrations</a>
39-
</p>
40-
</div>
19+
- Giving this repository a ⭐
20+
- [Following the @ctrf organization on GitHub](https://github.com/ctrf-io)
4121

4222
## Example
4323

@@ -53,7 +33,7 @@ Explore more <a href="https://www.ctrf.io/integrations">integrations</a>
5333

5434
**🧘 Super flexible:** start fast with powerful built-in reports or go fully custom with your own templates
5535

56-
**🤖 AI-powered analysis:** explains why tests failed and how to fix them
36+
**🤖 Continuous AI:** summarize failed tests and get explanations of them with contextual insights
5737

5838
**🔌 Framework-agnostic:** works with any testing tool
5939

@@ -63,16 +43,16 @@ Explore more <a href="https://www.ctrf.io/integrations">integrations</a>
6343
2. [Report Showcase](#report-showcase)
6444
3. [Visual Overview](#visual-overview)
6545
4. [Available Inputs](#available-inputs)
66-
5. [Pull Requests](#pull-requests)
67-
6. [Status Checks](#status-checks)
68-
7. [Build Your Own Report](#build-your-own-report)
69-
8. [Customizing Report Order](#customizing-report-order)
70-
9. [Community Reports](#community-reports)
71-
10. [GitHub Token](#github-token)
72-
11. [Storing Artifacts](#storing-artifacts)
73-
12. [Filtering](#filtering)
74-
13. [Integrations](#integrations)
75-
14. [Generating an AI Report](#generating-an-ai-report)
46+
5. [Generating an AI Report](#generating-an-ai-report)
47+
6. [Pull Requests](#pull-requests)
48+
7. [Status Checks](#status-checks)
49+
8. [Build Your Own Report](#build-your-own-report)
50+
9. [Customizing Report Order](#customizing-report-order)
51+
10. [Community Reports](#community-reports)
52+
11. [GitHub Token](#github-token)
53+
12. [Storing Artifacts](#storing-artifacts)
54+
13. [Filtering](#filtering)
55+
14. [Integrations](#integrations)
7656
15. [What is CTRF?](#what-is-ctrf)
7757

7858
## Usage
@@ -90,7 +70,7 @@ To get started add the following to your workflow file:
9070
9171
## Report Showcase
9272
93-
Checkout all the built-in reports [here](docs/report-showcase.md)
73+
Checkout the [built-in reports](docs/report-showcase.md)
9474
9575
## Visual Overview
9676
@@ -102,7 +82,7 @@ Checkout all the built-in reports [here](docs/report-showcase.md)
10282
10383
You need a CTRF or JUnit report.
10484
105-
This reporter works best with a CTRF as it's packed with modern properties. [CTRF reporters](https://ctrf.io/integrations) are available for most testing frameworks and easy to install.
85+
This reporter works best with CTRF as it's packed with modern properties. [CTRF reporters](https://ctrf.io/integrations) are available for most testing frameworks and easy to install.
10686
10787
For more details on using JUnit, see [JUnit integration instructions](docs/integrations.md#junit-to-ctrf-integration)
10888
@@ -176,12 +156,97 @@ For more advanced usage, there are several inputs available.
176156
group-by: 'filePath' # Specify grouping for applicable reports (e.g., suite or file path). Default is filePath
177157
always-group-by: false # Force grouping by suite or file path for all reports. Default is false
178158
report-order: 'summary-report,failed-report,flaky-report,skipped-report,test-report' # Comma-separated list of report types to specify the order in which reports should be displayed
159+
ai: '{}' # JSON configuration for AI-powered test analysis. See AI Configuration section below
179160
integrations-config: '{}' # JSON configuration for integrations with other developer tools
180161
if: always()
181162
```
182163
183164
Only `report-path` is required.
184165

166+
## Generating an AI Report
167+
168+
You can generate human-readable AI reports for your failed tests using models
169+
from the leading AI providers. The GitHub Test Reporter now features dedicated
170+
AI-first configuration for seamless integration with continuous AI workflows.
171+
172+
### AI Configuration
173+
174+
Use the `ai` input to configure AI-powered test analysis. Simply provide a JSON object
175+
with the provider and any optional settings:
176+
177+
```yaml
178+
- name: Publish Test Report with AI Analysis
179+
uses: ctrf-io/github-test-reporter@v1
180+
with:
181+
report-path: './ctrf/*.json'
182+
github-report: true
183+
ai: |
184+
{
185+
"provider": "openai",
186+
"model": "gpt-4",
187+
"temperature": 0.7,
188+
"maxTokens": 2000
189+
}
190+
env:
191+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
192+
if: always()
193+
```
194+
195+
### Supported AI Providers
196+
197+
The following AI providers are supported:
198+
199+
- **openai** - OpenAI (GPT-4, GPT-3.5, etc.)
200+
- **claude** - Anthropic Claude
201+
- **gemini** - Google Gemini
202+
- **azure-openai** - Azure OpenAI Service
203+
- **grok** - xAI Grok
204+
- **deepseek** - DeepSeek
205+
- **mistral** - Mistral AI
206+
- **perplexity** - Perplexity AI
207+
- **openrouter** - OpenRouter (access to multiple models)
208+
- **bedrock** - AWS Bedrock (Claude, Llama, Titan, etc.)
209+
210+
### AI Configuration Options
211+
212+
All configuration parameters are specified at the root level (all optional except `provider`):
213+
214+
```json
215+
{
216+
"provider": "openai", // Required: AI provider to use
217+
"model": "gpt-4", // AI model to use
218+
"temperature": 0.7, // Creativity (0-2)
219+
"maxTokens": 2000, // Max response length
220+
"systemPrompt": "...", // Custom system prompt
221+
"frequencyPenalty": 0, // Frequency penalty (0-2)
222+
"presencePenalty": 0, // Presence penalty (0-2)
223+
"topP": 1, // Nucleus sampling
224+
"maxMessages": 10, // Max failed tests to analyze
225+
"consolidate": true, // Consolidate multiple failures
226+
"log": false, // Enable logging
227+
"deploymentId": "..." // Azure OpenAI deployment ID (Azure only)
228+
}
229+
```
230+
231+
### Example with Claude
232+
233+
```yaml
234+
- name: Publish Test Report with Claude AI
235+
uses: ctrf-io/github-test-reporter@v1
236+
with:
237+
report-path: './ctrf/*.json'
238+
github-report: true
239+
ai: |
240+
{
241+
"provider": "claude",
242+
"model": "claude-3-5-sonnet-20241022",
243+
"maxTokens": 3000
244+
}
245+
env:
246+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
247+
if: always()
248+
```
249+
185250
## Pull Requests
186251

187252
You can add a pull request comment by using the `pull-request-report` input:
@@ -409,17 +474,12 @@ CTRF tooling offers seamless developer tool integration, allowing you to combine
409474
|------------|-------------|------------|
410475
| Slack Test Reporter | Send test results and notifications to Slack channels | [ctrf-io/slack-test-reporter](https://github.com/ctrf-io/slack-test-reporter) |
411476
| Microsoft Teams Test Reporter | Post test results and alerts to Teams channels | [ctrf-io/teams-test-reporter](https://github.com/ctrf-io/teams-test-reporter) |
412-
| AI Test Reporter | Intelligent test analysis using leading AI models | [ctrf-io/ai-test-reporter](https://github.com/ctrf-io/ai-test-reporter) |
413477

414478
For detailed information about configuring and using these integrations, see our [Integrations Documentation](docs/integrations.md).
415479

416480
Integrations are currently in beta. Please report any issues to the [GitHub Test Reporter repository](https://github.com/ctrf-io/github-test-reporter/issues).
417481

418-
## Generating an AI Report
419482

420-
You can generate human-readable AI report for your failed tests using models
421-
from the leading AI providers by using the AI Test Reporter integration or the
422-
[AI Test Reporter](https://github.com/ctrf-io/ai-test-reporter) directly.
423483

424484
## Further Processing
425485

__tests__/ctrf/helpers.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,4 +111,4 @@ describe('stripAnsi', () => {
111111
// @ts-expect-error Testing runtime error
112112
expect(() => stripAnsi({})).toThrow(TypeError)
113113
})
114-
})
114+
})

__tests__/ctrf/report-preparation.test.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -535,6 +535,7 @@ function createSingleReportInputs(): Inputs {
535535
groupBy: 'suite',
536536
alwaysGroupBy: false,
537537
integrationsConfig: {},
538+
ai: {},
538539
statusCheck: false,
539540
statusCheckName: '',
540541
reportOrder: []

action.yml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,15 @@ inputs:
262262
default: ''
263263

264264
# Integration Configuration
265+
ai:
266+
description: >
267+
JSON configuration for AI-powered test analysis. Supports multiple AI
268+
providers including OpenAI, Claude, Gemini, Grok, Deepseek, Mistral,
269+
Perplexity, Azure OpenAI, OpenRouter, and AWS Bedrock. Example:
270+
'{"provider": "openai", "model": "gpt-4", "temperature": 0.7, "maxTokens":
271+
2000}'
272+
required: false
273+
default: '{}'
265274
integrations-config:
266275
description:
267276
'JSON configuration for integrations with other developer tools'

badges/coverage.svg

Lines changed: 1 addition & 1 deletion
Loading

0 commit comments

Comments
 (0)