Skip to content

Commit 76c011c

Browse files
danielgerlagdanielgerlag
authored
Merge pull request #1400 from danielgerlag/copilot/fix-0d05f027-b580-44dc-9256-023003cb76e5
Add detailed individual test result reporting to GitHub Actions workflows
2 parents 5f934ab + c764721 commit 76c011c

File tree

3 files changed

+247
-9
lines changed

3 files changed

+247
-9
lines changed

‎.github/workflows/dotnet.yml‎

Lines changed: 135 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,21 @@ jobs:
2323
- name: Build
2424
run: dotnet build --no-restore
2525
- name: Unit Tests
26-
run: dotnet test test/WorkflowCore.UnitTests --no-build --verbosity normal -p:ParallelizeTestCollections=false
26+
run: dotnet test test/WorkflowCore.UnitTests --no-build --verbosity detailed --logger "trx;LogFileName=UnitTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
27+
- name: Publish Test Results
28+
uses: dorny/test-reporter@v1
29+
if: success() || failure()
30+
with:
31+
name: Unit Test Results
32+
path: test-results/*.trx
33+
reporter: dotnet-trx
34+
fail-on-error: false
35+
- name: Upload Test Results
36+
uses: actions/upload-artifact@v4
37+
if: always()
38+
with:
39+
name: unit-test-results
40+
path: test-results/
2741
Integration-Tests:
2842
runs-on: ubuntu-latest
2943
steps:
@@ -40,7 +54,21 @@ jobs:
4054
- name: Build
4155
run: dotnet build --no-restore
4256
- name: Integration Tests
43-
run: dotnet test test/WorkflowCore.IntegrationTests --no-build --verbosity normal -p:ParallelizeTestCollections=false
57+
run: dotnet test test/WorkflowCore.IntegrationTests --no-build --verbosity detailed --logger "trx;LogFileName=IntegrationTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
58+
- name: Publish Test Results
59+
uses: dorny/test-reporter@v1
60+
if: success() || failure()
61+
with:
62+
name: Integration Test Results
63+
path: test-results/*.trx
64+
reporter: dotnet-trx
65+
fail-on-error: false
66+
- name: Upload Test Results
67+
uses: actions/upload-artifact@v4
68+
if: always()
69+
with:
70+
name: integration-test-results
71+
path: test-results/
4472
MongoDB-Tests:
4573
runs-on: ubuntu-latest
4674
steps:
@@ -57,7 +85,21 @@ jobs:
5785
- name: Build
5886
run: dotnet build --no-restore
5987
- name: MongoDB Tests
60-
run: dotnet test test/WorkflowCore.Tests.MongoDB --no-build --verbosity normal -p:ParallelizeTestCollections=false
88+
run: dotnet test test/WorkflowCore.Tests.MongoDB --no-build --verbosity detailed --logger "trx;LogFileName=MongoDBTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
89+
- name: Publish Test Results
90+
uses: dorny/test-reporter@v1
91+
if: success() || failure()
92+
with:
93+
name: MongoDB Test Results
94+
path: test-results/*.trx
95+
reporter: dotnet-trx
96+
fail-on-error: false
97+
- name: Upload Test Results
98+
uses: actions/upload-artifact@v4
99+
if: always()
100+
with:
101+
name: mongodb-test-results
102+
path: test-results/
61103
MySQL-Tests:
62104
runs-on: ubuntu-latest
63105
steps:
@@ -74,7 +116,21 @@ jobs:
74116
- name: Build
75117
run: dotnet build --no-restore
76118
- name: MySQL Tests
77-
run: dotnet test test/WorkflowCore.Tests.MySQL --no-build --verbosity normal -p:ParallelizeTestCollections=false
119+
run: dotnet test test/WorkflowCore.Tests.MySQL --no-build --verbosity detailed --logger "trx;LogFileName=MySQLTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
120+
- name: Publish Test Results
121+
uses: dorny/test-reporter@v1
122+
if: success() || failure()
123+
with:
124+
name: MySQL Test Results
125+
path: test-results/*.trx
126+
reporter: dotnet-trx
127+
fail-on-error: false
128+
- name: Upload Test Results
129+
uses: actions/upload-artifact@v4
130+
if: always()
131+
with:
132+
name: mysql-test-results
133+
path: test-results/
78134
PostgreSQL-Tests:
79135
runs-on: ubuntu-latest
80136
steps:
@@ -91,7 +147,21 @@ jobs:
91147
- name: Build
92148
run: dotnet build --no-restore
93149
- name: PostgreSQL Tests
94-
run: dotnet test test/WorkflowCore.Tests.PostgreSQL --no-build --verbosity normal -p:ParallelizeTestCollections=false
150+
run: dotnet test test/WorkflowCore.Tests.PostgreSQL --no-build --verbosity detailed --logger "trx;LogFileName=PostgreSQLTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
151+
- name: Publish Test Results
152+
uses: dorny/test-reporter@v1
153+
if: success() || failure()
154+
with:
155+
name: PostgreSQL Test Results
156+
path: test-results/*.trx
157+
reporter: dotnet-trx
158+
fail-on-error: false
159+
- name: Upload Test Results
160+
uses: actions/upload-artifact@v4
161+
if: always()
162+
with:
163+
name: postgresql-test-results
164+
path: test-results/
95165
Redis-Tests:
96166
runs-on: ubuntu-latest
97167
steps:
@@ -108,7 +178,21 @@ jobs:
108178
- name: Build
109179
run: dotnet build --no-restore
110180
- name: Redis Tests
111-
run: dotnet test test/WorkflowCore.Tests.Redis --no-build --verbosity normal -p:ParallelizeTestCollections=false
181+
run: dotnet test test/WorkflowCore.Tests.Redis --no-build --verbosity detailed --logger "trx;LogFileName=RedisTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
182+
- name: Publish Test Results
183+
uses: dorny/test-reporter@v1
184+
if: success() || failure()
185+
with:
186+
name: Redis Test Results
187+
path: test-results/*.trx
188+
reporter: dotnet-trx
189+
fail-on-error: false
190+
- name: Upload Test Results
191+
uses: actions/upload-artifact@v4
192+
if: always()
193+
with:
194+
name: redis-test-results
195+
path: test-results/
112196
SQLServer-Tests:
113197
runs-on: ubuntu-latest
114198
steps:
@@ -125,7 +209,21 @@ jobs:
125209
- name: Build
126210
run: dotnet build --no-restore
127211
- name: SQL Server Tests
128-
run: dotnet test test/WorkflowCore.Tests.SqlServer --no-build --verbosity normal -p:ParallelizeTestCollections=false
212+
run: dotnet test test/WorkflowCore.Tests.SqlServer --no-build --verbosity detailed --logger "trx;LogFileName=SQLServerTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
213+
- name: Publish Test Results
214+
uses: dorny/test-reporter@v1
215+
if: success() || failure()
216+
with:
217+
name: SQL Server Test Results
218+
path: test-results/*.trx
219+
reporter: dotnet-trx
220+
fail-on-error: false
221+
- name: Upload Test Results
222+
uses: actions/upload-artifact@v4
223+
if: always()
224+
with:
225+
name: sqlserver-test-results
226+
path: test-results/
129227
Elasticsearch-Tests:
130228
runs-on: ubuntu-latest
131229
steps:
@@ -142,7 +240,21 @@ jobs:
142240
- name: Build
143241
run: dotnet build --no-restore
144242
- name: Elasticsearch Tests
145-
run: dotnet test test/WorkflowCore.Tests.Elasticsearch --no-build --verbosity normal -p:ParallelizeTestCollections=false
243+
run: dotnet test test/WorkflowCore.Tests.Elasticsearch --no-build --verbosity detailed --logger "trx;LogFileName=ElasticsearchTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
244+
- name: Publish Test Results
245+
uses: dorny/test-reporter@v1
246+
if: success() || failure()
247+
with:
248+
name: Elasticsearch Test Results
249+
path: test-results/*.trx
250+
reporter: dotnet-trx
251+
fail-on-error: false
252+
- name: Upload Test Results
253+
uses: actions/upload-artifact@v4
254+
if: always()
255+
with:
256+
name: elasticsearch-test-results
257+
path: test-results/
146258
Oracle-Tests:
147259
runs-on: ubuntu-latest
148260
steps:
@@ -159,4 +271,18 @@ jobs:
159271
- name: Build
160272
run: dotnet build --no-restore
161273
- name: Oracle Tests
162-
run: dotnet test test/WorkflowCore.Tests.Oracle --no-build --verbosity normal -p:ParallelizeTestCollections=false
274+
run: dotnet test test/WorkflowCore.Tests.Oracle --no-build --verbosity detailed --logger "trx;LogFileName=OracleTests.trx" --logger "console;verbosity=detailed" --results-directory ./test-results -p:ParallelizeTestCollections=false
275+
- name: Publish Test Results
276+
uses: dorny/test-reporter@v1
277+
if: success() || failure()
278+
with:
279+
name: Oracle Test Results
280+
path: test-results/*.trx
281+
reporter: dotnet-trx
282+
fail-on-error: false
283+
- name: Upload Test Results
284+
uses: actions/upload-artifact@v4
285+
if: always()
286+
with:
287+
name: oracle-test-results
288+
path: test-results/

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,3 +247,4 @@ ModelManifest.xml
247247
.idea/.idea.WorkflowCore/.idea
248248
riderModule.iml
249249
.DS_Store
250+
test-results/

‎docs/enhanced-test-reporting.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Enhanced Test Reporting for GitHub Actions
2+
3+
This document explains the enhanced test reporting capabilities that have been added to the GitHub Actions workflow.
4+
5+
## Overview
6+
7+
The GitHub Actions workflow has been enhanced to provide detailed, individual test results for all test suites in the Workflow Core project. This addresses the requirement to see detailed, individual test results from tests run by GitHub workflows.
8+
9+
## Key Enhancements
10+
11+
### 1. Detailed Test Output
12+
- **Enhanced Verbosity**: Changed from `--verbosity normal` to `--verbosity detailed`
13+
- **Detailed Console Logging**: Added `--logger "console;verbosity=detailed"` for comprehensive console output
14+
- **Individual Test Results**: Each test case now shows its execution status, duration, and any error details
15+
16+
### 2. TRX Test Result Files
17+
- **TRX Format**: Added `--logger "trx;LogFileName={TestSuite}.trx"` to generate XML test result files
18+
- **Structured Data**: TRX files contain structured test data including:
19+
- Test names and fully qualified names
20+
- Test outcomes (Passed, Failed, Skipped)
21+
- Execution times and durations
22+
- Error messages and stack traces for failed tests
23+
- Test categories and traits
24+
25+
### 3. GitHub Actions Test Reporting
26+
- **Test Reporter Integration**: Added `dorny/test-reporter@v1` action to display test results in the GitHub UI
27+
- **PR Integration**: Test results are automatically displayed in pull request checks
28+
- **Visual Test Summary**: Failed tests are highlighted with detailed error information
29+
- **Test Status Annotations**: Test results appear as GitHub Actions annotations
30+
31+
### 4. Test Result Artifacts
32+
- **Downloadable Results**: Test result files are uploaded as artifacts for each job
33+
- **Persistent Storage**: Test results are available for download even after workflow completion
34+
- **Individual Job Results**: Each test suite (Unit, Integration, MongoDB, etc.) has separate artifacts
35+
36+
## What You'll See
37+
38+
### In GitHub Actions Logs
39+
Before (old format):
40+
```
41+
Starting test execution, please wait...
42+
A total of 1 test files matched the specified pattern.
43+
Passed! - Failed: 0, Passed: 25, Skipped: 0, Total: 25
44+
```
45+
46+
After (enhanced format):
47+
```
48+
Starting test execution, please wait...
49+
A total of 1 test files matched the specified pattern.
50+
51+
Passed WorkflowCore.UnitTests.Services.ExecutionResultProcessorFixture.should_advance_workflow [< 1 ms]
52+
Passed WorkflowCore.UnitTests.Services.ExecutionResultProcessorFixture.should_branch_children [2 ms]
53+
Failed WorkflowCore.UnitTests.Services.SomeTest.example_failing_test [15 ms]
54+
Error Message:
55+
Assert.Equal() Failure
56+
Expected: True
57+
Actual: False
58+
Stack Trace:
59+
at WorkflowCore.UnitTests.Services.SomeTest.example_failing_test() in /path/to/test.cs:line 42
60+
61+
Test Run Summary:
62+
Total tests: 25
63+
Passed: 24
64+
Failed: 1
65+
Skipped: 0
66+
```
67+
68+
### In GitHub Pull Requests
69+
- ✅ **Test Status Checks**: Clear pass/fail status for each test suite
70+
- 📊 **Test Summary**: Number of passed, failed, and skipped tests
71+
- 🔍 **Detailed Failure Information**: Click-through to see specific test failures
72+
- 📁 **Downloadable Artifacts**: Access to complete test result files
73+
74+
### Available Artifacts
75+
Each test job now produces downloadable artifacts:
76+
- `unit-test-results`: Unit test TRX files and logs
77+
- `integration-test-results`: Integration test TRX files and logs
78+
- `mongodb-test-results`: MongoDB-specific test results
79+
- `mysql-test-results`: MySQL-specific test results
80+
- `postgresql-test-results`: PostgreSQL-specific test results
81+
- `redis-test-results`: Redis-specific test results
82+
- `sqlserver-test-results`: SQL Server-specific test results
83+
- `elasticsearch-test-results`: Elasticsearch-specific test results
84+
- `oracle-test-results`: Oracle-specific test results
85+
86+
## Benefits
87+
88+
1. **Individual Test Visibility**: See exactly which tests pass or fail
89+
2. **Debugging Support**: Detailed error messages and stack traces
90+
3. **Performance Monitoring**: Test execution times for performance analysis
91+
4. **Historical Data**: Downloadable test results for trend analysis
92+
5. **CI/CD Integration**: Better integration with GitHub's native test reporting features
93+
6. **Developer Experience**: Faster identification of test issues in pull requests
94+
95+
## File Structure
96+
97+
After test execution, the following files are generated:
98+
```
99+
test-results/
100+
├── UnitTests.trx
101+
├── IntegrationTests.trx
102+
├── MongoDBTests.trx
103+
├── MySQLTests.trx
104+
├── PostgreSQLTests.trx
105+
├── RedisTests.trx
106+
├── SQLServerTests.trx
107+
├── ElasticsearchTests.trx
108+
└── OracleTests.trx
109+
```
110+
111+
Each TRX file contains detailed XML data about the test execution results that can be consumed by various reporting tools and integrated development environments.

0 commit comments

Comments
 (0)