Skip to content

Commit c49224f

Browse files
KyleKincerKyleKincer
committed
add AGENTS.md
1 parent cf1f36b commit c49224f

File tree

1 file changed

+312
-0
lines changed

1 file changed

+312
-0
lines changed

‎AGENTS.md‎

Lines changed: 312 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,312 @@
1+
# 4D Unit Testing Framework
2+
3+
A comprehensive unit testing framework for the 4D 4GL platform with test tagging, filtering, and CI/CD integration.
4+
5+
## Project Overview
6+
7+
This project provides a complete testing framework for 4D applications featuring:
8+
9+
- **Auto test discovery** - Finds test classes ending with "Test"
10+
- **Comment-based tagging** - Organize tests with `// #tags: unit, integration, slow`
11+
- **Flexible filtering** - Run specific test subsets by name, pattern, or tags
12+
- **Multiple output formats** - Human-readable and JSON output with terse/verbose modes
13+
- **CI/CD ready** - Structured JSON output for automated testing pipelines
14+
- **Parallel test execution** - Run test suites concurrently for improved performance
15+
- **Automatic transaction management** - Test isolation with automatic rollback
16+
- **Manual transaction control** - Full transaction lifecycle management for advanced scenarios
17+
18+
## Running Tests
19+
20+
### Quick Start with Makefile
21+
22+
```bash
23+
# Run all tests
24+
make test
25+
26+
# Pass parameters directly to test command
27+
make test format=json
28+
make test tags=unit
29+
make test format=json tags=unit excludeTags=slow
30+
make test test=ExampleTest
31+
32+
# Alternative named commands
33+
make test-json # Run all tests with JSON output
34+
make test-class CLASS=ExampleTest
35+
make test-tags TAGS=unit
36+
make test-tags TAGS=integration,performance
37+
make test-exclude-tags TAGS=slow
38+
make test-require-tags TAGS=unit,fast
39+
40+
# Convenience shortcuts
41+
make test-unit # Run only unit tests
42+
make test-integration # Run only integration tests
43+
make test-unit-json # Run unit tests with JSON output
44+
45+
# JUnit XML output for CI/CD integration
46+
make test-junit # Run all tests with JUnit XML output
47+
make test-ci # Run tests for CI/CD (saves to test-results/junit.xml)
48+
make test-unit-junit # Run unit tests with JUnit XML output
49+
make test-integration-junit # Run integration tests with JUnit XML output
50+
51+
# Parallel execution
52+
make test-parallel # Run all tests in parallel
53+
make test-parallel-json # Run tests in parallel with JSON output
54+
make test-parallel-unit # Run unit tests in parallel
55+
make test-parallel-workers WORKERS=4 # Run with custom worker count
56+
57+
# Show all available commands
58+
make help
59+
```
60+
61+
### Manual Test Execution (Advanced)
62+
63+
If you need more control or the Makefile doesn't meet your needs:
64+
65+
```bash
66+
# Run all tests with human output
67+
/Applications/tool4d.app/Contents/MacOS/tool4d --project $(PWD)/testing/Project/testing.4DProject --skip-onstartup --dataless --startup-method "test"
68+
69+
# Run all tests with JSON output
70+
/Applications/tool4d.app/Contents/MacOS/tool4d --project $(PWD)/testing/Project/testing.4DProject --skip-onstartup --dataless --startup-method "test" --user-param "format=json"
71+
72+
# Run all tests with JUnit XML output (saves to test-results/junit.xml)
73+
/Applications/tool4d.app/Contents/MacOS/tool4d --project $(PWD)/testing/Project/testing.4DProject --skip-onstartup --dataless --startup-method "test" --user-param "format=junit"
74+
```
75+
76+
### Test Filtering Parameters
77+
78+
```bash
79+
# Run specific test class
80+
--user-param "test=ExampleTest"
81+
82+
# Run tests by tags
83+
--user-param "tags=unit"
84+
--user-param "tags=integration,performance"
85+
--user-param "excludeTags=slow"
86+
--user-param "requireTags=unit,fast"
87+
88+
# Combined filtering
89+
--user-param "tags=unit excludeTags=slow"
90+
--user-param "format=json tags=integration"
91+
--user-param "format=junit tags=unit"
92+
--user-param "format=junit outputPath=results/junit.xml"
93+
94+
# Parallel execution
95+
--user-param "parallel=true"
96+
--user-param "parallel=true maxWorkers=4"
97+
--user-param "parallel=true format=json tags=unit"
98+
```
99+
100+
### Current Test Status
101+
102+
- **Total Tests**: 121 tests across 14 test suites
103+
- **Pass Rate**: 100%
104+
- **Key Test Classes**: TaggingSystemTest, TaggingExampleTest, TestRunnerTest, TestSuiteTest
105+
106+
## Project Structure
107+
108+
```
109+
testing/Project/Sources/Classes/
110+
├── TestRunner.4dm # Main test orchestration
111+
├── TestSuite.4dm # Individual test class management
112+
├── TestFunction.4dm # Single test execution & tagging
113+
├── Assert.4dm # Assertion library
114+
├── Testing.4dm # Test context
115+
├── TaggingExampleTest.4dm # Tagging examples
116+
├── TaggingSystemTest.4dm # Tagging functionality tests
117+
└── [other test classes]
118+
```
119+
120+
The framework automatically discovers and runs any class ending with "Test" that contains methods starting with "test_".
121+
122+
## JUnit XML Output for CI/CD Integration
123+
124+
The framework supports JUnit XML output format for integration with GitLab CI/CD and other continuous integration systems.
125+
126+
### JUnit XML Features
127+
128+
- **GitLab Integration**: Test results appear in merge requests and pipeline views
129+
- **File Artifacts**: XML files can be archived as CI artifacts for historical tracking
130+
- **Test Navigation**: Direct links to failing test files in GitLab UI
131+
- **Standard Format**: Compatible with Jenkins, CircleCI, and other CI tools
132+
- **Detailed Reporting**: Includes test timing, failure messages, and stack traces
133+
134+
### Basic Usage
135+
136+
```bash
137+
# Generate JUnit XML (saves to test-results/junit.xml)
138+
make test-junit
139+
140+
# Custom output location
141+
make test format=junit outputPath=custom/path/results.xml
142+
143+
# Combined with filtering
144+
make test-unit-junit # Unit tests only
145+
make test format=junit tags=integration # Integration tests only
146+
```
147+
148+
### GitLab CI Integration
149+
150+
Add this to your `.gitlab-ci.yml`:
151+
152+
```yaml
153+
test:
154+
script:
155+
- make test-ci # Generates test-results/junit.xml
156+
artifacts:
157+
reports:
158+
junit: test-results/junit.xml
159+
paths:
160+
- test-results/
161+
when: always
162+
expire_in: 30 days
163+
coverage: '/Lines:\s*(\d+\.?\d*)%/'
164+
```
165+
166+
### JUnit XML Structure
167+
168+
The generated XML includes:
169+
170+
- **Test Suites**: One per test class
171+
- **Test Cases**: Individual test methods with timing
172+
- **Failures**: Detailed failure messages with location info
173+
- **File References**: Links to source test files
174+
- **Timestamps**: Test execution timing for performance tracking
175+
176+
Example output structure:
177+
```xml
178+
<?xml version="1.0" encoding="UTF-8"?>
179+
<testsuites name="4D Test Results" tests="121" failures="0" errors="0" time="1.234">
180+
<testsuite name="_ExampleTest" tests="5" failures="0" errors="0" time="0.156">
181+
<testcase classname="_ExampleTest" name="test_areEqual_pass"
182+
file="testing/Project/Sources/Classes/_ExampleTest.4dm" time="0.023"/>
183+
</testsuite>
184+
</testsuites>
185+
```
186+
187+
## Parallel Test Execution
188+
189+
The framework supports parallel execution of test suites to significantly reduce total test runtime while maintaining test isolation.
190+
191+
### Enabling Parallel Execution
192+
193+
```bash
194+
# Enable parallel execution (uses CPU core count as default worker count)
195+
make test-parallel
196+
197+
# Enable parallel execution with custom worker count
198+
make test-parallel-workers WORKERS=4
199+
200+
# Combine parallel execution with other options
201+
make test parallel=true format=json tags=unit maxWorkers=6
202+
```
203+
204+
### How Parallel Execution Works
205+
206+
1. **Suite-Level Parallelism**: Test suites run concurrently, but individual tests within a suite run sequentially
207+
2. **Worker Pool**: Creates worker processes up to the specified maximum (default: CPU core count, max: 8)
208+
3. **Automatic Load Balancing**: Distributes test suites across available workers
209+
4. **Test Isolation**: Each worker runs in its own process with separate transaction scope
210+
5. **Result Aggregation**: Collects and merges results from all workers before generating final report
211+
212+
### Parallel Execution Opt-Out
213+
214+
Test suites can opt out of parallel execution using comment annotations:
215+
216+
```4d
217+
// Test class that requires sequential execution
218+
// #parallel: false
219+
220+
Class constructor()
221+
222+
Function test_database_exclusive_operation($t : cs:Testing)
223+
// This test requires exclusive database access
224+
// and will run sequentially even in parallel mode
225+
```
226+
227+
### Performance Benefits
228+
229+
- **30-60% reduction** in total test runtime for typical test suites
230+
- **Better resource utilization** on multi-core machines
231+
- **Improved developer experience** with faster feedback loops
232+
- **CI/CD optimization** reducing pipeline duration
233+
234+
### Best Practices for Parallel Execution
235+
236+
1. **Design for Independence**: Ensure test suites don't depend on each other's state
237+
2. **Use Transactions**: Leverage automatic transaction management for database isolation
238+
3. **Opt-Out When Needed**: Use `// #parallel: false` for tests requiring exclusive resources
239+
4. **Monitor Performance**: Compare sequential vs parallel execution times
240+
5. **Tune Worker Count**: Adjust `maxWorkers` based on your hardware and test characteristics
241+
242+
## Transaction Management
243+
244+
The framework provides automatic transaction management for test isolation and manual transaction control for advanced scenarios.
245+
246+
### Automatic Transaction Management
247+
248+
By default, each test runs in its own transaction that is automatically rolled back after completion, ensuring:
249+
- **Test Isolation**: Tests cannot interfere with each other's data
250+
- **Clean Environment**: Database state is restored after each test
251+
- **No Side Effects**: Failed tests don't leave partial data
252+
253+
### Controlling Transaction Behavior
254+
255+
Use comment-based annotations to control transaction behavior:
256+
257+
```4d
258+
Function test_withTransactions($t : cs:C1710.Testing)
259+
// Automatic transactions enabled (default)
260+
// Test data changes will be rolled back
261+
262+
Function test_withoutTransactions($t : cs:C1710.Testing)
263+
// #transaction: false
264+
// Disables automatic transaction management
265+
```
266+
267+
### Manual Transaction Control
268+
269+
The Testing context provides methods for manual transaction management:
270+
271+
```4d
272+
Function test_manualTransactions($t : cs:C1710.Testing)
273+
// #transaction: false
274+
275+
// Start transaction manually
276+
$t.startTransaction()
277+
278+
// Check transaction status
279+
If ($t.inTransaction())
280+
// Perform database operations
281+
End if
282+
283+
// Validate or cancel transaction
284+
If ($success)
285+
$t.validateTransaction()
286+
Else
287+
$t.cancelTransaction()
288+
End if
289+
290+
Function test_transactionWrapper($t : cs:C1710.Testing)
291+
// #transaction: false
292+
293+
// Execute operation within transaction (auto-rollback)
294+
var $success : Boolean
295+
$success:=$t.withTransaction(Formula(
296+
// Database operations here
297+
// Will be rolled back automatically
298+
))
299+
300+
// Execute operation with validation (persists data)
301+
$success:=$t.withTransactionValidate(Formula(
302+
// Database operations here
303+
// Will be validated if test succeeds
304+
))
305+
```
306+
307+
### Transaction Control Comments
308+
309+
| Comment | Effect |
310+
| ------------------------ | ---------------------------------------- |
311+
| `// #transaction: false` | Disables automatic transactions |
312+
| No comment | Enables automatic transactions (default) |

0 commit comments

Comments
 (0)