Skip to content

Commit 43c36a9

Browse files
aepfligemini-code-assist[bot]toddbaert
authored
test: improve gherkin suite with spec information and testdata (#336)
Adding newer evaluation tests, in the usual style. Additionally i am adding a json file with test data, so we do not need to manually create it all the time, but could use a json parser --------- Signed-off-by: Simon Schrottner <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Todd Baert <[email protected]>
1 parent 969e11c commit 43c36a9

File tree

6 files changed

+973
-7
lines changed

6 files changed

+973
-7
lines changed

‎specification/assets/gherkin/README.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# Test Assets
2+
3+
This directory contains test assets for the OpenFeature specification, including Gherkin test scenarios and structured JSON test data for comprehensive implementation validation.
4+
5+
## Overview
6+
7+
The test assets improve the existing Gherkin test suite by providing structured JSON test data that eliminates the need for manual test data creation. The data includes various flag types with different targeting scenarios to test edge cases and standard behavior.
8+
9+
## Test Data Structure (`test-flags.json`)
10+
11+
The [JSON test data](./test-flags.json) contains flags organized into several categories based on their behavior and purpose:
12+
13+
### Standard Flags
14+
Basic feature flags with straightforward evaluation:
15+
- `boolean-flag`: Boolean flag with variants `on` (true) and `off` (false), defaults to `on`
16+
- `string-flag`: String flag with variants `greeting` ("hi") and `parting` ("bye"), defaults to `greeting`
17+
- `integer-flag`: Integer flag with variants `one` (1) and `ten` (10), defaults to `ten`
18+
- `float-flag`: Float flag with variants `tenth` (0.1) and `half` (0.5), defaults to `half`
19+
- `object-flag`: Object flag with `empty` ({}) and `template` variants, defaults to `template`
20+
21+
### Zero Value Flags
22+
Flags specifically designed to test zero/empty value handling:
23+
- `boolean-zero-flag`: Boolean flag that defaults to `zero` variant (false)
24+
- `string-zero-flag`: String flag that defaults to `zero` variant (empty string "")
25+
- `integer-zero-flag`: Integer flag that defaults to `zero` variant (0)
26+
- `float-zero-flag`: Float flag that defaults to `zero` variant (0.0)
27+
- `object-zero-flag`: Object flag that defaults to `zero` variant (empty object {})
28+
29+
### Targeted Zero Flags
30+
Flags with CEL expressions that can evaluate to zero values based on context:
31+
- `boolean-targeted-zero-flag`: Uses CEL targeting, defaults to `zero` (false)
32+
- `string-targeted-zero-flag`: Uses CEL targeting, defaults to `zero` (empty string)
33+
- `integer-targeted-zero-flag`: Uses CEL targeting, defaults to `zero` (0)
34+
- `float-targeted-zero-flag`: Uses CEL targeting, defaults to `zero` (0.0)
35+
- `object-targeted-zero-flag`: Uses CEL targeting, defaults to `zero` (empty object)
36+
37+
### Disabled Flags
38+
Flags that are statically disabled:
39+
- `boolean-disabled-flag`: Disabled Flag
40+
- `string-disabled-flag`: Disabled Flag
41+
- `integer-disabled-flag`: Disabled Flag
42+
- `float-disabled-flag`: Disabled Flag
43+
- `object-disabled-flag`: Disabled Flag
44+
45+
### Special Testing Flags
46+
Flags for testing edge cases and metadata:
47+
- `metadata-flag`: Boolean flag with rich metadata including string, integer, boolean, and float values
48+
- `complex-targeted`: String flag with complex CEL expression for internal/external user distinction
49+
- `null-default-flag`: Flag with explicitly null default variant
50+
- `undefined-default-flag`: Flag with no default variant defined
51+
- `wrong-flag`: Flag for testing error scenarios
52+
53+
## CEL Expression Variables
54+
55+
The test data uses Common Expression Language (CEL) expressions in the `contextEvaluator` field. Based on the expressions in the test data, the following context variables are expected:
56+
57+
### Available Context Variables
58+
- `email`: User's email address (string)
59+
- `customer`: Boolean flag indicating customer status
60+
- `age`: User's age (integer)
61+
62+
### CEL Expressions Used
63+
64+
#### Simple Email Targeting
65+
```cel
66+
email == '[email protected]' ? 'zero' : ''
67+
```
68+
Used in: `boolean-targeted-zero-flag`, `string-targeted-zero-flag`, `integer-targeted-zero-flag`, `float-targeted-zero-flag`, `object-targeted-zero-flag`
69+
70+
#### Complex Multi-Condition Targeting
71+
```cel
72+
!customer && email == '[email protected]' && age > 10 ? 'internal' : ''
73+
```
74+
Used in: `complex-targeted`
75+
76+
## Flag Structure Schema
77+
78+
Each flag in the test data follows this structure:
79+
80+
```json
81+
{
82+
"flag-name": {
83+
"variants": {
84+
"variant-key": "variant-value"
85+
},
86+
"defaultVariant": "variant-key-or-null",
87+
"contextEvaluator": "CEL-expression", // Optional
88+
"flagMetadata": {} // Optional
89+
}
90+
}
91+
```
92+
93+
### Key Components
94+
- **variants**: Object containing all possible flag values mapped to variant keys
95+
- **defaultVariant**: The variant key to use when no targeting rules match (can be null or omitted)
96+
- **contextEvaluator**: Optional CEL expression for dynamic targeting
97+
- **flagMetadata**: Optional metadata object containing additional flag information
98+
99+
## Usage
100+
101+
1. **Test Implementation**: Parse the JSON file with your preferred JSON library
102+
2. **Context Setup**: Ensure your test contexts include the required variables (`email`, `customer`, `age`)
103+
3. **CEL Evaluation**: Implement CEL expression evaluation for flags with `contextEvaluator`
104+
4. **Edge Case Testing**: Use the zero flags and special flags to test boundary conditions
105+
106+
## Test Context Examples
107+
108+
For comprehensive testing, use these context combinations:
109+
110+
```json
111+
// Triggers targeted zero variants
112+
{
113+
"targetingKey": "user1",
114+
"email": "[email protected]",
115+
"customer": false,
116+
"age": 25
117+
}
118+
119+
// Triggers complex targeting
120+
{
121+
"targetingKey": "user2",
122+
"email": "[email protected]",
123+
"customer": false,
124+
"age": 15
125+
}
126+
127+
// Triggers alternative targeting
128+
{
129+
"targetingKey": "user3",
130+
"email": "[email protected]"
131+
}
132+
133+
// Default behavior (no targeting matches)
134+
{
135+
"targetingKey": "user4",
136+
"email": "[email protected]",
137+
"customer": true,
138+
"age": 30
139+
}
140+
```
141+
142+
## Contributing
143+
144+
When modifying test data:
145+
1. Maintain the category structure (standard, zero, targeted-zero, disabled)
146+
2. Validate CEL expressions for syntax correctness
147+
3. Ensure all required context variables are documented
148+
4. Test both matching and non-matching targeting scenarios

‎specification/assets/gherkin/evaluation.feature‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
@deprecated
12
Feature: Flag evaluation
23

34
# This test suite contains scenarios to test the flag evaluation API.
@@ -57,11 +58,11 @@ Feature: Flag evaluation
5758

5859
# errors
5960
Scenario: Flag not found
60-
When a non-existent string flag with key "missing-flag" is evaluated with details and a default value "uh-oh"
61+
When a non-existent string flag with key "missing-flag" is evaluated with details and a fallback value "uh-oh"
6162
Then the default string value should be returned
6263
And the reason should indicate an error and the error code should indicate a missing flag with "FLAG_NOT_FOUND"
6364

6465
Scenario: Type error
65-
When a string flag with key "wrong-flag" is evaluated as an integer, with details and a default value 13
66+
When a string flag with key "wrong-flag" is evaluated as an integer, with details and a fallback value 13
6667
Then the default integer value should be returned
6768
And the reason should indicate an error and the error code should indicate a type mismatch with "TYPE_MISMATCH"

0 commit comments

Comments
 (0)