Skip to content

Commit 5465887

Browse files
DavidRajnohaDavidRajnoha
committed
feat(Cypress): add Prometheus query mocking system for incident scenarios
- Implement Prometheus query_range API interception for mocking the incidents page data - Add complete YAML-based incident fixture system with JSON Schema validation - Support complex incident timelines with severity changes and resolved/ongoing states - Include 5 predefined incident scenario fixtures - Add custom Cypress commands: cy.mockIncidents() and cy.mockIncidentFixture() - Support both YAML and JSON fixture formats - Add timezone configuration via CYPRESS_TIMEZONE environment variable (necessary for correct timeline mocking) - Implement schema validation with CLI tool for fixture validation - Add Cursor commands to generate the scenarios in .cursor/commands/ - Add example test demonstrating different mocking approaches - Integrate mocking system into existing incidents test (01.incidents.cy.ts) - Add dependencies: ajv, js-yaml, @types/ajv, @types/js-yaml - Fix Cypress fixtures folder path from 'fixtures' to 'cypress/fixtures'
1 parent 81a7e26 commit 5465887

29 files changed

+2614
-1388
lines changed

‎.cursor/commands/fixture-schema-reference.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# Fixture Schema Reference
2+
3+
## Overview
4+
Quick reference for the YAML fixture schema structure, valid values, and common patterns.
5+
6+
## Schema Structure
7+
8+
```yaml
9+
name: string # Required: Human-readable scenario name
10+
description: string # Required: Detailed description
11+
incidents: array # Required: Array of incident objects
12+
- id: string # Required: Unique incident identifier
13+
component: string # Required: Component affected
14+
layer: string # Required: Layer (core/Others)
15+
timeline: object # Optional: Timeline information
16+
start: string # Required: When incident started
17+
end: string # Optional: When incident ended
18+
severityChanges: array # Optional: Severity changes over time
19+
alerts: array # Required: Array of alert objects
20+
- name: string # Required: Alert name
21+
namespace: string # Required: Alert namespace
22+
severity: string # Required: Alert severity
23+
firing: boolean # Optional: Firing state
24+
managed_cluster: string # Optional: Managed cluster ID
25+
```
26+
27+
## Valid Values
28+
29+
### Components
30+
- `monitoring` - Monitoring infrastructure
31+
- `storage` - Storage systems
32+
- `network` - Network components
33+
- `compute` - Compute resources
34+
- `api-server` - Kubernetes API server
35+
- `etcd` - etcd cluster
36+
- `version` - Cluster version
37+
- `Others` - Other components
38+
39+
### Layers
40+
- `core` - Core OpenShift components
41+
- `Others` - Non-core components
42+
43+
### Severities
44+
- `critical` - Critical alerts
45+
- `warning` - Warning alerts
46+
- `info` - Informational alerts
47+
48+
### Duration Format
49+
- Pattern: `^\d+[smhd]$`
50+
- Examples: `"30m"`, `"2h"`, `"7d"`, `"1h"`
51+
- Units: `s` (seconds), `m` (minutes), `h` (hours), `d` (days)
52+
53+
## Common Alert Names
54+
55+
### Monitoring Component
56+
```yaml
57+
- name: "AlertmanagerReceiversNotConfigured"
58+
namespace: "openshift-monitoring"
59+
severity: "warning"
60+
61+
- name: "KubeDeploymentReplicasMismatch"
62+
namespace: "openshift-monitoring"
63+
severity: "critical"
64+
65+
- name: "KubePodCrashLooping"
66+
namespace: "openshift-monitoring"
67+
severity: "warning"
68+
69+
- name: "PrometheusConfigReloadFailure"
70+
namespace: "openshift-monitoring"
71+
severity: "warning"
72+
73+
- name: "Watchdog"
74+
namespace: "openshift-monitoring"
75+
severity: "info"
76+
```
77+
78+
### Storage Component
79+
```yaml
80+
- name: "KubePersistentVolumeFillingUp"
81+
namespace: "openshift-storage"
82+
severity: "critical"
83+
84+
- name: "DiskSpaceRunningLow"
85+
namespace: "openshift-storage"
86+
severity: "critical"
87+
88+
- name: "VolumeReadOnly"
89+
namespace: "openshift-storage"
90+
severity: "critical"
91+
```
92+
93+
### Network Component
94+
```yaml
95+
- name: "NetworkLatencyHigh"
96+
namespace: "openshift-network"
97+
severity: "warning"
98+
99+
- name: "EtcdMemberCommunicationSlow"
100+
namespace: "openshift-etcd"
101+
severity: "warning"
102+
```
103+
104+
### Compute Component
105+
```yaml
106+
- name: "NodeNotReady"
107+
namespace: "openshift-machine-api"
108+
severity: "critical"
109+
110+
- name: "KubeNodeUnreachable"
111+
namespace: "openshift-machine-api"
112+
severity: "critical"
113+
```
114+
115+
## Timeline Patterns
116+
117+
### Simple Ongoing Incident
118+
```yaml
119+
timeline:
120+
start: "2h" # Started 2 hours ago
121+
```
122+
123+
### Resolved Incident
124+
```yaml
125+
timeline:
126+
start: "4h" # Started 4 hours ago
127+
end: "1h" # Resolved 1 hour ago
128+
```
129+
130+
### Severity Escalation
131+
```yaml
132+
timeline:
133+
start: "3h" # Started 3 hours ago
134+
severityChanges:
135+
- time: "3h" # Started as warning
136+
severity: "warning"
137+
- time: "1h" # Escalated to critical 1 hour ago
138+
severity: "critical"
139+
```
140+
141+
## Validation Rules
142+
143+
### Required Fields
144+
- Root: `name`, `description`, `incidents`
145+
- Incident: `id`, `component`, `layer`, `alerts`
146+
- Alert: `name`, `namespace`, `severity`
147+
- Timeline: `start`
148+
149+
### Format Constraints
150+
- Incident ID: Pattern `^[a-zA-Z0-9-_]+$`
151+
- Duration: Pattern `^\d+[smhd]$`
152+
- Component/Layer/Severity: Must be valid enum values
153+
154+
### Array Constraints
155+
- Incidents: `minItems: 0`
156+
- Alerts: `minItems: 1` per incident
157+
158+
## Usage in Tests
159+
160+
```typescript
161+
// Load YAML fixture
162+
cy.mockIncidentFixture('cypress/fixtures/incident-scenarios/critical-monitoring-issues.yaml');
163+
164+
// Load JSON fixture (backward compatibility)
165+
cy.mockIncidentFixture('cypress/fixtures/incident-scenarios/some-scenario.json');
166+
```
167+
168+
## Validation Tools
169+
170+
### CLI Validation
171+
```bash
172+
cd web/cypress/support/incidents_prometheus_query_mocks
173+
node validate-fixtures.js --all
174+
node validate-fixtures.js specific-file.yaml
175+
```
176+
177+
### Schema Files
178+
- Schema: `web/cypress/support/incidents_prometheus_query_mocks/fixture-schema.json`
179+
- Validator: `web/cypress/support/incidents_prometheus_query_mocks/schema-validator.ts`
180+
- CLI Tool: `web/cypress/support/incidents_prometheus_query_mocks/validate-fixtures.js`
181+
182+
## Best Practices
183+
184+
1. **Naming**: Use descriptive, specific names
185+
2. **Comments**: Add helpful comments for timeline values
186+
3. **Realistic Data**: Use actual OpenShift alert names
187+
4. **Consistent Format**: Follow established patterns
188+
5. **Validation**: Always validate before committing

‎.cursor/commands/generate-incident-fixture.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
description: Generate valid YAML incident fixtures for Cypress testing
3+
---
4+
5+
# Generate Incident Fixture
6+
7+
Generate valid YAML fixtures for incident scenarios based on text descriptions or UI screenshots.
8+
9+
## Process
10+
11+
### 1. Analyze Input
12+
- **Text Description**: Extract incident components, alert names, severities, and timeline
13+
- **Screenshot**: Identify visible incidents, alerts, and timeline - **CONVERT ALL DATES TO RELATIVE TIME**. There will be usually one chart 'Incident Timeline' displaying the incident timelines and second one 'Alert Timelines' which displays alerts for **a single particular incident**. When the Alert timeline is visible, include timestamps for the individual alerts within the incident, which may differ from the incident one.
14+
- Focus on capturing the precise start and end dates for incidents and individual alerts so that the variability from the screenshot is preserved.
15+
16+
### 2. Generate YAML Structure
17+
Create fixture following the schema in [fixture-schema.json](mdc:web/cypress/support/incidents_prometheus_query_mocks/fixture-schema.json). Prefer this schema over existing fixtures.
18+
19+
```yaml
20+
name: "[Scenario Name]"
21+
description: "[Detailed description]"
22+
incidents:
23+
- id: "[unique-incident-id]"
24+
component: "[component-name]"
25+
layer: "[core|Others]"
26+
timeline:
27+
start: "[duration]"
28+
# end: "[duration]" # Only if resolved
29+
alerts:
30+
- name: "[alert-name]"
31+
namespace: "[namespace]"
32+
severity: "[critical|warning|info]"
33+
firing: [true|false]
34+
timeline:
35+
start: "[duration]"
36+
```
37+
38+
### 3. Apply Constraints
39+
- **Name and Description**: Use generic name capturing the essence of the scenario
40+
- **File name**: Use generic name capturing the essence of the scenario
41+
- **Components**: `monitoring`, `storage`, `network`, `compute`, `api-server`, `etcd`, `version`, `Others`
42+
- **Layers**: `core`, `Others`
43+
- **Severities**: `critical`, `warning`, `info`
44+
- **Duration Format**: `<number><unit>` (e.g., `"2h"`, `"30m"`, `"7d"`)
45+
- **Alert Names**: Use descriptive, artificial names with unique indices (e.g., `"MonitoringAlertmanagerReceiversCritical001"`)
46+
47+
### 4. Time Conversion Rules
48+
- **NEVER use absolute dates/timestamps**
49+
- Convert screenshot dates to relative durations from the end of the chart
50+
- Example: If screenshot shows "2024-01-15 14:30" and current time is "2024-01-15 16:30", use `"2h"`
51+
52+
### 5. Run validation
53+
- run the validation using [fixture-schema.json](mdc:web/cypress/support/incident_prometheus_query_mocks/validate_fixture.js)
54+
55+
```yaml
56+
57+
## Validation Checklist
58+
- [ ] All required fields present (name, description, incidents)
59+
- [ ] Incident IDs are unique and follow naming convention
60+
- [ ] Components and layers use valid enum values from schema
61+
- [ ] Duration format matches pattern `^\d+[smhd]$`
62+
- [ ] **NO absolute dates or timestamps used - only relative durations**
63+
- [ ] Alert names are descriptive and artificial with unique indices
64+
- [ ] Alerts have timelines if different when the incidents
65+
- [ ] Namespaces follow OpenShift conventions
66+
- [ ] Severity levels are valid (critical, warning, info)
67+
- [ ] YAML syntax is correct with proper indentation
68+
- [ ] Schema validation passes against [fixture-schema.json](mdc:web/cypress/support/incidents_prometheus_query_mocks/fixture-schema.json)
69+
70+
## Output
71+
Provide complete YAML fixture ready for use with `cy.mockIncidentFixture()` in Cypress tests.

‎.cursor/commands/validate-incident-fixtures.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Validate Incident Fixtures
2+
3+
## Overview
4+
Validate existing YAML incident fixture files against the JSON Schema to ensure they are properly structured and will work correctly in tests.
5+
6+
## Process
7+
8+
### 1. Locate Fixture Files
9+
- Check `web/cypress/fixtures/incident-scenarios/` directory
10+
- Identify all `.yaml` and `.yml` files
11+
- Also validate any `.json` fixture files for backward compatibility
12+
13+
### 2. Run Schema Validation
14+
Use the validation tool to check each fixture:
15+
```bash
16+
cd web/cypress/support/incidents_prometheus_query_mocks
17+
node validate-fixtures.js --all
18+
```
19+
20+
### 3. Analyze Results
21+
For each fixture file:
22+
- ✅ **Valid**: Fixture passes all schema validation
23+
- ❌ **Invalid**: Identify specific validation errors
24+
- 🔧 **Fixable**: Determine if errors can be automatically corrected
25+
26+
### 4. Fix Validation Issues
27+
Common issues and fixes:
28+
29+
#### Missing Required Fields
30+
- Add missing `name`, `description`, or `incidents` fields
31+
- Ensure all incidents have required `id`, `component`, `layer`, `alerts`
32+
33+
#### Invalid Duration Format
34+
- Fix duration strings to use format: `"2h"`, `"30m"`, `"7d"`
35+
- Remove invalid formats like `"2 hours"`, `"30 minutes"`
36+
37+
#### Invalid Enum Values
38+
- **Components**: Use only valid values (monitoring, storage, network, etc.)
39+
- **Layers**: Use only `"core"` or `"Others"`
40+
- **Severities**: Use only `"critical"`, `"warning"`, or `"info"`
41+
42+
#### YAML Syntax Errors
43+
- Fix indentation issues
44+
- Correct quote usage
45+
- Ensure proper array formatting
46+
47+
#### Alert Structure Issues
48+
- Ensure alerts have `name`, `namespace`, `severity`
49+
- Verify alert names are realistic OpenShift alert names
50+
- Check namespace formats match OpenShift conventions
51+
52+
## Validation Checklist
53+
54+
### Schema Compliance
55+
- [ ] All required fields present
56+
- [ ] Valid component enum values
57+
- [ ] Valid layer enum values
58+
- [ ] Valid severity enum values
59+
- [ ] Proper duration format
60+
- [ ] Correct YAML syntax
61+
62+
### Content Quality
63+
- [ ] Realistic alert names
64+
- [ ] Proper OpenShift namespaces
65+
- [ ] Logical timeline values
66+
- [ ] Descriptive scenario names
67+
- [ ] Clear descriptions
68+
69+
### Test Readiness
70+
- [ ] Fixtures can be loaded without errors
71+
- [ ] Schema validation passes
72+
- [ ] Ready for use in Cypress tests
73+
74+
## Common Fixes
75+
76+
### Fix Duration Format
77+
```yaml
78+
# Before
79+
start: "2 hours ago"
80+
end: "30 minutes ago"
81+
82+
# After
83+
start: "2h" # Started 2 hours ago
84+
end: "30m" # Resolved 30 minutes ago
85+
```
86+
87+
### Fix Component Names
88+
```yaml
89+
# Before
90+
component: "monitoring-system"
91+
92+
# After
93+
component: "monitoring"
94+
```
95+
96+
### Fix Alert Structure
97+
```yaml
98+
# Before
99+
alerts:
100+
- alert_name: "SomeAlert"
101+
alert_namespace: "monitoring"
102+
alert_severity: "high"
103+
104+
# After
105+
alerts:
106+
- name: "AlertmanagerReceiversNotConfigured"
107+
namespace: "openshift-monitoring"
108+
severity: "critical"
109+
firing: true
110+
```
111+
112+
## Output
113+
Provide:
114+
1. **Validation Summary**: List of all fixtures and their validation status
115+
2. **Error Details**: Specific validation errors for any failed fixtures
116+
3. **Fix Suggestions**: Recommended changes for invalid fixtures
117+
4. **Corrected Fixtures**: Updated YAML content for any fixtures that need fixes
118+
119+
## Tools Available
120+
- Schema validator: `web/cypress/support/incidents_prometheus_query_mocks/schema-validator.ts`
121+
- CLI validator: `web/cypress/support/incidents_prometheus_query_mocks/validate-fixtures.js`
122+
- JSON Schema: `web/cypress/support/incidents_prometheus_query_mocks/fixture-schema.json`

‎web/cypress.config.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,10 @@ export default defineConfig({
2121
),
2222
LOGIN_USERNAME: process.env.CYPRESS_LOGIN_USERS.split(',')[0].split(':')[0],
2323
LOGIN_PASSWORD: process.env.CYPRESS_LOGIN_USERS.split(',')[0].split(':')[1],
24+
TIMEZONE: process.env.CYPRESS_TIMEZONE || 'UTC',
2425
typeDelay: 200,
2526
},
26-
fixturesFolder: 'fixtures',
27+
fixturesFolder: 'cypress/fixtures',
2728
defaultCommandTimeout: 80000, //due to performance loading issue on console
2829
readyTimeoutMilliseconds: 120000,
2930
installTimeoutMilliseconds: 600000,

0 commit comments

Comments
 (0)