Skip to content

Commit f5c2173

Browse files
joejstuartjoejstuart
committed
document rule filtering process
1 parent 338efbf commit f5c2173

File tree

1 file changed

+221
-0
lines changed

1 file changed

+221
-0
lines changed

‎.cursor/rules/rule_filtering_process.mdc‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
---
2+
description:
3+
globs:
4+
alwaysApply: false
5+
---
6+
# Rule Filtering Process in Conftest Evaluator
7+
8+
This document explains the complex rule filtering process that occurs in the `Evaluate` function of the `conftestEvaluator`. The filtering happens after the `conftestRunner` executes all policy rules but before the final results are returned.
9+
10+
## Overview
11+
12+
The rule filtering process consists of several stages:
13+
14+
1. **Initial Rule Execution** - All policy rules are executed by the conftest runner
15+
2. **Result Processing** - Each result type (warnings, failures, exceptions, skipped) is processed
16+
3. **Inclusion/Exclusion Filtering** - Results are filtered based on policy configuration
17+
4. **Severity-Based Filtering** - Results can be promoted or demoted between warning/failure categories
18+
5. **Effective Time Filtering** - Future-effective failures are demoted to warnings
19+
6. **Success Computation** - Success results are computed from unmatched rules
20+
7. **Missing Includes Handling** - Warnings are generated for unmatched include criteria
21+
8. **Dependency Trimming** - Results with unsatisfied dependencies are removed
22+
9. **Final Validation** - Ensures at least some rules were processed
23+
24+
## Detailed Process
25+
26+
### 1. Initial Rule Execution
27+
28+
```go
29+
runResults, err := r.Run(ctx, target.Inputs)
30+
```
31+
32+
The `conftestRunner` executes all policy rules and returns raw results containing:
33+
- **Warnings**: Rules that generated warning violations
34+
- **Failures**: Rules that generated failure violations
35+
- **Exceptions**: Rules that were explicitly excepted
36+
- **Skipped**: Rules that were skipped
37+
- **Successes**: Count of successful rules (not detailed results)
38+
39+
### 2. Result Processing by Type
40+
41+
For each namespace result, the following processing occurs:
42+
43+
#### Warning Processing
44+
45+
```go
46+
for i := range result.Warnings {
47+
warning := result.Warnings[i]
48+
addRuleMetadata(ctx, &warning, rules)
49+
50+
if !c.isResultIncluded(warning, target.Target, missingIncludes) {
51+
// Filter out excluded warnings
52+
continue
53+
}
54+
55+
if getSeverity(warning) == severityFailure {
56+
// Promote to failure if severity indicates it should be a failure
57+
failures = append(failures, warning)
58+
} else {
59+
warnings = append(warnings, warning)
60+
}
61+
}
62+
```
63+
64+
**Filtering Logic:**
65+
1. **Metadata Addition**: Rule metadata is added from policy annotations
66+
2. **Inclusion Check**: Warning is checked against include/exclude criteria
67+
3. **Severity Promotion**: Warnings with `severity: failure` are promoted to failures
68+
69+
#### Failure Processing
70+
71+
```go
72+
for i := range result.Failures {
73+
failure := result.Failures[i]
74+
addRuleMetadata(ctx, &failure, rules)
75+
76+
if !c.isResultIncluded(failure, target.Target, missingIncludes) {
77+
// Filter out excluded failures
78+
continue
79+
}
80+
81+
if getSeverity(failure) == severityWarning || !isResultEffective(failure, effectiveTime) {
82+
// Demote to warning if severity indicates or if not yet effective
83+
warnings = append(warnings, failure)
84+
} else {
85+
failures = append(failures, failure)
86+
}
87+
}
88+
```
89+
90+
**Filtering Logic:**
91+
1. **Metadata Addition**: Rule metadata is added from policy annotations
92+
2. **Inclusion Check**: Failure is checked against include/exclude criteria
93+
3. **Severity Demotion**: Failures with `severity: warning` are demoted to warnings
94+
4. **Effective Time Check**: Failures with future `effective_on` dates are demoted to warnings
95+
96+
#### Exception and Skipped Processing
97+
98+
```go
99+
for i := range result.Exceptions {
100+
exception := result.Exceptions[i]
101+
addRuleMetadata(ctx, &exception, rules)
102+
exceptions = append(exceptions, exception)
103+
}
104+
105+
for i := range result.Skipped {
106+
skip := result.Skipped[i]
107+
addRuleMetadata(ctx, &skip, rules)
108+
skipped = append(skipped, skip)
109+
}
110+
```
111+
112+
**Processing:**
113+
- Exceptions and skipped results only have metadata added
114+
- No inclusion/exclusion filtering is applied to these result types
115+
116+
### 3. Inclusion/Exclusion Filtering
117+
118+
The `isResultIncluded` function determines whether a result should be included based on:
119+
120+
#### Include/Exclude Criteria Structure
121+
122+
```go
123+
type Criteria struct {
124+
defaultItems []string // Apply to all targets
125+
digestItems map[string][]string // Apply to specific image digests
126+
}
127+
```
128+
129+
Results are scored based on specificity:
130+
- `@collection` patterns score 10 points
131+
- `*` (wildcard) scores 1 point
132+
- Package names (`pkg`, `pkg.*`) score 10 points per namespace level
133+
- Specific rules (`pkg.rule`) score additional 100 points
134+
- Terms (`:term` suffix) score additional 100 points
135+
136+
**Decision Logic:**
137+
```go
138+
includeScore := scoreMatches(ruleMatchers, c.include.get(target), missingIncludes)
139+
excludeScore := scoreMatches(ruleMatchers, c.exclude.get(target), map[string]bool{})
140+
return includeScore > excludeScore
141+
```
142+
143+
A result is included if its include score is higher than its exclude score.
144+
145+
#### Scoring Example
146+
147+
Consider a rule with code `pipeline.required_tasks` and term `build`:
148+
149+
**Possible matchers generated:**
150+
- `pipeline` (package) - scores 10
151+
- `pipeline.*` (package wildcard) - scores 10
152+
- `pipeline.required_tasks` (specific rule) - scores 110 (10 + 100)
153+
- `pipeline:build` (package with term) - scores 110 (10 + 100)
154+
- `pipeline.*:build` (package wildcard with term) - scores 110 (10 + 100)
155+
- `pipeline.required_tasks:build` (specific rule with term) - scores 210 (10 + 100 + 100)
156+
- `*` (global wildcard) - scores 1
157+
- `@mandatory` (collection, if rule belongs to it) - scores 10
158+
159+
**Evaluation scenarios:**
160+
1. **Include:** `["pipeline.*"]`, **Exclude:** `[]` → Include score: 10, Exclude score: 0 → **INCLUDED**
161+
2. **Include:** `["*"]`, **Exclude:** `["pipeline.required_tasks"]` → Include score: 1, Exclude score: 110 → **EXCLUDED**
162+
3. **Include:** `["pipeline.required_tasks:build"]`, **Exclude:** `["pipeline.*"]` → Include score: 210, Exclude score: 10 → **INCLUDED**
163+
164+
### 4. Success Computation
165+
166+
```go
167+
result.Successes = c.computeSuccesses(result, rules, target.Target, missingIncludes)
168+
```
169+
170+
Success results are computed by:
171+
1. Identifying all rules that didn't appear in warnings, failures, exceptions, or skipped
172+
2. Checking if each unmatched rule should be included based on include/exclude criteria
173+
3. Creating success results for included rules
174+
175+
### 5. Missing Includes Handling
176+
177+
```go
178+
for missingInclude, isMissing := range missingIncludes {
179+
if isMissing {
180+
results = append(results, Outcome{
181+
Warnings: []Result{{
182+
Message: fmt.Sprintf("Include criterion '%s' doesn't match any policy rule", missingInclude),
183+
}},
184+
})
185+
}
186+
}
187+
```
188+
189+
Any include criteria that didn't match any rules generate warning results.
190+
191+
### 6. Dependency Trimming
192+
193+
```go
194+
trim(&results)
195+
```
196+
197+
The `trim` function removes results that depend on rules that have been reported as failures, warnings, or skipped:
198+
199+
1. **Dependency Collection**: Identifies all rules marked as failed/warned/skipped
200+
2. **Dependent Removal**: Removes any results that declare dependencies on those rules via `depends_on` metadata
201+
3. **Exclusion Notes**: Adds notes to remaining failures suggesting how to exclude them
202+
203+
### 7. Final Validation
204+
205+
```go
206+
if totalRules == 0 {
207+
return nil, fmt.Errorf("no successes, warnings, or failures, check input")
208+
}
209+
```
210+
211+
The process ensures that at least some rules were processed. If no rules were evaluated, it indicates an input error.
212+
213+
## Key Considerations
214+
215+
1. **Order Matters**: Warnings can be promoted to failures, and failures can be demoted to warnings
216+
2. **Inclusion Trumps Source**: A rule's original category (warning vs failure) can be overridden by include/exclude logic
217+
3. **Time-Based Logic**: Rules with future effective dates are treated as warnings until they become effective
218+
4. **Dependency Chains**: Failed dependencies can cascade to remove dependent rule results
219+
5. **Missing Includes**: Unmatched include patterns generate their own warning results
220+
221+
This filtering system provides fine-grained control over which policy violations are reported and how they're categorized, allowing for gradual policy rollouts and context-specific rule management.

0 commit comments

Comments
 (0)