Skip to content

Commit 2dc31bf

Browse files
committed
v0.36.0: vstack.redaction + health + priority_queue + snippet + aggregate
Five more feature modules (thirty-four total): - vstack.redaction: PII/secret scrubbing for traces. Built-in patterns for email, phone, SSN, credit card, AWS keys, sk-prefixed API keys, Bearer tokens, JWT, IPv4, URL credentials. Redactor tracks match counts; scrub_trace() returns redacted copy with original unmutated. - vstack.health: composite health checks. Check protocol + CallableCheck adapter. HealthReport with HEALTHY/DEGRADED/ UNHEALTHY status. HealthMonitor with interval-aware tick(). - vstack.priority_queue: finding priority queue with aging. Score = severity_weight × confidence_multiplier + age_boost + manual_boost. Aging prevents low-severity starvation. boost/remove_pattern/snapshot helpers. - vstack.snippet: minimal trace excerpts. find_relevant_steps() via token-overlap heuristic. extract_snippet() pulls context around relevant steps with omission counts. render_snippet() produces markdown. - vstack.aggregate: cross-report aggregation. AggregateSummary with per-pattern stats + severity/agent counts. top_n_patterns, top_n_agents, severity_distribution, cooccurrence_matrix. 113 new tests; all 3,127 tests pass (was 3,014).
1 parent 325d6fb commit 2dc31bf

18 files changed

Lines changed: 2336 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,55 @@ project adheres to [Semantic Versioning](https://semver.org/) from
66
`1.0.0` onward. During the `0.x` series, minor bumps may include
77
breaking changes (see API stability promise in `vstack/__init__.py`).
88

9+
## [0.36.0] — 2026-06-09
10+
11+
Five more feature modules: redaction + health + priority_queue +
12+
snippet + aggregate. Thirty-four feature modules total since v0.23.0.
13+
14+
### Added
15+
16+
- **`vstack.redaction`** — PII / secret scrubbing for traces.
17+
`RedactionPattern` (regex + replacement); built-in
18+
`DEFAULT_PATTERNS` covering email, US/CA phone, SSN, credit card,
19+
AWS access/secret keys, sk-prefixed API keys, Bearer tokens, JWT,
20+
IPv4, URLs with user:pass@ credentials. `Redactor` tracks per-
21+
pattern match counts; `scrub_trace()` returns a redacted copy of
22+
the trace with goal/outcome/step content scrubbed (original
23+
unmutated).
24+
- **`vstack.health`** — composite health checks. `Check` protocol +
25+
`CallableCheck` adapter. `HealthReport` aggregates with
26+
HEALTHY / DEGRADED / UNHEALTHY status (critical UNHEALTHY →
27+
UNHEALTHY; non-critical UNHEALTHY → DEGRADED). `HealthMonitor`
28+
with `tick()` (interval-aware) + `force_tick()` for scheduler-
29+
driven probes.
30+
- **`vstack.priority_queue`** — finding priority queue with aging
31+
boost. `FindingPriorityQueue` heap-backed; score = severity_weight
32+
(high=100/med=10/low=1) × confidence_multiplier (0.5-1.0) +
33+
age_boost (aging_multiplier × hours_elapsed) + manual_boost.
34+
`boost()` / `remove_pattern()` / `snapshot()` helpers. Aging
35+
prevents low-severity starvation.
36+
- **`vstack.snippet`** — minimal trace excerpts. `find_relevant_steps()`
37+
uses token-overlap (lowercase, stopword-filtered, ≥3 chars)
38+
between finding text and step content. `extract_snippet()` pulls
39+
N context steps around relevant steps with omission counts.
40+
`render_snippet()` produces markdown with `` markers on relevant
41+
steps + elision for long content.
42+
- **`vstack.aggregate`** — cross-report aggregation. `aggregate_reports()`
43+
returns `AggregateSummary` with per-pattern stats (high/med/low
44+
+ severity_score), severity_counts, agent_counts.
45+
`top_n_patterns()` / `top_n_agents()` (optionally severity-
46+
filtered), `severity_distribution()`, `cooccurrence_matrix()`
47+
(pairs that appear in the same report).
48+
49+
### Changed
50+
51+
- Test count: 3,014 → 3,127 (+113 from the five new modules).
52+
53+
### Compatibility
54+
55+
- All 3,127 tests pass (1 skipped: crewai not installed).
56+
- Public API surface strictly expanded.
57+
958
## [0.35.0] — 2026-06-09
1059

1160
Three more feature modules: alerting + eval_gates + intervention_tracker.

_aggregate/lib/__init__.py

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
"""vstack.aggregate — cross-report aggregation utilities.
2+
3+
Aggregate findings across many reports for fleet-level analysis:
4+
5+
- Top-N patterns by frequency.
6+
- Severity distribution.
7+
- Per-agent finding counts.
8+
- Co-occurrence: which pattern pairs appear together?
9+
- Trend: severity rate over time.
10+
11+
Quick start
12+
-----------
13+
14+
from vstack.aggregate import (
15+
aggregate_reports,
16+
top_n_patterns,
17+
cooccurrence_matrix,
18+
)
19+
20+
summary = aggregate_reports(reports)
21+
print(summary.total_findings, summary.unique_patterns)
22+
23+
top = top_n_patterns(reports, n=5)
24+
for pattern, count in top:
25+
print(pattern, count)
26+
"""
27+
28+
from __future__ import annotations
29+
30+
from ._aggregate import (
31+
AggregateSummary,
32+
PatternStats,
33+
aggregate_reports,
34+
cooccurrence_matrix,
35+
severity_distribution,
36+
top_n_agents,
37+
top_n_patterns,
38+
)
39+
40+
__all__ = [
41+
"AggregateSummary",
42+
"PatternStats",
43+
"aggregate_reports",
44+
"cooccurrence_matrix",
45+
"severity_distribution",
46+
"top_n_agents",
47+
"top_n_patterns",
48+
]

_aggregate/lib/_aggregate.py

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
"""Cross-report aggregation."""
2+
3+
from __future__ import annotations
4+
5+
from collections import Counter
6+
from dataclasses import dataclass, field
7+
from typing import Any
8+
9+
10+
def _get_findings(report: Any) -> list[Any]:
11+
if isinstance(report, dict):
12+
return list(report.get("findings", []))
13+
if hasattr(report, "findings"):
14+
return list(report.findings)
15+
if isinstance(report, list):
16+
return list(report)
17+
return []
18+
19+
20+
def _f(finding: Any, field: str, default: Any = "") -> Any:
21+
if isinstance(finding, dict):
22+
return finding.get(field, default)
23+
return getattr(finding, field, default)
24+
25+
26+
@dataclass
27+
class PatternStats:
28+
"""Per-pattern aggregate stats."""
29+
30+
pattern: str
31+
total: int = 0
32+
high: int = 0
33+
medium: int = 0
34+
low: int = 0
35+
36+
@property
37+
def severity_score(self) -> float:
38+
"""Weighted severity (high=3, med=2, low=1)."""
39+
return self.high * 3 + self.medium * 2 + self.low
40+
41+
def to_dict(self) -> dict[str, Any]:
42+
return {
43+
"pattern": self.pattern,
44+
"total": self.total,
45+
"high": self.high,
46+
"medium": self.medium,
47+
"low": self.low,
48+
"severity_score": self.severity_score,
49+
}
50+
51+
52+
@dataclass
53+
class AggregateSummary:
54+
"""Cross-report summary."""
55+
56+
total_reports: int = 0
57+
total_findings: int = 0
58+
unique_patterns: int = 0
59+
pattern_stats: dict[str, PatternStats] = field(default_factory=dict)
60+
severity_counts: dict[str, int] = field(default_factory=dict)
61+
agent_counts: dict[str, int] = field(default_factory=dict)
62+
63+
def top_patterns(self, n: int = 10) -> list[PatternStats]:
64+
items = sorted(
65+
self.pattern_stats.values(),
66+
key=lambda p: (-p.total, -p.severity_score, p.pattern),
67+
)
68+
return items[:n]
69+
70+
def to_dict(self) -> dict[str, Any]:
71+
return {
72+
"total_reports": self.total_reports,
73+
"total_findings": self.total_findings,
74+
"unique_patterns": self.unique_patterns,
75+
"pattern_stats": {k: v.to_dict() for k, v in self.pattern_stats.items()},
76+
"severity_counts": dict(self.severity_counts),
77+
"agent_counts": dict(self.agent_counts),
78+
}
79+
80+
81+
def aggregate_reports(
82+
reports: list[Any],
83+
*,
84+
agent_id_extractor: Any = None,
85+
) -> AggregateSummary:
86+
"""Cross-report aggregation.
87+
88+
Args:
89+
reports: list of report objects (dict or attr-style).
90+
agent_id_extractor: callable(report) → str. Default: pulls
91+
'agent_id' from the report dict.
92+
"""
93+
summary = AggregateSummary(total_reports=len(reports))
94+
95+
if agent_id_extractor is None:
96+
97+
def agent_id_extractor(r: Any) -> str:
98+
if isinstance(r, dict):
99+
return str(r.get("agent_id", "unknown"))
100+
return str(getattr(r, "agent_id", "unknown"))
101+
102+
for report in reports:
103+
findings = _get_findings(report)
104+
agent = agent_id_extractor(report)
105+
106+
for finding in findings:
107+
pattern = str(_f(finding, "pattern", "unknown"))
108+
severity = str(_f(finding, "severity", "low"))
109+
110+
stats = summary.pattern_stats.setdefault(pattern, PatternStats(pattern=pattern))
111+
stats.total += 1
112+
if severity == "high":
113+
stats.high += 1
114+
elif severity == "medium":
115+
stats.medium += 1
116+
else:
117+
stats.low += 1
118+
119+
summary.severity_counts[severity] = summary.severity_counts.get(severity, 0) + 1
120+
summary.agent_counts[agent] = summary.agent_counts.get(agent, 0) + 1
121+
summary.total_findings += 1
122+
123+
summary.unique_patterns = len(summary.pattern_stats)
124+
return summary
125+
126+
127+
def top_n_patterns(reports: list[Any], n: int = 10) -> list[tuple[str, int]]:
128+
"""Return [(pattern, count)] for the top-N patterns by frequency."""
129+
counter: Counter[str] = Counter()
130+
for report in reports:
131+
for finding in _get_findings(report):
132+
pattern = str(_f(finding, "pattern", "unknown"))
133+
counter[pattern] += 1
134+
return counter.most_common(n)
135+
136+
137+
def top_n_agents(
138+
reports: list[Any],
139+
*,
140+
n: int = 10,
141+
severity: str | None = None,
142+
) -> list[tuple[str, int]]:
143+
"""Return [(agent_id, finding_count)] for the noisiest agents.
144+
145+
If ``severity`` is given, only count findings of that severity.
146+
"""
147+
counter: Counter[str] = Counter()
148+
for report in reports:
149+
agent = "unknown"
150+
if isinstance(report, dict):
151+
agent = str(report.get("agent_id", "unknown"))
152+
elif hasattr(report, "agent_id"):
153+
agent = str(report.agent_id)
154+
155+
for finding in _get_findings(report):
156+
if severity is not None and _f(finding, "severity") != severity:
157+
continue
158+
counter[agent] += 1
159+
return counter.most_common(n)
160+
161+
162+
def severity_distribution(reports: list[Any]) -> dict[str, int]:
163+
"""Count of findings per severity bucket."""
164+
dist: dict[str, int] = {"high": 0, "medium": 0, "low": 0}
165+
for report in reports:
166+
for finding in _get_findings(report):
167+
severity = str(_f(finding, "severity", "low"))
168+
dist[severity] = dist.get(severity, 0) + 1
169+
return dist
170+
171+
172+
def cooccurrence_matrix(reports: list[Any]) -> dict[tuple[str, str], int]:
173+
"""Count co-occurrence of pattern pairs within the same report.
174+
175+
Returns a dict keyed by (pattern_a, pattern_b) where a < b
176+
(tuples ordered canonically). Value = number of reports in
177+
which both patterns appeared.
178+
"""
179+
matrix: dict[tuple[str, str], int] = {}
180+
for report in reports:
181+
patterns = set()
182+
for finding in _get_findings(report):
183+
patterns.add(str(_f(finding, "pattern", "unknown")))
184+
# All pairs.
185+
plist = sorted(patterns)
186+
for i, a in enumerate(plist):
187+
for b in plist[i + 1 :]:
188+
key = (a, b)
189+
matrix[key] = matrix.get(key, 0) + 1
190+
return matrix

0 commit comments

Comments
 (0)