Skip to content

Commit 74305b9

Browse files
TamTunnelTamTunnel
committed
feat(v0.4): Add US governance, lineage tracking, human approvals
NIST AI RMF Alignment: - Map platform to Govern/Map/Measure/Manage functions - Document sectoral applicability (HIPAA, GLBA, CCPA) Data Classification: - Add DataSensitivity enum (public, internal, pii, phi, pci) - Add DataClassification enum (public, internal, confidential, restricted) - Add jurisdiction and sector fields to ModelRegistry Lineage Tracking: - Add Dataset model with sensitivity/classification - Add ModelDatasetLink for dataset-to-model lineage - Add ModelDependency for model-to-model lineage - Add lineage API endpoints (datasets, dependencies) Human-in-the-Loop Approvals: - Add approved_by_user_id, approved_at, approval_notes to ModelRegistry - Require approval_notes when approving models - Log approval metadata in audit trail SSO Documentation: - Add SSO/IdP integration section to README - Document reverse proxy and OIDC patterns Tests: - Add test_lineage.py for lineage models - Add test_classification.py for classification enums
1 parent b4ef06a commit 74305b9

File tree

11 files changed

+656
-159
lines changed

11 files changed

+656
-159
lines changed

‎README.md‎

Lines changed: 119 additions & 150 deletions
Original file line numberDiff line numberDiff line change
@@ -5,193 +5,166 @@
55

66
## What Is This?
77

8-
A **centralized platform** for managing your organization's AI models—a registry that tracks every AI system, its versions, performance metrics, risk profiles, and a complete audit trail for EU AI Act compliance.
8+
A **centralized platform** for managing your organization's AI models—supporting both **EU AI Act** and **US AI governance** (NIST AI RMF) requirements.
99

10-
**v0.3** adds: **Policy Engine** with enforcement, **Multi-tenancy** with org/environment scoping, and **Prometheus-compatible metrics** for enterprise observability.
11-
12-
---
13-
14-
## Who Is This For?
15-
16-
| Role | Value |
17-
|------|-------|
18-
| **ML Engineers** | Register models, track versions, store evaluation metrics via CI/CD |
19-
| **Compliance Officers** | Define policies, approve models, generate PDF reports |
20-
| **CTOs/CISOs** | Dashboard overview, policy enforcement, observability metrics |
21-
| **Auditors** | Read-only access to models, audit logs, policy violations |
10+
**v0.4** adds: **Data Classification**, **Lineage Tracking**, **Human-in-the-Loop Approvals**, and **NIST AI RMF alignment**.
2211

2312
---
2413

2514
## Key Features
2615

2716
| Feature | Description |
2817
|---------|-------------|
29-
| **Model Registry** | Register AI models with name, owner, risk profile |
30-
| **Risk Profiles** | EU AI Act levels (minimal, limited, high, unacceptable) |
31-
| **Compliance Lifecycle** | Status: draft → under_review → approved → retired |
32-
| **Policy Engine** | Define and enforce governance rules automatically |
33-
| **Policy Violations** | Track blocked actions with full audit trail |
34-
| **Multi-Tenancy** | Organization + environment (dev/test/prod) scoping |
35-
| **Prometheus Metrics** | `/api/v1/metrics` endpoint for observability |
36-
| **PDF Reports** | EU AI Act style compliance documentation |
37-
| **RBAC** | admin, model_owner, auditor roles |
18+
| **Model Registry** | Central catalog of AI models |
19+
| **Risk Profiles** | EU AI Act & NIST AI RMF classification |
20+
| **Data Classification** | Sensitivity (PII/PHI/PCI) and classification levels |
21+
| **Lineage Tracking** | Datasets and model dependencies |
22+
| **Human Approval** | Capture approver, approval notes, timestamps |
23+
| **Policy Engine** | Define and enforce governance rules |
24+
| **Multi-Tenancy** | Organization + environment scoping |
25+
| **SSO Ready** | Designed for IdP integration |
3826

3927
---
4028

41-
## Policy Engine
42-
43-
### Supported Policy Types
29+
## US AI Governance & NIST AI RMF Alignment
4430

45-
| Policy | Description |
46-
|--------|-------------|
47-
| `require_evaluation_before_approval` | Models must have evaluation metrics before approval |
48-
| `block_high_risk_without_approval` | High-risk models cannot skip `under_review` status |
49-
| `require_review_for_high_risk` | High-risk models require explicit review |
31+
This platform supports alignment with the **NIST AI Risk Management Framework (AI RMF)**.
5032

51-
### How It Works
33+
### NIST AI RMF Function Mapping
5234

53-
1. **Define a policy** via API or UI (`/policies`)
54-
2. **Policy engine evaluates** on compliance status changes
55-
3. **Violations are blocked** with clear error messages
56-
4. **PolicyViolation record** created with full details
57-
5. **Audit log entry** captures the blocked action
35+
| NIST Function | Platform Capability |
36+
|---------------|---------------------|
37+
| **GOVERN** | RBAC, policies, organization scoping, audit logs |
38+
| **MAP** | Model registry, risk profiles, data classification, lineage |
39+
| **MEASURE** | Evaluation metrics, version tracking, performance history |
40+
| **MANAGE** | Compliance lifecycle, policy enforcement, human approvals |
5841

59-
### Example: Create a Policy
42+
### Sectoral Applicability
6043

61-
```bash
62-
curl -X POST "http://localhost:8000/api/v1/policies/" \
63-
-H "Content-Type: application/json" \
64-
-d '{
65-
"name": "Require Evaluation Before Approval",
66-
"description": "Models must have metrics before being approved",
67-
"scope": "global",
68-
"condition_type": "require_evaluation_before_approval",
69-
"is_active": true
70-
}'
71-
```
44+
| Regulation | Relevant Features |
45+
|------------|-------------------|
46+
| **HIPAA** | `data_sensitivity: phi`, audit logging |
47+
| **GLBA/FFIEC** | Risk profiles, data classification, approval workflows |
48+
| **CCPA/CPRA** | PII tracking, data sources documentation |
49+
| **FedRAMP** | Organization scoping, audit trails, security controls |
7250

7351
---
7452

75-
## Organizations & Environments
53+
## Data Classification & Sensitivity
7654

77-
### Conceptual Model
55+
### Sensitivity Levels
7856

79-
```
80-
Organization (e.g., "Agency A", "Department B")
81-
└── Environment (dev, test, staging, prod)
82-
└── Models (scoped by org + env)
83-
```
57+
| Level | Description | Example Use Case |
58+
|-------|-------------|------------------|
59+
| `public` | Non-sensitive data | Public datasets |
60+
| `internal` | Internal business data | Operational metrics |
61+
| `pii` | Personally Identifiable Information | Customer names, emails |
62+
| `phi` | Protected Health Information (HIPAA) | Medical records |
63+
| `pci` | Payment Card Industry data | Credit card numbers |
8464

85-
### How to Use
65+
### Classification Levels
8666

87-
- **Models** include `organization_id` and `environment` fields
88-
- **API queries** can be filtered by organization/environment
89-
- **Users** belong to organizations (planned: multi-org access)
67+
| Level | Description |
68+
|-------|-------------|
69+
| `public` | Open to external parties |
70+
| `internal` | Internal use only |
71+
| `confidential` | Restricted access |
72+
| `restricted` | Highly restricted (need-to-know) |
9073

91-
### Enterprise Mapping
74+
### Jurisdiction
9275

93-
| Enterprise Concept | Platform Feature |
94-
|-------------------|------------------|
95-
| Department/Agency | Organization |
96-
| SDLC Stage | Environment (dev/test/prod) |
97-
| Data Classification | Risk Level |
76+
Track data residency requirements with the `jurisdiction` field (e.g., "US", "EU", "Global").
9877

9978
---
10079

101-
## Observability
80+
## Lineage & Traceability
10281

103-
### Prometheus Metrics
82+
### Why Lineage Matters
10483

105-
Access metrics at: `GET /api/v1/metrics`
84+
- **Audit compliance**: Know exactly what data trained your models
85+
- **Incident response**: Quickly identify affected models when data issues arise
86+
- **Reproducibility**: Track model dependencies for retraining
10687

107-
```prometheus
108-
# HELP ai_governance_models_total Total number of registered AI models
109-
ai_governance_models_total 42
88+
### Data Model
11089

111-
# HELP ai_governance_violations_total Total policy violations
112-
ai_governance_violations_total 3
113-
114-
# Models by risk level
115-
ai_governance_models_by_risk{risk_level="high"} 5
116-
ai_governance_models_by_risk{risk_level="minimal"} 20
90+
```
91+
Dataset (training data, validation data, etc.)
92+
↓ linked via ModelDatasetLink
93+
ModelRegistry (your AI model)
94+
↓ linked via ModelDependency
95+
ModelRegistry (parent models, fine-tuning sources)
11796
```
11897

119-
### Grafana Integration
120-
121-
1. Add Prometheus data source pointing to your Prometheus server
122-
2. Create dashboard with key metrics:
123-
- Model counts by risk level
124-
- Policy violations over time
125-
- Compliance status distribution
126-
127-
---
128-
129-
## Security & Deployment Considerations
130-
131-
### Network Placement
98+
### Example: Link a Dataset
13299

133-
> [!IMPORTANT]
134-
> This application should be deployed **behind a reverse proxy** (Nginx, Envoy, API Gateway) and not directly exposed to the internet.
100+
```bash
101+
# Create a dataset
102+
curl -X POST "http://localhost:8000/api/v1/datasets/" \
103+
-H "Content-Type: application/json" \
104+
-d '{
105+
"name": "Customer Transactions Q4",
106+
"source_system": "Snowflake",
107+
"data_sensitivity": "pii",
108+
"data_classification": "confidential"
109+
}'
135110

136-
**Recommended topology:**
137-
```
138-
Internet → Load Balancer → Nginx/Envoy (TLS) → AI Governance Hub → PostgreSQL
111+
# Link to a model
112+
curl -X POST "http://localhost:8000/api/v1/models/1/datasets/" \
113+
-H "Content-Type: application/json" \
114+
-d '{"dataset_id": 1, "dataset_type": "training"}'
139115
```
140116

141-
### TLS/HTTPS
117+
---
142118

143-
- Configure TLS termination at the reverse proxy
144-
- Use Let's Encrypt or organizational certificates
145-
- Enforce HTTPS redirects
119+
## Human-in-the-Loop Approvals
146120

147-
### Secrets Management
121+
When a model is approved, the system captures:
148122

149-
| Secret | Source |
150-
|--------|--------|
151-
| `DATABASE_URL` | Environment variable |
152-
| `SECRET_KEY` | Environment variable (min 32 chars) |
153-
| `POSTGRES_PASSWORD` | Environment variable or secrets manager |
123+
| Field | Description |
124+
|-------|-------------|
125+
| `approved_by_user_id` | Who approved the model |
126+
| `approved_at` | Timestamp of approval |
127+
| `approval_notes` | Required justification |
154128

155-
> [!CAUTION]
156-
> Never commit secrets to version control. Use `.env` files for local development only.
129+
**Approval notes are mandatory** when changing status to `approved`.
157130

158-
### Rate Limiting
131+
---
159132

160-
Rate limiting should be configured at the reverse proxy level:
133+
## SSO / Identity Provider Integration (Planned)
161134

162-
```nginx
163-
# Example Nginx rate limiting
164-
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
165-
location /api/ {
166-
limit_req zone=api burst=20 nodelay;
167-
}
168-
```
135+
### Recommended Patterns
136+
137+
1. **Reverse proxy + headers**: Deploy behind Nginx/Envoy with IdP, pass user info via headers
138+
2. **App-native OIDC**: Integrate directly with Okta, Azure AD, or Keycloak
169139

170-
### Logging & SIEM Integration
140+
### IdP Role Mapping
171141

172-
- All governance actions are logged to `ComplianceLog` table
173-
- Logs include: entity, action, user, timestamp, details (JSON)
174-
- Export logs to SIEM via database replication or API polling
142+
| IdP Group | Maps to Role |
143+
|-----------|--------------|
144+
| `ai-admins` | `admin` |
145+
| `ml-engineers` | `model_owner` |
146+
| `compliance-team` | `auditor` |
175147

176148
---
177149

178-
## Backup & Data Export
150+
## Security & Deployment
179151

180-
### PostgreSQL Backup
152+
### Network Placement
181153

182-
```bash
183-
# Full backup
184-
pg_dump -h localhost -U postgres ai_governance > backup.sql
154+
> [!IMPORTANT]
155+
> Deploy behind a reverse proxy with TLS termination.
185156
186-
# Restore
187-
psql -h localhost -U postgres ai_governance < backup.sql
188-
```
157+
### Secrets Management
189158

190-
### API Export
159+
| Secret | Source |
160+
|--------|--------|
161+
| `DATABASE_URL` | Environment variable |
162+
| `SECRET_KEY` | Environment variable (min 32 chars) |
191163

192-
- `GET /api/v1/models/` - Export all models
193-
- `GET /api/v1/audit-logs/` - Export audit trail
194-
- `GET /api/v1/policies/violations/` - Export policy violations
164+
### Observability
165+
166+
- Prometheus metrics: `GET /api/v1/metrics`
167+
- Structured audit logs in `ComplianceLog` table
195168

196169
---
197170

@@ -207,8 +180,6 @@ docker compose up --build
207180
| Service | URL |
208181
|---------|-----|
209182
| Frontend | http://localhost:3000 |
210-
| Dashboard | http://localhost:3000/dashboard |
211-
| Policies | http://localhost:3000/policies |
212183
| API Docs | http://localhost:8000/docs |
213184
| Metrics | http://localhost:8000/api/v1/metrics |
214185

@@ -218,32 +189,30 @@ docker compose up --build
218189

219190
| Status | Feature |
220191
|--------|---------|
221-
| ✅ Done | Policy Engine with enforcement |
222-
| ✅ Done | Organization + environment scoping |
223-
| ✅ Done | Prometheus metrics endpoint |
224-
| 🔜 Planned | SSO/SAML integration |
225-
| 🔜 Planned | Webhooks for status changes |
226-
| 🔜 Planned | MLflow integration |
227-
| 🔜 Planned | Kubernetes operator |
192+
| ✅ Done | Policy Engine |
193+
| ✅ Done | Multi-tenancy |
194+
| ✅ Done | Data Classification (v0.4) |
195+
| ✅ Done | Lineage Tracking (v0.4) |
196+
| ✅ Done | Human Approvals (v0.4) |
197+
| 🔜 | SSO/OIDC integration |
198+
| 🔜 | MLflow integration |
199+
| 🔜 | Advanced policy DSL |
200+
| 🔜 | Model lineage visualization |
228201

229202
---
230203

231204
## Technology Stack
232205

233206
| Layer | Technology |
234207
|-------|------------|
235-
| Frontend | React, TypeScript, Vite, Mantine UI |
236-
| Backend | Python 3.11, FastAPI, SQLModel, Pydantic |
208+
| Frontend | React, TypeScript, Mantine |
209+
| Backend | Python 3.11, FastAPI, SQLModel |
237210
| Database | PostgreSQL 15 |
238-
| Auth | OAuth2, JWT, bcrypt, RBAC |
239-
| Observability | Prometheus-compatible metrics |
240-
| Infrastructure | Docker, Docker Compose, Nginx |
241-
| CI/CD | GitHub Actions |
211+
| Auth | OAuth2, JWT, RBAC |
212+
| Infrastructure | Docker, Nginx |
242213

243214
---
244215

245216
## License
246217

247-
Licensed under **Apache License 2.0** — enterprise-friendly, permits commercial use.
248-
249-
See [LICENSE](LICENSE) for details.
218+
Apache License 2.0 — See [LICENSE](LICENSE).

0 commit comments

Comments
 (0)