Skip to content

Commit f18763c

Browse files
TamTunnelTamTunnel
committed
feat: Add risk profiles, compliance lifecycle, RBAC, dashboard, CI integration docs
- Add RiskLevel enum (minimal, limited, high, unacceptable) - Add ComplianceStatus lifecycle (draft, under_review, approved, retired) - Add risk_level, domain, potential_harm, intended_purpose, data_sources, oversight_plan to models - Add compliance status change endpoints with audit logging - Add dashboard stats API endpoint - Add UserRole enum (admin, model_owner, auditor) with RBAC - Create ComplianceDashboard React component - Update ModelList with risk level and status badges - Add examples/ci-integration.yml GitHub Actions workflow - Comprehensive README with Who is this for, Usage Scenarios, RBAC docs, EU AI Act mapping, Roadmap
1 parent 5dd9f4b commit f18763c

File tree

11 files changed

+705
-113
lines changed

11 files changed

+705
-113
lines changed

‎README.md‎

Lines changed: 140 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -5,28 +5,83 @@
55

66
## What Is This?
77

8-
A **centralized platform** for managing your organization's AI models—a registry that tracks every AI system, its versions, performance metrics, and a complete audit trail. It helps answer:
8+
A **centralized platform** for managing your organization's AI models—a registry that tracks every AI system, its versions, performance metrics, risk profiles, and a complete audit trail for EU AI Act compliance.
99

10-
- *"What AI models are we running in production?"*
11-
- *"Who owns this model? When was it last updated?"*
12-
- *"Can we prove compliance for regulatory audits (EU AI Act)?"*
10+
---
11+
12+
## Who Is This For?
13+
14+
| Role | Value |
15+
|------|-------|
16+
| **ML Engineers** | Register models, track versions, store evaluation metrics automatically via CI/CD |
17+
| **Compliance Officers** | View audit trails, approve compliance status, generate PDF reports for regulators |
18+
| **CTOs/Engineering Leaders** | Dashboard overview of model risk levels and compliance status across the organization |
19+
| **Auditors** | Read-only access to model registry, versions, metrics, and audit logs |
20+
21+
---
22+
23+
## Usage Scenarios
24+
25+
### Scenario 1: New Model Deployment
26+
1. ML team trains a new fraud detection model
27+
2. CI/CD pipeline automatically registers the model and pushes evaluation metrics
28+
3. Model starts in `draft` status with `high` risk level (finance domain)
29+
4. Compliance team reviews via `/dashboard`, updates to `under_review`
30+
5. After approval, status changes to `approved` with full audit trail
31+
32+
### Scenario 2: EU AI Act Audit
33+
1. Regulator requests documentation for high-risk AI systems
34+
2. Compliance officer filters models by `risk_level=high`
35+
3. Downloads PDF compliance report for each model
36+
4. Report includes: intended purpose, data sources, evaluation metrics, oversight plan
37+
38+
### Scenario 3: Model Retirement
39+
1. Old recommendation model needs to be retired
40+
2. Admin changes compliance status to `retired` with reason
41+
3. Audit log captures the change for future reference
42+
4. Model remains in registry for historical records
1343

1444
---
1545

1646
## Key Features
1747

1848
| Feature | Description |
1949
|---------|-------------|
20-
| **Model Registry** | Register and catalog AI models with name, owner, and description |
21-
| **Version Tracking** | Track model versions (v1.0, v2.1) and artifact storage locations (S3) |
22-
| **Evaluation Metrics** | Store accuracy, F1 score, bias metrics per version |
23-
| **Audit Logging** | Automatic immutable compliance trail for all create/update actions |
24-
| **API Schemas (DTOs)** | Pydantic validation schemas separate from database models |
25-
| **OAuth2 Authentication** | JWT-based auth with user registration and login |
26-
| **PDF Compliance Reports** | Generate EU AI Act style compliance reports for any model |
27-
| **Health Monitoring** | `/health` endpoint for load balancers and Kubernetes probes |
28-
| **Secrets Management** | Environment-based configuration, no hardcoded credentials |
29-
| **CI/CD Pipeline** | GitHub Actions for linting, testing, building, and deployment |
50+
| **Model Registry** | Register AI models with name, owner, description |
51+
| **Risk Profiles** | Classify models by EU AI Act risk levels (minimal, limited, high, unacceptable) |
52+
| **Compliance Lifecycle** | Track status: draft → under_review → approved → retired |
53+
| **Version Tracking** | Track model versions and artifact locations (S3) |
54+
| **Evaluation Metrics** | Store accuracy, F1, bias scores per version |
55+
| **Audit Logging** | Automatic immutable trail for all changes |
56+
| **Compliance Dashboard** | Visual overview of models by risk level and status |
57+
| **PDF Reports** | Generate EU AI Act style compliance documentation |
58+
| **Role-Based Access** | admin, model_owner, auditor roles |
59+
| **OAuth2 Auth** | JWT authentication |
60+
| **CI/CD Integration** | GitHub Actions workflow examples |
61+
62+
---
63+
64+
## Roles & Permissions (RBAC)
65+
66+
| Role | Permissions |
67+
|------|-------------|
68+
| `admin` | Full access - create, modify, delete models and users |
69+
| `model_owner` | Create/modify models, change compliance status |
70+
| `auditor` | Read-only - view models, audit logs, download reports |
71+
72+
---
73+
74+
## EU AI Act Feature Mapping
75+
76+
| EU AI Act Requirement | Platform Feature |
77+
|----------------------|------------------|
78+
| Risk Classification | `risk_level` field (minimal, limited, high, unacceptable) |
79+
| Intended Purpose Documentation | `intended_purpose` field |
80+
| Data Sources Transparency | `data_sources` field |
81+
| Performance Metrics | Evaluation metrics per version |
82+
| Human Oversight Plan | `oversight_plan` field |
83+
| Change Audit Trail | Automatic compliance logs |
84+
| Lifecycle Management | `compliance_status` (draft → approved → retired) |
3085

3186
---
3287

@@ -35,21 +90,21 @@ A **centralized platform** for managing your organization's AI models—a regist
3590
```
3691
┌─────────────────────────────────────────────────────────────┐
3792
│ FRONTEND (React) │
38-
│ • Model List Dashboard • Register New Model Form │
39-
│ • Version & Metrics View • Mantine UI Components │
93+
│ • Model Registry List • Compliance Dashboard │
94+
│ • Risk Level Badges • Status Filters │
4095
└─────────────────────────────────────────────────────────────┘
4196
│ REST API
4297
▼
4398
┌─────────────────────────────────────────────────────────────┐
4499
│ BACKEND (FastAPI) │
45-
│ /api/v1/models → Model registry CRUD │
46-
│ /api/v1/versions → Version management │
47-
│ /api/v1/metrics → Evaluation data storage │
48-
│ /api/v1/audit-logs → Compliance audit history │
49-
│ /api/v1/auth/register → User registration │
50-
│ /api/v1/auth/token → JWT login │
100+
│ /api/v1/models → Model registry CRUD │
101+
│ /api/v1/models/{id}/risk-profile → Update risk profile │
102+
│ /api/v1/models/{id}/compliance-status → Change status │
103+
│ /api/v1/dashboard/stats → Dashboard statistics │
104+
│ /api/v1/versions → Version management │
105+
│ /api/v1/metrics → Evaluation metrics │
106+
│ /api/v1/audit-logs → Compliance history │
51107
│ /api/v1/reports/{id}/compliance-report → PDF download │
52-
│ /api/v1/health → System health check │
53108
└─────────────────────────────────────────────────────────────┘
54109
│
55110
▼
@@ -70,49 +125,80 @@ A **centralized platform** for managing your organization's AI models—a regist
70125

71126
### Run with Docker
72127
```bash
73-
# Clone the repo
74128
git clone https://github.com/TamTunnel/AI-Governance-Hub.git
75129
cd AI-Governance-Hub
76-
77-
# Copy environment template
78130
cp .env.example .env
79-
80-
# Start all services
81131
docker compose up --build
82132
```
83133

84134
**URLs:**
85135
| Service | URL |
86136
|---------|-----|
87137
| Frontend | http://localhost:3000 |
88-
| API Docs (Swagger) | http://localhost:8000/docs |
89-
| Health Check | http://localhost:8000/api/v1/health |
138+
| Dashboard | http://localhost:3000/dashboard |
139+
| API Docs | http://localhost:8000/docs |
90140

91-
### Development Mode
92-
```bash
93-
# Backend
94-
cd backend && poetry install && poetry run uvicorn app.main:app --reload
141+
---
95142

96-
# Frontend (separate terminal)
97-
cd frontend && npm install && npm run dev
143+
## CI Integration
144+
145+
### Example: Register model after training
146+
147+
```bash
148+
# Register a new model
149+
curl -X POST "http://localhost:8000/api/v1/models/" \
150+
-H "Content-Type: application/json" \
151+
-d '{
152+
"name": "fraud-detector-v2",
153+
"owner": "ML Team",
154+
"risk_level": "high",
155+
"domain": "finance",
156+
"intended_purpose": "Detect fraudulent transactions"
157+
}'
158+
159+
# Create a version
160+
curl -X POST "http://localhost:8000/api/v1/versions/" \
161+
-H "Content-Type: application/json" \
162+
-d '{
163+
"model_id": 1,
164+
"version_tag": "v1.0.0",
165+
"s3_path": "s3://models/fraud-detector/v1.0.0"
166+
}'
167+
168+
# Push evaluation metric
169+
curl -X POST "http://localhost:8000/api/v1/metrics/" \
170+
-H "Content-Type: application/json" \
171+
-d '{
172+
"version_id": 1,
173+
"metric_name": "accuracy",
174+
"value": 0.95
175+
}'
176+
177+
# Update compliance status
178+
curl -X PATCH "http://localhost:8000/api/v1/models/1/compliance-status" \
179+
-H "Content-Type: application/json" \
180+
-d '{
181+
"status": "under_review",
182+
"reason": "Ready for compliance review"
183+
}'
98184
```
99185

186+
See [`examples/ci-integration.yml`](examples/ci-integration.yml) for a complete GitHub Actions workflow.
187+
100188
---
101189

102-
## API Endpoints
103-
104-
| Method | Endpoint | Description |
105-
|--------|----------|-------------|
106-
| `POST` | `/api/v1/auth/register` | Create user account |
107-
| `POST` | `/api/v1/auth/token` | Login, get JWT token |
108-
| `POST` | `/api/v1/models/` | Register new AI model |
109-
| `GET` | `/api/v1/models/` | List all models |
110-
| `GET` | `/api/v1/models/{id}` | Get model details |
111-
| `POST` | `/api/v1/versions/` | Add model version |
112-
| `POST` | `/api/v1/metrics/` | Add evaluation metric |
113-
| `GET` | `/api/v1/audit-logs/` | View compliance audit trail |
114-
| `GET` | `/api/v1/reports/models/{id}/compliance-report` | Download PDF report |
115-
| `GET` | `/api/v1/health` | System health status |
190+
## Roadmap
191+
192+
### Planned Features
193+
194+
| Priority | Feature | Description |
195+
|----------|---------|-------------|
196+
| 🔴 High | **Policy Engine** | Define and enforce compliance rules automatically |
197+
| 🔴 High | **Model Lineage** | Track data and model dependencies |
198+
| 🟡 Medium | **Notifications** | Webhooks and email alerts for status changes |
199+
| 🟡 Medium | **MLflow Integration** | Import models directly from MLflow |
200+
| 🟢 Future | **Kubernetes Operator** | Auto-register models deployed to K8s |
201+
| 🟢 Future | **LLM Governance** | Prompt tracking and response auditing |
116202

117203
---
118204

@@ -123,45 +209,15 @@ cd frontend && npm install && npm run dev
123209
| Frontend | React, TypeScript, Vite, Mantine UI |
124210
| Backend | Python 3.11, FastAPI, SQLModel, Pydantic |
125211
| Database | PostgreSQL 15 |
126-
| Auth | OAuth2, JWT (python-jose), bcrypt |
127-
| Reports | ReportLab (PDF generation) |
212+
| Auth | OAuth2, JWT, bcrypt, RBAC |
213+
| Reports | ReportLab (PDF) |
128214
| Infrastructure | Docker, Docker Compose, Nginx |
129215
| CI/CD | GitHub Actions |
130216

131217
---
132218

133-
## Configuration
134-
135-
Copy `.env.example` to `.env` and configure:
136-
137-
```bash
138-
# Database
139-
POSTGRES_USER=postgres
140-
POSTGRES_PASSWORD=your_secure_password
141-
DATABASE_URL=postgresql://postgres:password@db:5432/ai_governance
142-
143-
# Authentication
144-
SECRET_KEY=your-secret-key-minimum-32-characters
145-
146-
# Frontend
147-
VITE_API_URL=http://localhost:8000/api/v1
148-
```
149-
150-
---
151-
152-
## CI/CD Pipeline
153-
154-
The GitHub Actions workflow (`.github/workflows/ci.yml`) includes:
155-
156-
1. **Backend Tests** - Runs pytest with PostgreSQL service
157-
2. **Frontend Build** - Lints and builds the React app
158-
3. **Docker Build** - Builds production images
159-
4. **Deploy** - Placeholder for staging/production deployment
160-
161-
---
162-
163219
## License
164220

165-
This project is licensed under the **Apache License 2.0** — an enterprise-friendly open-source license that permits commercial use, modification, and distribution.
221+
Licensed under **Apache License 2.0** — enterprise-friendly, permits commercial use.
166222

167-
See [LICENSE](LICENSE) for full details.
223+
See [LICENSE](LICENSE) for details.

0 commit comments

Comments
 (0)