Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions db/README_TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Database Testing Guide

## Overview

Unit tests for the database layer use an **in-memory SQLite database** for fast, isolated testing without requiring external dependencies.

## Approach

### In-Memory SQLite Database
- Tests create a fresh `:memory:` SQLite database for each test
- Schema is migrated using GORM's AutoMigrate
- No cleanup required - database is garbage collected after test completion

### Test Structure

1. **`setupTestDB(t)`** - Creates and migrates a clean in-memory database
2. **`seedTestData(t, db)`** - Populates the database with test fixtures
3. **Individual test functions** - Test specific functionality

## Example: TestGetMaintainersByProject

```go
func TestGetMaintainersByProject(t *testing.T) {
db := setupTestDB(t)
company, project1, project2, m1, m2, m3 := seedTestData(t, db)
store := NewSQLStore(db)

// Test cases...
}
```

### Test Fixtures

The `seedTestData` function creates:
- 1 company ("Test Company")
- 2 projects (kubernetes, prometheus)
- 3 maintainers (Alice, Bob, Charlie)
- Associations:
- kubernetes → Alice, Bob
- prometheus → Bob, Charlie

### What Gets Tested

1. ✅ Returns correct maintainers for a project
2. ✅ Company relationships are preloaded
3. ✅ Empty results for projects with no maintainers
4. ✅ Empty results for non-existent projects
5. ✅ All maintainer fields are populated correctly
6. ✅ Projects field is NOT preloaded (as expected)

## Running Tests

```bash
# Run all db tests
go test ./db

# Run specific test
go test ./db -run TestGetMaintainersByProject

# Run with verbose output
go test -v ./db

# Run with coverage
go test -cover ./db
```

## Benefits of This Approach

1. **Fast** - In-memory database is extremely fast
2. **Isolated** - Each test gets a fresh database
3. **No dependencies** - No need for Docker, external DB, or test infrastructure
4. **Deterministic** - Tests always start with the same state
5. **Parallel-safe** - Each test has its own database instance

## Adding New Tests

To add a new test:

1. Use `setupTestDB(t)` to get a fresh database
2. Create your own fixtures or use `seedTestData(t, db)` if appropriate
3. Test your function
4. Assert expected behavior

Example:
```go
func TestYourFunction(t *testing.T) {
db := setupTestDB(t)
// Create custom test data if needed
project := model.Project{Name: "test"}
require.NoError(t, db.Create(&project).Error)

store := NewSQLStore(db)
result, err := store.YourFunction(project.ID)

require.NoError(t, err)
assert.Equal(t, expectedValue, result)
}
```
237 changes: 237 additions & 0 deletions deploy/README.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
# Deployment notes

MaintainerD - input data (List of Projects and associated Maintainers)

## Kubernetes resources overview
```mermaid
graph RL
subgraph maintainerd["Namespace: maintainerd"]
subgraph pod["Pod"]
subgraph initContainerBootstrap["initContainer: bootstrap"]
bootstrap["Loads Project → Maintainer data <br>into sqlite3 database<br>from CNCF-Internal Google Sheet"]
end
subgraph maintainerdServer["maintainerd-server"]
server["<br>GitHub WebHooks<br>labels can start on-boarding processes<br>issue comments can progress workflows<br>/healthz establishes availability by maintainerd-service."]
end
end

pvc[(PersistentVolumeClaim<br/>sqlite3 maintainerd-db)]
svc{{Service<br/>maintainerd-service<br/>type=LoadBalancer}}
svc -->|"ports 80/443"| ext[(External LB IP<br/>github-events.cncf.io/webhook)]
end

bootstrap -->|"SQL INSERTs into PROJECT and MAINTAINERS tables"| pvc
```


deploy -->|creates| pod
deploy -->|mounts| pvc
deploy -->|envFrom| secrets
deploy -->|imagePullSecrets| ghcr
pod -->|exposes 2525| svc
svc -->|TLS| tls
pod -->|volumeMount| pvc




## Maintainer Database ER diagram

```mermaid
erDiagram
companies {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
TEXT name
}

projects {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
TEXT name
INTEGER parent_project_id "FK to projects.id"
TEXT maturity
TEXT maintainer_ref
TEXT onboarding_issue
TEXT mailing_list
}

services {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
TEXT name
TEXT description
}

service_projects {
INTEGER project_id PK "FK to projects.id"
INTEGER service_id PK "FK to services.id"
}

maintainers {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
TEXT name
TEXT email
TEXT git_hub_account
TEXT git_hub_email
TEXT maintainer_status
TEXT import_warnings
DATETIME registered_at
INTEGER company_id "FK to companies.id"
}

maintainer_projects {
INTEGER maintainer_id PK "FK to maintainers.id"
INTEGER project_id PK "FK to projects.id"
DATETIME joined_at
}

collaborators {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
TEXT name
TEXT email
TEXT git_hub_email
TEXT git_hub_account
DATETIME last_login
DATETIME registered_at
}

service_teams {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
INTEGER project_id "FK to projects.id"
INTEGER service_id "FK to services.id"
TEXT service_team_name
TEXT project_name
}

service_users {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
INTEGER service_id "FK to services.id"
INTEGER service_user_id
TEXT service_email
TEXT service_ref
TEXT service_git_hub_name
}

service_user_teams {
INTEGER id PK
DATETIME created_at
DATETIME updated_at
DATETIME deleted_at
INTEGER service_id "FK to services.id"
INTEGER service_user_id "FK to service_users.id"
INTEGER service_team_id "FK to service_teams.id"
INTEGER maintainer_id "FK to maintainers.id"
INTEGER collaborator_id "FK to collaborators.id"
}

companies ||--o{ maintainers : employs
projects ||--o{ projects : parent_of
projects ||--o{ service_projects : includes
services ||--o{ service_projects : provides
maintainers ||--o{ maintainer_projects : assigned_to
projects ||--o{ maintainer_projects : hosts
service_teams ||--o{ service_user_teams : has
services ||--o{ service_users : has
```



## init-container
- bootstrap process loads project and maintainer data from
- a CNCF-Internal worksheet
- in future, should be loaded from a combination of PCC user profiles (keyed by GitHub user account) and a registered list of project.yaml files on a per-project basis.

## GitHub Event Listener

### cert-manager
Manages the server cert associated with the maintainer-d event listener that listens for changes to onboarding issues.

Steps to integrate with Let's Encrypt on OKE:

1. Create the OCI DNS credentials secret:
```bash
kubectl create secret generic oci-dns-credentials -n cert-manager \
--from-literal=tenancyOCID=<tenancy_ocid> \
--from-literal=userOCID=<user_ocid> \
--from-literal=fingerprint=<api_key_fingerprint> \
--from-file=privateKey=<path_to_api_key_private_key_pem>
```
2. Apply a `ClusterIssuer` using the OCI DNS solver (update compartment OCID, DNS zone, and email):
```yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns
spec:
acme:
email: [email protected]
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-dns-account-key
solvers:
- dns01:
oci:
compartmentOCID: ocid1.compartment.oc1..aaaaaaaa22icap66vxktktubjlhf6oxvfhev6n7udgje2chahyrtq65ga63a
dnsZoneName: cncf.io.
secretRef:
name: oci-dns-credentials
key: privateKey
tenancyOCID: ocid1.tenancy.oc1... # must match the secret
userOCID: ocid1.user.oc1... # must match the secret
fingerprint: <api_key_fingerprint>
```
3. Request the certificate in the `maintainerd` namespace:
```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: maintainerd-cert
namespace: maintainerd
spec:
secretName: maintainerd-tls
issuerRef:
name: letsencrypt-dns
kind: ClusterIssuer
dnsNames:
- github-events.cncf.io
```
4. Update `deploy/manifests/service.yaml` to expose HTTPS and reference the secret:
```yaml
metadata:
annotations:
service.beta.kubernetes.io/oci-load-balancer-ssl-ports: https
service.beta.kubernetes.io/oci-load-balancer-tls-secret: maintainerd/maintainerd-tls
spec:
ports:
- name: http
port: 80
targetPort: 2525
- name: https
port: 443
targetPort: 2525
```
5. Apply the manifests (`kubectl apply -f deploy/manifests/service.yaml`) and verify:
- `kubectl describe certificate maintainerd-cert -n maintainerd`
- `kubectl get secret maintainerd-tls -n maintainerd`
- `curl -vk https://github-events.cncf.io/healthz`

Ensure your OCI IAM policies allow the OKE dynamic group to manage DNS records in the relevant compartment; otherwise the ACME solver cannot create the TXT challenges.
43 changes: 43 additions & 0 deletions deploy/cert-ops.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Certificate Maintenance for maintainer-d

maintainer-d uses a manually issued Let’s Encrypt certificate stored in the `maintainerd-tls` secret.

Repeat these steps before the cert expires (every ~60–90 days):

1. **Request a new certificate via certbot**
```bash
sudo certbot certonly \
--manual \
--preferred-challenges dns \
--key-type rsa \
-d github-events.cncf.io \
--email <EMAIL_ADDRESS> \
--agree-tos
```
Certbot prints a `_acme-challenge.github-events.cncf.io` TXT record. Ask the DNSimple admin to create it (TTL 60). Press Enter once DNS propagates.

2. **Load the cert into Kubernetes**
```bash
kubectl create secret tls maintainerd-tls \
--cert=/etc/letsencrypt/live/github-events.cncf.io/fullchain.pem \
--key=/etc/letsencrypt/live/github-events.cncf.io/privkey.pem \
-n maintainerd \
--dry-run=client -o yaml | kubectl apply -f -
```

3. **Recreate the Service so OCI reloads the cert**
```bash
kubectl delete svc maintainerd -n maintainerd
kubectl apply -f deploy/manifests/service.yaml
kubectl get svc maintainerd -n maintainerd --watch
```
Wait until `EXTERNAL-IP` shows `170.9.21.206` again.

4. **Verify**
```bash
kubectl describe svc maintainerd -n maintainerd
curl -vk https://github-events.cncf.io/healthz
openssl s_client -connect github-events.cncf.io:443 -servername github-events.cncf.io -tls1_2
```

Keep `/etc/letsencrypt` backed up or document the certbot host. If you ever automate DNS updates, you can replace the manual step with cert-manager and remove the monthly coordination with DNSimple.
Loading