Skip to content

Commit ae9455b

Browse files
RobertKieltyRobertKielty
authored
streamlines FOSSA invite reporting (#32)
- aggregates maintainer invite/pending/already-member output and guard FOSSA team creation failures - makes issue comments more concise especially if workflow is run twice - teach the mock client/tests about team creation errors - tack on the latest README updates for db, deploy, github-events-service, and fossa user invites
2 parents fa0d1f5 + 1452736 commit ae9455b

File tree

8 files changed

+514
-27
lines changed

8 files changed

+514
-27
lines changed

‎db/README_TESTING.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Database Testing Guide
2+
3+
## Overview
4+
5+
Unit tests for the database layer use an **in-memory SQLite database** for fast, isolated testing without requiring external dependencies.
6+
7+
## Approach
8+
9+
### In-Memory SQLite Database
10+
- Tests create a fresh `:memory:` SQLite database for each test
11+
- Schema is migrated using GORM's AutoMigrate
12+
- No cleanup required - database is garbage collected after test completion
13+
14+
### Test Structure
15+
16+
1. **`setupTestDB(t)`** - Creates and migrates a clean in-memory database
17+
2. **`seedTestData(t, db)`** - Populates the database with test fixtures
18+
3. **Individual test functions** - Test specific functionality
19+
20+
## Example: TestGetMaintainersByProject
21+
22+
```go
23+
func TestGetMaintainersByProject(t *testing.T) {
24+
db := setupTestDB(t)
25+
company, project1, project2, m1, m2, m3 := seedTestData(t, db)
26+
store := NewSQLStore(db)
27+
28+
// Test cases...
29+
}
30+
```
31+
32+
### Test Fixtures
33+
34+
The `seedTestData` function creates:
35+
- 1 company ("Test Company")
36+
- 2 projects (kubernetes, prometheus)
37+
- 3 maintainers (Alice, Bob, Charlie)
38+
- Associations:
39+
- kubernetes → Alice, Bob
40+
- prometheus → Bob, Charlie
41+
42+
### What Gets Tested
43+
44+
1. ✅ Returns correct maintainers for a project
45+
2. ✅ Company relationships are preloaded
46+
3. ✅ Empty results for projects with no maintainers
47+
4. ✅ Empty results for non-existent projects
48+
5. ✅ All maintainer fields are populated correctly
49+
6. ✅ Projects field is NOT preloaded (as expected)
50+
51+
## Running Tests
52+
53+
```bash
54+
# Run all db tests
55+
go test ./db
56+
57+
# Run specific test
58+
go test ./db -run TestGetMaintainersByProject
59+
60+
# Run with verbose output
61+
go test -v ./db
62+
63+
# Run with coverage
64+
go test -cover ./db
65+
```
66+
67+
## Benefits of This Approach
68+
69+
1. **Fast** - In-memory database is extremely fast
70+
2. **Isolated** - Each test gets a fresh database
71+
3. **No dependencies** - No need for Docker, external DB, or test infrastructure
72+
4. **Deterministic** - Tests always start with the same state
73+
5. **Parallel-safe** - Each test has its own database instance
74+
75+
## Adding New Tests
76+
77+
To add a new test:
78+
79+
1. Use `setupTestDB(t)` to get a fresh database
80+
2. Create your own fixtures or use `seedTestData(t, db)` if appropriate
81+
3. Test your function
82+
4. Assert expected behavior
83+
84+
Example:
85+
```go
86+
func TestYourFunction(t *testing.T) {
87+
db := setupTestDB(t)
88+
// Create custom test data if needed
89+
project := model.Project{Name: "test"}
90+
require.NoError(t, db.Create(&project).Error)
91+
92+
store := NewSQLStore(db)
93+
result, err := store.YourFunction(project.ID)
94+
95+
require.NoError(t, err)
96+
assert.Equal(t, expectedValue, result)
97+
}
98+
```

‎deploy/README.MD‎

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# Deployment notes
2+
3+
MaintainerD - input data (List of Projects and associated Maintainers)
4+
5+
## Kubernetes resources overview
6+
```mermaid
7+
graph RL
8+
subgraph maintainerd["Namespace: maintainerd"]
9+
subgraph pod["Pod"]
10+
subgraph initContainerBootstrap["initContainer: bootstrap"]
11+
bootstrap["Loads Project → Maintainer data <br>into sqlite3 database<br>from CNCF-Internal Google Sheet"]
12+
end
13+
subgraph maintainerdServer["maintainerd-server"]
14+
server["<br>GitHub WebHooks<br>labels can start on-boarding processes<br>issue comments can progress workflows<br>/healthz establishes availability by maintainerd-service."]
15+
end
16+
end
17+
18+
pvc[(PersistentVolumeClaim<br/>sqlite3 maintainerd-db)]
19+
svc{{Service<br/>maintainerd-service<br/>type=LoadBalancer}}
20+
svc -->|"ports 80/443"| ext[(External LB IP<br/>github-events.cncf.io/webhook)]
21+
end
22+
23+
bootstrap -->|"SQL INSERTs into PROJECT and MAINTAINERS tables"| pvc
24+
```
25+
26+
27+
deploy -->|creates| pod
28+
deploy -->|mounts| pvc
29+
deploy -->|envFrom| secrets
30+
deploy -->|imagePullSecrets| ghcr
31+
pod -->|exposes 2525| svc
32+
svc -->|TLS| tls
33+
pod -->|volumeMount| pvc
34+
35+
36+
37+
38+
## Maintainer Database ER diagram
39+
40+
```mermaid
41+
erDiagram
42+
companies {
43+
INTEGER id PK
44+
DATETIME created_at
45+
DATETIME updated_at
46+
DATETIME deleted_at
47+
TEXT name
48+
}
49+
50+
projects {
51+
INTEGER id PK
52+
DATETIME created_at
53+
DATETIME updated_at
54+
DATETIME deleted_at
55+
TEXT name
56+
INTEGER parent_project_id "FK to projects.id"
57+
TEXT maturity
58+
TEXT maintainer_ref
59+
TEXT onboarding_issue
60+
TEXT mailing_list
61+
}
62+
63+
services {
64+
INTEGER id PK
65+
DATETIME created_at
66+
DATETIME updated_at
67+
DATETIME deleted_at
68+
TEXT name
69+
TEXT description
70+
}
71+
72+
service_projects {
73+
INTEGER project_id PK "FK to projects.id"
74+
INTEGER service_id PK "FK to services.id"
75+
}
76+
77+
maintainers {
78+
INTEGER id PK
79+
DATETIME created_at
80+
DATETIME updated_at
81+
DATETIME deleted_at
82+
TEXT name
83+
TEXT email
84+
TEXT git_hub_account
85+
TEXT git_hub_email
86+
TEXT maintainer_status
87+
TEXT import_warnings
88+
DATETIME registered_at
89+
INTEGER company_id "FK to companies.id"
90+
}
91+
92+
maintainer_projects {
93+
INTEGER maintainer_id PK "FK to maintainers.id"
94+
INTEGER project_id PK "FK to projects.id"
95+
DATETIME joined_at
96+
}
97+
98+
collaborators {
99+
INTEGER id PK
100+
DATETIME created_at
101+
DATETIME updated_at
102+
DATETIME deleted_at
103+
TEXT name
104+
TEXT email
105+
TEXT git_hub_email
106+
TEXT git_hub_account
107+
DATETIME last_login
108+
DATETIME registered_at
109+
}
110+
111+
service_teams {
112+
INTEGER id PK
113+
DATETIME created_at
114+
DATETIME updated_at
115+
DATETIME deleted_at
116+
INTEGER project_id "FK to projects.id"
117+
INTEGER service_id "FK to services.id"
118+
TEXT service_team_name
119+
TEXT project_name
120+
}
121+
122+
service_users {
123+
INTEGER id PK
124+
DATETIME created_at
125+
DATETIME updated_at
126+
DATETIME deleted_at
127+
INTEGER service_id "FK to services.id"
128+
INTEGER service_user_id
129+
TEXT service_email
130+
TEXT service_ref
131+
TEXT service_git_hub_name
132+
}
133+
134+
service_user_teams {
135+
INTEGER id PK
136+
DATETIME created_at
137+
DATETIME updated_at
138+
DATETIME deleted_at
139+
INTEGER service_id "FK to services.id"
140+
INTEGER service_user_id "FK to service_users.id"
141+
INTEGER service_team_id "FK to service_teams.id"
142+
INTEGER maintainer_id "FK to maintainers.id"
143+
INTEGER collaborator_id "FK to collaborators.id"
144+
}
145+
146+
companies ||--o{ maintainers : employs
147+
projects ||--o{ projects : parent_of
148+
projects ||--o{ service_projects : includes
149+
services ||--o{ service_projects : provides
150+
maintainers ||--o{ maintainer_projects : assigned_to
151+
projects ||--o{ maintainer_projects : hosts
152+
service_teams ||--o{ service_user_teams : has
153+
services ||--o{ service_users : has
154+
```
155+
156+
157+
158+
## init-container
159+
- bootstrap process loads project and maintainer data from
160+
- a CNCF-Internal worksheet
161+
- in future, should be loaded from a combination of PCC user profiles (keyed by GitHub user account) and a registered list of project.yaml files on a per-project basis.
162+
163+
## GitHub Event Listener
164+
165+
### cert-manager
166+
Manages the server cert associated with the maintainer-d event listener that listens for changes to onboarding issues.
167+
168+
Steps to integrate with Let's Encrypt on OKE:
169+
170+
1. Create the OCI DNS credentials secret:
171+
```bash
172+
kubectl create secret generic oci-dns-credentials -n cert-manager \
173+
--from-literal=tenancyOCID=<tenancy_ocid> \
174+
--from-literal=userOCID=<user_ocid> \
175+
--from-literal=fingerprint=<api_key_fingerprint> \
176+
--from-file=privateKey=<path_to_api_key_private_key_pem>
177+
```
178+
2. Apply a `ClusterIssuer` using the OCI DNS solver (update compartment OCID, DNS zone, and email):
179+
```yaml
180+
apiVersion: cert-manager.io/v1
181+
kind: ClusterIssuer
182+
metadata:
183+
name: letsencrypt-dns
184+
spec:
185+
acme:
186+
email: [email protected]
187+
server: https://acme-v02.api.letsencrypt.org/directory
188+
privateKeySecretRef:
189+
name: letsencrypt-dns-account-key
190+
solvers:
191+
- dns01:
192+
oci:
193+
compartmentOCID: ocid1.compartment.oc1..aaaaaaaa22icap66vxktktubjlhf6oxvfhev6n7udgje2chahyrtq65ga63a
194+
dnsZoneName: cncf.io.
195+
secretRef:
196+
name: oci-dns-credentials
197+
key: privateKey
198+
tenancyOCID: ocid1.tenancy.oc1... # must match the secret
199+
userOCID: ocid1.user.oc1... # must match the secret
200+
fingerprint: <api_key_fingerprint>
201+
```
202+
3. Request the certificate in the `maintainerd` namespace:
203+
```yaml
204+
apiVersion: cert-manager.io/v1
205+
kind: Certificate
206+
metadata:
207+
name: maintainerd-cert
208+
namespace: maintainerd
209+
spec:
210+
secretName: maintainerd-tls
211+
issuerRef:
212+
name: letsencrypt-dns
213+
kind: ClusterIssuer
214+
dnsNames:
215+
- github-events.cncf.io
216+
```
217+
4. Update `deploy/manifests/service.yaml` to expose HTTPS and reference the secret:
218+
```yaml
219+
metadata:
220+
annotations:
221+
service.beta.kubernetes.io/oci-load-balancer-ssl-ports: https
222+
service.beta.kubernetes.io/oci-load-balancer-tls-secret: maintainerd/maintainerd-tls
223+
spec:
224+
ports:
225+
- name: http
226+
port: 80
227+
targetPort: 2525
228+
- name: https
229+
port: 443
230+
targetPort: 2525
231+
```
232+
5. Apply the manifests (`kubectl apply -f deploy/manifests/service.yaml`) and verify:
233+
- `kubectl describe certificate maintainerd-cert -n maintainerd`
234+
- `kubectl get secret maintainerd-tls -n maintainerd`
235+
- `curl -vk https://github-events.cncf.io/healthz`
236+
237+
Ensure your OCI IAM policies allow the OKE dynamic group to manage DNS records in the relevant compartment; otherwise the ACME solver cannot create the TXT challenges.

‎deploy/cert-ops.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Certificate Maintenance for maintainer-d
2+
3+
maintainer-d uses a manually issued Let’s Encrypt certificate stored in the `maintainerd-tls` secret.
4+
5+
Repeat these steps before the cert expires (every ~60–90 days):
6+
7+
1. **Request a new certificate via certbot**
8+
```bash
9+
sudo certbot certonly \
10+
--manual \
11+
--preferred-challenges dns \
12+
--key-type rsa \
13+
-d github-events.cncf.io \
14+
--email <EMAIL_ADDRESS> \
15+
--agree-tos
16+
```
17+
Certbot prints a `_acme-challenge.github-events.cncf.io` TXT record. Ask the DNSimple admin to create it (TTL 60). Press Enter once DNS propagates.
18+
19+
2. **Load the cert into Kubernetes**
20+
```bash
21+
kubectl create secret tls maintainerd-tls \
22+
--cert=/etc/letsencrypt/live/github-events.cncf.io/fullchain.pem \
23+
--key=/etc/letsencrypt/live/github-events.cncf.io/privkey.pem \
24+
-n maintainerd \
25+
--dry-run=client -o yaml | kubectl apply -f -
26+
```
27+
28+
3. **Recreate the Service so OCI reloads the cert**
29+
```bash
30+
kubectl delete svc maintainerd -n maintainerd
31+
kubectl apply -f deploy/manifests/service.yaml
32+
kubectl get svc maintainerd -n maintainerd --watch
33+
```
34+
Wait until `EXTERNAL-IP` shows `170.9.21.206` again.
35+
36+
4. **Verify**
37+
```bash
38+
kubectl describe svc maintainerd -n maintainerd
39+
curl -vk https://github-events.cncf.io/healthz
40+
openssl s_client -connect github-events.cncf.io:443 -servername github-events.cncf.io -tls1_2
41+
```
42+
43+
Keep `/etc/letsencrypt` backed up or document the certbot host. If you ever automate DNS updates, you can replace the manual step with cert-manager and remove the monthly coordination with DNSimple.

0 commit comments

Comments
 (0)