Skip to content

Commit b7f499c

Browse files
Arbitrary secret key refs and templating in collectors (#1895)
* Uses secrets from cluster * updated gitignore to stop ignoring needed files * Delete specs.go.bak * make fmt * added preflight to generic loader * Tells user to run in cluster if using secretKeyRef * Update loader.go * Update loader.go
1 parent deab5e4 commit b7f499c

24 files changed

+2799
-9
lines changed

.gitignore

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,9 @@ sbom/
5050
!testdata/supportbundle/*.tar.gz
5151
!test/baselines/**/baseline.tar.gz
5252

53-
# Ignore built binaries
54-
troubleshoot
55-
troubleshoot-test
53+
# Ignore built binaries (use / prefix to avoid catching source files)
54+
/troubleshoot
55+
/troubleshoot-test
5656
cmd/troubleshoot/troubleshoot
5757
cmd/*/troubleshoot
58-
support-bundle
58+
/support-bundle
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
apiVersion: troubleshoot.sh/v1beta3
2+
kind: SupportBundle
3+
metadata:
4+
name: test-v1beta3-secretref
5+
spec:
6+
collectors:
7+
# Test 1: PostgreSQL with URI from secret
8+
- postgres:
9+
collectorName: postgres-with-secret
10+
uri:
11+
valueFrom:
12+
secretKeyRef:
13+
name: test-database-credentials
14+
key: postgres-uri
15+
# This will fail to connect (fake server) but that's OK -
16+
# we're testing secret resolution, not actual DB connectivity
17+
18+
# Test 2: PostgreSQL with TLS certs from secret
19+
- postgres:
20+
collectorName: postgres-with-tls
21+
uri:
22+
value: "postgresql://testuser:testpass@localhost:5432/testdb"
23+
tls:
24+
cacert:
25+
valueFrom:
26+
secretKeyRef:
27+
name: test-database-credentials
28+
key: ca.crt
29+
clientCert:
30+
valueFrom:
31+
secretKeyRef:
32+
name: test-database-credentials
33+
key: client.crt
34+
clientKey:
35+
valueFrom:
36+
secretKeyRef:
37+
name: test-database-credentials
38+
key: client.key
39+
40+
# Test 3: MySQL with URI from secret
41+
- mysql:
42+
collectorName: mysql-with-secret
43+
uri:
44+
valueFrom:
45+
secretKeyRef:
46+
name: test-database-credentials
47+
key: mysql-uri
48+
49+
# Test 4: Redis with URI from secret
50+
- redis:
51+
collectorName: redis-with-secret
52+
uri:
53+
valueFrom:
54+
secretKeyRef:
55+
name: test-database-credentials
56+
key: redis-uri
57+
58+
# Test 5: Literal value (no secret) for comparison
59+
- clusterInfo: {}
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
# Secret containing database credentials
3+
apiVersion: v1
4+
kind: Secret
5+
metadata:
6+
name: test-database-credentials
7+
namespace: default
8+
type: Opaque
9+
stringData:
10+
# PostgreSQL connection URI
11+
postgres-uri: "postgresql://testuser:[email protected]:5432/testdb?sslmode=require"
12+
13+
# MySQL connection URI
14+
mysql-uri: "mysql://testuser:[email protected]:3306/testdb"
15+
16+
# Redis connection URI
17+
redis-uri: "redis://:[email protected]:6379"
18+
19+
# TLS certificates (example data)
20+
ca.crt: |
21+
-----BEGIN CERTIFICATE-----
22+
MIICpDCCAYwCCQDU+pQ3ZUD30jANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAls
23+
b2NhbGhvc3QwHhcNMjQwMTAxMDAwMDAwWhcNMjUwMTAxMDAwMDAwWjAUMRIwEAYD
24+
VQQDDAlsb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7
25+
VJTUt9Us8cKjMzEfYyjiWA4R4/M2bS1+fWIcPm15A8IgC0qC1J3xGhE=
26+
-----END CERTIFICATE-----
27+
28+
client.crt: |
29+
-----BEGIN CERTIFICATE-----
30+
MIICpDCCAYwCCQDU+pQ3ZUD30jANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAls
31+
b2NhbGhvc3QwHhcNMjQwMTAxMDAwMDAwWhcNMjUwMTAxMDAwMDAwWjAUMRIwEAYD
32+
VQQDDA5jbGllbnQtY2VydA==
33+
-----END CERTIFICATE-----
34+
35+
client.key: |
36+
-----BEGIN PRIVATE KEY-----
37+
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj
38+
MzEfYyjiWA4R4/M2bS1+fWIcPm15A8IgC0qC1J3xGhE=
39+
-----END PRIVATE KEY-----
Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
# v1beta3 Support Bundle Examples
2+
3+
This directory contains example Support Bundle specs using the v1beta3 API, which introduces `StringOrValueFrom` support for securely referencing Kubernetes Secrets and ConfigMaps in collector fields.
4+
5+
## Features
6+
7+
### StringOrValueFrom Pattern
8+
9+
The v1beta3 API introduces a Kubernetes-native pattern for referencing sensitive values:
10+
11+
```yaml
12+
uri:
13+
valueFrom:
14+
secretKeyRef:
15+
name: my-secret
16+
key: connection-uri
17+
```
18+
19+
or
20+
21+
```yaml
22+
uri: "postgresql://localhost:5432/db" # Literal value
23+
```
24+
25+
### Supported Collectors
26+
27+
Currently, v1beta3 supports `StringOrValueFrom` for:
28+
29+
- **Database collectors**: `postgres`, `mysql`, `redis`, `mssql`
30+
- `uri` field - Connection strings from secrets
31+
- `tls` fields - CA cert, client cert, and client key from secrets
32+
33+
## Examples
34+
35+
### 1. postgres-with-secret.yaml
36+
Basic PostgreSQL collector with connection URI from a secret.
37+
38+
**Use case**: Securely store database credentials without hardcoding them in the spec.
39+
40+
```bash
41+
kubectl apply -f postgres-with-secret.yaml
42+
```
43+
44+
### 2. postgres-with-tls.yaml
45+
PostgreSQL with TLS configuration from secrets.
46+
47+
**Use case**: Secure database connections with mutual TLS, storing certificates in secrets.
48+
49+
```bash
50+
kubectl apply -f postgres-with-tls.yaml
51+
```
52+
53+
### 3. multiple-databases.yaml
54+
Multiple database collectors (PostgreSQL, MySQL, Redis, MSSQL) with various configurations.
55+
56+
**Use case**: Collect diagnostics from multiple databases in your application stack.
57+
58+
```bash
59+
kubectl apply -f multiple-databases.yaml
60+
```
61+
62+
### 4. cross-namespace-secrets.yaml
63+
Accessing secrets from different namespaces.
64+
65+
**Use case**: Centralized credential management in a shared namespace.
66+
67+
```bash
68+
kubectl apply -f cross-namespace-secrets.yaml
69+
```
70+
71+
**RBAC Requirements**: The support bundle service account needs `get` permission on secrets in the referenced namespaces.
72+
73+
### 5. optional-secrets.yaml
74+
Using the `optional` field for graceful degradation.
75+
76+
**Use case**: Collect diagnostics even when some credentials are unavailable (e.g., optional secondary databases).
77+
78+
```bash
79+
kubectl apply -f optional-secrets.yaml
80+
```
81+
82+
### 6. configmap-example.yaml
83+
Using ConfigMaps for non-sensitive configuration.
84+
85+
**Use case**: Store non-sensitive connection strings (e.g., development databases) in ConfigMaps.
86+
87+
```bash
88+
kubectl apply -f configmap-example.yaml
89+
```
90+
91+
## Key Concepts
92+
93+
### Secret vs ConfigMap
94+
95+
- **Secrets**: Use for sensitive data (passwords, tokens, certificates)
96+
- **ConfigMaps**: Use for non-sensitive configuration (development endpoints, feature flags)
97+
98+
### Optional Field
99+
100+
```yaml
101+
uri:
102+
valueFrom:
103+
secretKeyRef:
104+
name: my-secret
105+
key: uri
106+
optional: true # Returns empty string if secret/key doesn't exist
107+
```
108+
109+
- `optional: false` (default): Collection fails if secret is missing
110+
- `optional: true`: Returns empty string if secret/key is missing
111+
112+
### Cross-Namespace Access
113+
114+
```yaml
115+
uri:
116+
valueFrom:
117+
secretKeyRef:
118+
name: shared-secret
119+
key: uri
120+
namespace: other-namespace # Access secrets in different namespaces
121+
```
122+
123+
If `namespace` is not specified, uses the support bundle's namespace.
124+
125+
### Backward Compatibility
126+
127+
v1beta3 maintains backward compatibility with v1beta2 TLS configuration:
128+
129+
```yaml
130+
tls:
131+
secret: # v1beta2 style
132+
name: tls-secret
133+
namespace: default
134+
```
135+
136+
## RBAC Configuration
137+
138+
Support bundles need appropriate RBAC permissions to read secrets:
139+
140+
```yaml
141+
apiVersion: rbac.authorization.k8s.io/v1
142+
kind: Role
143+
metadata:
144+
name: troubleshoot-secret-reader
145+
rules:
146+
- apiGroups: [""]
147+
resources: ["secrets"]
148+
resourceNames: ["postgres-connection", "redis-creds"] # Restrict to specific secrets
149+
verbs: ["get"]
150+
---
151+
apiVersion: rbac.authorization.k8s.io/v1
152+
kind: RoleBinding
153+
metadata:
154+
name: troubleshoot-secret-reader-binding
155+
roleRef:
156+
apiGroup: rbac.authorization.k8s.io
157+
kind: Role
158+
name: troubleshoot-secret-reader
159+
subjects:
160+
- kind: ServiceAccount
161+
name: troubleshoot
162+
namespace: default
163+
```
164+
165+
## Migration from v1beta2
166+
167+
### Before (v1beta2):
168+
```yaml
169+
apiVersion: troubleshoot.sh/v1beta2
170+
kind: SupportBundle
171+
spec:
172+
collectors:
173+
- postgres:
174+
uri: "postgresql://user:password@host:5432/db" # Hardcoded
175+
```
176+
177+
### After (v1beta3):
178+
```yaml
179+
apiVersion: troubleshoot.sh/v1beta3
180+
kind: SupportBundle
181+
spec:
182+
collectors:
183+
- postgres:
184+
uri:
185+
valueFrom:
186+
secretKeyRef:
187+
name: postgres-connection
188+
key: connection-uri
189+
```
190+
191+
## Limitations
192+
193+
1. **No value composition**: Cannot combine multiple secrets into a single value
194+
```yaml
195+
# NOT SUPPORTED
196+
uri: "postgresql://$(USERNAME):$(PASSWORD)@host:5432/db"
197+
```
198+
Store the complete connection string in a single secret key.
199+
200+
2. **Collector scope**: Only database collectors support `StringOrValueFrom` initially
201+
- Future versions will extend to HTTP, Data, and other collectors
202+
203+
3. **No templating**: The entire field value comes from one source
204+
205+
## Security Best Practices
206+
207+
1. **Use resourceNames in RBAC**: Restrict access to specific secrets
208+
2. **Separate secrets**: Don't reuse secrets across applications
209+
3. **Rotate credentials**: Update secrets regularly
210+
4. **Audit access**: Monitor secret access logs
211+
5. **Redact output**: Ensure connection strings are redacted in bundle output
212+
213+
## Troubleshooting
214+
215+
### Error: "failed to get secret default/my-secret"
216+
217+
- **Cause**: Secret doesn't exist or RBAC denied access
218+
- **Solution**: Verify secret exists: `kubectl get secret my-secret`
219+
- **Solution**: Check RBAC: `kubectl auth can-i get secret/my-secret`
220+
221+
### Error: "key 'uri' not found in secret"
222+
223+
- **Cause**: Secret exists but doesn't contain the specified key
224+
- **Solution**: Check secret keys: `kubectl get secret my-secret -o jsonpath='{.data}'`
225+
226+
### Error: "cannot specify both 'value' and 'valueFrom'"
227+
228+
- **Cause**: Both literal value and secret reference provided
229+
- **Solution**: Use only one: either `value: "string"` or `valueFrom: {...}`
230+
231+
## Additional Resources
232+
233+
- [Troubleshoot Documentation](https://troubleshoot.sh)
234+
- [v1beta3 API Reference](https://troubleshoot.sh/docs/v1beta3/)
235+
- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
236+
- [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)

0 commit comments

Comments
 (0)