Skip to content

Commit f45dfdf

Browse files
committed
docs: add Cloud API testing strategy and script
Add comprehensive testing guide and automated script for safe Cloud API testing: - CLOUD_TESTING_STRATEGY.md with safety procedures and cleanup patterns - scripts/test_cloud_api.sh with automatic resource cleanup - Trap handling to ensure cleanup on interruption - Resource naming conventions for easy identification Related to #462
1 parent 46ead3e commit f45dfdf

File tree

2 files changed

+472
-0
lines changed

2 files changed

+472
-0
lines changed

CLOUD_TESTING_STRATEGY.md

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
# Cloud Testing Strategy
2+
3+
## Overview
4+
5+
Testing against live Redis Cloud API requires careful resource management to avoid:
6+
- Leaving test resources running (costs money)
7+
- Creating orphaned resources
8+
- Hitting rate limits
9+
- Interfering with production resources
10+
11+
## Pre-Testing Setup
12+
13+
### 1. Credentials
14+
```bash
15+
export REDIS_CLOUD_API_KEY="your-api-key"
16+
export REDIS_CLOUD_SECRET_KEY="your-secret-key"
17+
```
18+
19+
### 2. Test Profile
20+
```bash
21+
redisctl profile set cloud-test \
22+
--deployment cloud \
23+
--api-key "$REDIS_CLOUD_API_KEY" \
24+
--api-secret "$REDIS_CLOUD_SECRET_KEY"
25+
```
26+
27+
### 3. Baseline Inventory
28+
Before starting, record current resources:
29+
```bash
30+
redisctl --profile cloud-test cloud subscription list -o json > pre-test-subscriptions.json
31+
redisctl --profile cloud-test cloud database list -o json > pre-test-databases.json
32+
```
33+
34+
## Testing Workflow
35+
36+
### Phase 1: Read-Only Operations (Safe)
37+
Test all GET operations first - no resource creation:
38+
```bash
39+
# Account operations
40+
redisctl --profile cloud-test cloud account get
41+
redisctl --profile cloud-test cloud account regions list
42+
redisctl --profile cloud-test cloud account modules list
43+
44+
# List operations
45+
redisctl --profile cloud-test cloud subscription list
46+
redisctl --profile cloud-test cloud database list
47+
redisctl --profile cloud-test cloud user list
48+
redisctl --profile cloud-test cloud acl list
49+
```
50+
51+
### Phase 2: Create → Verify → Delete (Controlled)
52+
53+
**IMPORTANT**: For each create operation, immediately note the resource ID for cleanup.
54+
55+
#### Database Testing Pattern
56+
```bash
57+
# 1. Create with minimal config
58+
SUBSCRIPTION_ID="<existing-sub-id>"
59+
cat > test-db.json <<EOF
60+
{
61+
"name": "test-db-$(date +%s)",
62+
"protocol": "redis",
63+
"memoryLimitInGb": 1.0,
64+
"replication": false,
65+
"dataEvictionPolicy": "allkeys-lru"
66+
}
67+
EOF
68+
69+
# 2. Create and capture ID
70+
DB_RESULT=$(redisctl --profile cloud-test cloud database create \
71+
--subscription "$SUBSCRIPTION_ID" \
72+
test-db.json \
73+
--wait \
74+
-o json)
75+
76+
DB_ID=$(echo "$DB_RESULT" | jq -r '.databaseId')
77+
echo "Created database: $DB_ID"
78+
79+
# 3. Verify
80+
redisctl --profile cloud-test cloud database get \
81+
--subscription "$SUBSCRIPTION_ID" \
82+
--database "$DB_ID"
83+
84+
# 4. Test operations on the database
85+
redisctl --profile cloud-test cloud database update \
86+
--subscription "$SUBSCRIPTION_ID" \
87+
--database "$DB_ID" \
88+
'{"memoryLimitInGb": 2.0}'
89+
90+
# 5. CLEANUP - Delete immediately
91+
redisctl --profile cloud-test cloud database delete \
92+
--subscription "$SUBSCRIPTION_ID" \
93+
--database "$DB_ID" \
94+
--wait
95+
96+
echo "Deleted database: $DB_ID"
97+
```
98+
99+
#### User/ACL Testing Pattern
100+
```bash
101+
# Create
102+
USER_RESULT=$(redisctl --profile cloud-test cloud user create \
103+
'{"name":"test-user-'$(date +%s)'","role":"manager"}' \
104+
-o json)
105+
USER_ID=$(echo "$USER_RESULT" | jq -r '.userId')
106+
107+
# Test
108+
redisctl --profile cloud-test cloud user get "$USER_ID"
109+
110+
# CLEANUP
111+
redisctl --profile cloud-test cloud user delete "$USER_ID" --wait
112+
```
113+
114+
### Phase 3: Async Operations Testing
115+
116+
Test `--wait` functionality:
117+
```bash
118+
# Create with --wait
119+
redisctl --profile cloud-test cloud database create \
120+
--subscription "$SUBSCRIPTION_ID" \
121+
test-db.json \
122+
--wait \
123+
--wait-timeout 300 \
124+
--wait-interval 5
125+
126+
# Verify task tracking works
127+
```
128+
129+
### Phase 4: Error Handling
130+
131+
Test error scenarios:
132+
```bash
133+
# Invalid database config
134+
redisctl --profile cloud-test cloud database create \
135+
--subscription "$SUBSCRIPTION_ID" \
136+
'{"invalid": "config"}' 2>&1
137+
138+
# Non-existent resource
139+
redisctl --profile cloud-test cloud database get \
140+
--subscription "$SUBSCRIPTION_ID" \
141+
--database 999999 2>&1
142+
143+
# Invalid subscription ID
144+
redisctl --profile cloud-test cloud database list \
145+
--subscription 999999 2>&1
146+
```
147+
148+
## Cleanup Procedure
149+
150+
### During Testing
151+
Maintain a cleanup script as you test:
152+
```bash
153+
#!/bin/bash
154+
# cleanup.sh - Run this if testing is interrupted
155+
156+
set -x
157+
158+
# Add each created resource
159+
redisctl --profile cloud-test cloud database delete --subscription 12345 --database 67890 --wait
160+
redisctl --profile cloud-test cloud user delete 111 --wait
161+
# etc.
162+
```
163+
164+
### After Testing
165+
```bash
166+
# Compare current vs baseline
167+
redisctl --profile cloud-test cloud subscription list -o json > post-test-subscriptions.json
168+
redisctl --profile cloud-test cloud database list -o json > post-test-databases.json
169+
170+
# Identify any orphaned resources
171+
diff <(jq -r '.[].id' pre-test-subscriptions.json) \
172+
<(jq -r '.[].id' post-test-subscriptions.json)
173+
```
174+
175+
### Emergency Cleanup
176+
If you lose track of test resources:
177+
```bash
178+
# List all databases and manually review
179+
redisctl --profile cloud-test cloud database list -o json | \
180+
jq -r '.[] | select(.name | startswith("test-")) | "\(.subscriptionId) \(.databaseId) \(.name)"'
181+
182+
# Delete any with "test-" prefix (DANGEROUS - verify first!)
183+
```
184+
185+
## Resource Naming Convention
186+
187+
Always prefix test resources for easy identification:
188+
- Databases: `test-db-<timestamp>`
189+
- Users: `test-user-<timestamp>`
190+
- ACL Rules: `test-acl-<timestamp>`
191+
- Subscriptions: **DO NOT CREATE** - use existing only
192+
193+
## Rate Limiting
194+
195+
Redis Cloud API limits:
196+
- 400 requests per minute per account
197+
- Add delays if testing many operations:
198+
```bash
199+
sleep 0.2 # 200ms between requests = ~300 req/min
200+
```
201+
202+
## Safety Checklist
203+
204+
Before running tests:
205+
- [ ] Using test profile (not production)
206+
- [ ] Baseline inventory captured
207+
- [ ] Cleanup script ready
208+
- [ ] Resource naming convention clear
209+
- [ ] Testing on non-critical subscription
210+
211+
During tests:
212+
- [ ] Note each created resource ID
213+
- [ ] Delete resources immediately after testing
214+
- [ ] Monitor for failures
215+
- [ ] Check rate limit warnings
216+
217+
After tests:
218+
- [ ] All test resources deleted
219+
- [ ] Baseline comparison clean
220+
- [ ] No unexpected costs
221+
- [ ] Cleanup script cleared
222+
223+
## What NOT to Test
224+
225+
**DO NOT** test these operations on live account:
226+
- ❌ Subscription creation (costs money immediately)
227+
- ❌ Subscription deletion (cannot undo)
228+
- ❌ Payment method changes
229+
- ❌ Account-level settings changes
230+
- ❌ VPC/Peering/TGW (complex setup, hard to clean up)
231+
232+
Use mocked tests (wiremock) for these instead.
233+
234+
## Test Coverage Goals
235+
236+
Focus Cloud testing on:
237+
1. ✅ Read operations (all handlers)
238+
2. ✅ Database CRUD (critical path)
239+
3. ✅ User/ACL CRUD (security critical)
240+
4. ✅ Async operation handling (`--wait`)
241+
5. ✅ Output format validation (JSON, YAML, table)
242+
6. ✅ Error handling and messages
243+
7. ⏸️ Complex operations (use mocks instead)
244+
245+
## Emergency Contact
246+
247+
If things go wrong:
248+
1. Run cleanup script
249+
2. Check Redis Cloud console
250+
3. Delete resources manually via UI if needed
251+
4. Note any orphaned resources for billing review

0 commit comments

Comments
 (0)