Skip to content

Commit a783dfa

Browse files
leogdionclaude
andcommitted
feat: add scheduled CloudKit sync workflow with environment support
Add GitHub Actions workflow that automatically syncs macOS restore images, Xcode versions, and Swift versions to CloudKit every 12 hours. Code changes: - Add environment parameter to BushelCloudKitService for dev/prod selection - Update SyncEngine to support both file-based and environment variable PEM authentication - Add --environment flag and CLOUDKIT_ENVIRONMENT support to SyncCommand - Support PEM content via CLOUDKIT_PRIVATE_KEY environment variable (CI/CD friendly) Workflow features: - Runs every 12 hours (00:00 and 12:00 UTC) on ubuntu-latest - Swift 6.2 with build caching for 2-4 minute runtime after initial run - Respects fetch intervals to avoid API rate limits - Manual trigger available via workflow_dispatch - Cost-efficient: ~120-240 Linux minutes/month (within free tier) Documentation: - CLOUDKIT_SYNC_SETUP.md: comprehensive setup guide with troubleshooting - SECRETS_SETUP.md: quick checklist for GitHub secrets configuration Single key pair approach: - Same S2S key works for both development and production environments - Environment selected via CLOUDKIT_ENVIRONMENT variable, not separate keys - Simpler setup with just 2 GitHub secrets (CLOUDKIT_KEY_ID, CLOUDKIT_PRIVATE_KEY) Resolves #8 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 7364c82 commit a783dfa

File tree

6 files changed

+585
-11
lines changed

6 files changed

+585
-11
lines changed

.github/CLOUDKIT_SYNC_SETUP.md

Lines changed: 288 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# CloudKit Sync Workflow Setup
2+
3+
This document explains how to configure the scheduled CloudKit sync workflow.
4+
5+
## Prerequisites
6+
7+
1. Access to the CloudKit Dashboard: https://icloud.developer.apple.com/dashboard/
8+
2. GitHub repository admin permissions (to add secrets)
9+
3. Server-to-Server authentication key created in CloudKit
10+
11+
## Setup Steps
12+
13+
### 1. Create Server-to-Server Key (If Not Already Created)
14+
15+
1. Visit https://icloud.developer.apple.com/dashboard/
16+
2. Select your container: `iCloud.com.brightdigit.Bushel`
17+
3. Navigate to: **API Access → Server-to-Server Keys**
18+
4. Click **+** to create a new key
19+
5. Download the `.pem` file immediately (you can only download it once)
20+
6. Save the file securely (e.g., `~/Downloads/AuthKey_XXXXXXXXXX.pem`)
21+
7. Note the **Key ID** (32-character hex string)
22+
23+
### 2. Add GitHub Secrets
24+
25+
**Note**: A single S2S key works for both development and production environments. The environment is selected via the `CLOUDKIT_ENVIRONMENT` variable, not the key itself.
26+
27+
1. Go to your repository on GitHub
28+
2. Navigate to: **Settings → Secrets and variables → Actions**
29+
3. Click **New repository secret**
30+
31+
#### Add CLOUDKIT_KEY_ID
32+
33+
- **Name:** `CLOUDKIT_KEY_ID`
34+
- **Value:** Your 32-character key ID from step 1.7
35+
- Click **Add secret**
36+
37+
#### Add CLOUDKIT_PRIVATE_KEY
38+
39+
- **Name:** `CLOUDKIT_PRIVATE_KEY`
40+
- **Value:** Full contents of your `.pem` file
41+
42+
To get the content:
43+
```bash
44+
cat ~/Downloads/AuthKey_XXXXXXXXXX.pem | pbcopy
45+
```
46+
47+
Or open in a text editor and copy all lines including:
48+
```
49+
-----BEGIN PRIVATE KEY-----
50+
[base64 encoded key data]
51+
-----END PRIVATE KEY-----
52+
```
53+
54+
- Click **Add secret**
55+
56+
#### Optional: Separate Keys for Production (Advanced)
57+
58+
For enhanced security in production environments, you can optionally use separate keys:
59+
60+
**Benefits:**
61+
- Compromised dev key doesn't affect production
62+
- Different access controls per environment
63+
- Independent key rotation schedules
64+
65+
**Setup:**
66+
- Create a second S2S key in CloudKit Dashboard for production
67+
- Add `CLOUDKIT_PROD_KEY_ID` and `CLOUDKIT_PROD_PRIVATE_KEY` secrets
68+
- Update workflow to use prod secrets when `CLOUDKIT_ENVIRONMENT: production`
69+
70+
This is optional and recommended only for production deployments with real user data.
71+
72+
### 3. Verify Setup
73+
74+
#### Option 1: Manual Trigger (Recommended)
75+
76+
1. Go to: **Actions → Scheduled CloudKit Sync**
77+
2. Click **Run workflow**
78+
3. Select branch: `main`
79+
4. Click **Run workflow**
80+
5. Monitor the run for errors
81+
82+
#### Option 2: Wait for Scheduled Run
83+
84+
The workflow runs automatically every 12 hours at:
85+
- 00:00 UTC (midnight)
86+
- 12:00 UTC (noon)
87+
88+
### 4. Monitor Sync Status
89+
90+
- **Actions tab:** View workflow run history and logs
91+
- **Email notifications:** GitHub sends emails on workflow failures (configure in Settings → Notifications)
92+
- **Status badge (optional):** Add to README.md:
93+
```markdown
94+
![CloudKit Sync](https://github.com/brightdigit/BushelCloud-Schedule/actions/workflows/cloudkit-sync.yml/badge.svg)
95+
```
96+
97+
## Troubleshooting
98+
99+
### Authentication Failed
100+
101+
**Error:** `AUTHENTICATION_FAILED` or credential errors
102+
103+
**Solution:**
104+
- Verify `CLOUDKIT_KEY_ID` matches the key in CloudKit Dashboard
105+
- Ensure `CLOUDKIT_PRIVATE_KEY` includes header/footer lines
106+
- Check for extra whitespace or newlines in secret values
107+
- Confirm the PEM format is correct (use `-----BEGIN PRIVATE KEY-----`, not `-----BEGIN EC PRIVATE KEY-----`)
108+
109+
### Container Not Found
110+
111+
**Error:** `Cannot find container`
112+
113+
**Solution:**
114+
- Verify container ID: `iCloud.com.brightdigit.Bushel`
115+
- Ensure your Apple Developer account has access to this container
116+
- Check that the S2S key has permissions for this container
117+
118+
### Quota Exceeded
119+
120+
**Error:** `QUOTA_EXCEEDED`
121+
122+
**Solution:**
123+
- This is expected if the workflow runs too frequently
124+
- The workflow respects default fetch intervals to prevent this
125+
- Do not use manual triggers more than once per hour
126+
127+
### Build Failures
128+
129+
**Error:** Swift build errors
130+
131+
**Solution:**
132+
- Check if dependencies are accessible (MistKit, BushelKit)
133+
- Verify Package.resolved is committed to repository
134+
- Review workflow logs for specific compilation errors
135+
- Try clearing the cache: Delete the cache key in Actions → Caches
136+
137+
### Network Timeout
138+
139+
**Error:** Timeout during data fetch or sync
140+
141+
**Solution:**
142+
- External data sources may be temporarily unavailable
143+
- The workflow will retry on the next scheduled run
144+
- Check status of data sources: ipsw.me, xcodereleases.com, etc.
145+
146+
## Security Best Practices
147+
148+
1. **Never commit `.pem` files to version control**
149+
2. **Rotate keys every 90 days**
150+
3. **Audit key usage in CloudKit Dashboard regularly**
151+
4. **Revoke compromised keys immediately**
152+
5. **Consider separate keys for production** (optional, for enhanced security when handling real user data)
153+
154+
## Updating Secrets
155+
156+
### Rotating Keys (Recommended Every 90 Days)
157+
158+
1. Create a new S2S key in CloudKit Dashboard
159+
2. Update GitHub secrets with new values:
160+
- Go to Settings → Secrets and variables → Actions
161+
- Click on `CLOUDKIT_KEY_ID` → Update secret
162+
- Click on `CLOUDKIT_PRIVATE_KEY` → Update secret
163+
3. Test with a manual workflow run
164+
4. Revoke the old key in CloudKit Dashboard (only after confirming new key works)
165+
166+
**Tip**: The same key works for both development and production, so you only need to update it once.
167+
168+
## Performance & Cost
169+
170+
### Resource Usage
171+
172+
- **Estimated runtime:**
173+
- First run (cache miss): 8-12 minutes
174+
- Subsequent runs (cache hit): 2-4 minutes
175+
- **Frequency:** 2 runs per day = 60 runs per month
176+
- **GitHub Actions usage:** ~120-240 Linux minutes per month
177+
- **Cost:** Well within GitHub free tier (2,000 minutes/month)
178+
179+
### Build Caching
180+
181+
The workflow caches the Swift build directory (`.build`) to speed up subsequent runs:
182+
- Cache key: Based on `Package.resolved` file hash
183+
- Cache invalidation: Automatic when dependencies change
184+
- Cache size: ~50-100 MB
185+
186+
To clear the cache:
187+
1. Go to: **Actions → Caches**
188+
2. Find caches starting with `Linux-swift-build-`
189+
3. Click delete icon
190+
191+
## CloudKit Considerations
192+
193+
### Development vs Production
194+
195+
This workflow currently uses the **development** CloudKit environment:
196+
- Changes don't affect production data
197+
- Free API calls for public database
198+
- Ideal for testing and demos
199+
- Uses the same S2S key as production (environment is selected via `CLOUDKIT_ENVIRONMENT`)
200+
201+
**Switching to Production** (when ready):
202+
203+
Simply change the environment variable in `.github/workflows/cloudkit-sync.yml`:
204+
205+
```yaml
206+
- name: Run CloudKit sync
207+
env:
208+
CLOUDKIT_KEY_ID: ${{ secrets.CLOUDKIT_KEY_ID }}
209+
CLOUDKIT_PRIVATE_KEY: ${{ secrets.CLOUDKIT_PRIVATE_KEY }}
210+
CLOUDKIT_ENVIRONMENT: production # ← Change from 'development' to 'production'
211+
CLOUDKIT_CONTAINER_ID: iCloud.com.brightdigit.Bushel
212+
```
213+
214+
**Important**: The same key works for both environments. The environment parameter tells CloudKit which database to use.
215+
216+
**Recommended Approach**: Create a separate workflow file (e.g., `cloudkit-sync-production.yml`) for production syncs with manual trigger only (`workflow_dispatch`). This prevents accidental production changes and allows you to test workflow modifications in development first.
217+
218+
### Data Sources
219+
220+
The sync fetches from these sources (with default fetch intervals):
221+
- **ipsw.me** (12 hours) - macOS restore images
222+
- **TheAppleWiki** (12 hours) - macOS restore images
223+
- **Apple MESU** (1 hour) - macOS restore images signing status
224+
- **Mr. Macintosh** (12 hours) - macOS restore images
225+
- **xcodereleases.com** (12 hours) - Xcode versions
226+
- **swiftversion.net** (12 hours) - Swift versions
227+
228+
The workflow respects these intervals to avoid overwhelming data sources.
229+
230+
## Advanced Configuration
231+
232+
### Changing Sync Frequency
233+
234+
Edit `.github/workflows/cloudkit-sync.yml`:
235+
236+
```yaml
237+
schedule:
238+
- cron: '0 0 * * *' # Once daily at midnight UTC
239+
- cron: '0 */6 * * *' # Every 6 hours
240+
- cron: '0 0 * * 0' # Weekly on Sundays
241+
```
242+
243+
### Changing CloudKit Environment
244+
245+
The workflow uses the `CLOUDKIT_ENVIRONMENT` environment variable:
246+
247+
```yaml
248+
env:
249+
CLOUDKIT_ENVIRONMENT: development # or 'production'
250+
```
251+
252+
Or override with command-line flag:
253+
254+
```yaml
255+
.build/x86_64-unknown-linux-gnu/release/bushel-cloud sync \
256+
--verbose \
257+
--environment production
258+
```
259+
260+
**Valid values**: `development`, `dev`, `production`, `prod`
261+
262+
### Sync Specific Record Types Only
263+
264+
Add flags to the sync command in the workflow:
265+
266+
```yaml
267+
.build/x86_64-unknown-linux-gnu/release/bushel-cloud sync \
268+
--verbose \
269+
--restore-images-only # Or --xcode-only, --swift-only
270+
```
271+
272+
### Force Fetch (Ignore Intervals)
273+
274+
Add `--force` flag to bypass fetch throttling:
275+
276+
```yaml
277+
.build/x86_64-unknown-linux-gnu/release/bushel-cloud sync \
278+
--verbose \
279+
--force # Fetch fresh data regardless of intervals
280+
```
281+
282+
**Warning:** This increases load on external data sources and may trigger rate limits.
283+
284+
## Questions or Issues?
285+
286+
- Review project documentation: [CLAUDE.md](../CLAUDE.md)
287+
- Check S2S authentication details: [.claude/s2s-auth-details.md](../.claude/s2s-auth-details.md)
288+
- File an issue: https://github.com/brightdigit/BushelCloud-Schedule/issues

.github/SECRETS_SETUP.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# GitHub Secrets Setup Checklist
2+
3+
This file lists exactly what secrets you need to configure for the scheduled CloudKit sync workflow.
4+
5+
## Prerequisites
6+
7+
Before adding secrets, you need a CloudKit Server-to-Server key:
8+
9+
1. Visit https://icloud.developer.apple.com/dashboard/
10+
2. Select container: `iCloud.com.brightdigit.Bushel`
11+
3. Navigate to: **API Access → Server-to-Server Keys**
12+
4. Click **+** to create a new key
13+
5. Download the `.pem` file (you can only download once!)
14+
6. Copy the Key ID (32-character hex string)
15+
16+
## Required Secrets
17+
18+
You need to add **2 secrets** to your GitHub repository:
19+
20+
### 1. CLOUDKIT_KEY_ID
21+
22+
**Where to add:**
23+
- Repository → Settings → Secrets and variables → Actions → New repository secret
24+
25+
**Secret configuration:**
26+
- **Name:** `CLOUDKIT_KEY_ID`
27+
- **Value:** Your 32-character key ID from CloudKit Dashboard
28+
- **Example:** `a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6`
29+
30+
### 2. CLOUDKIT_PRIVATE_KEY
31+
32+
**Where to add:**
33+
- Repository → Settings → Secrets and variables → Actions → New repository secret
34+
35+
**Secret configuration:**
36+
- **Name:** `CLOUDKIT_PRIVATE_KEY`
37+
- **Value:** Full contents of your `.pem` file
38+
39+
**Getting the PEM content:**
40+
41+
Option A - Using terminal:
42+
```bash
43+
cat ~/Downloads/AuthKey_XXXXXXXXXX.pem | pbcopy
44+
```
45+
46+
Option B - Using text editor:
47+
1. Open the `.pem` file in a text editor
48+
2. Copy **everything** including the header and footer lines:
49+
```
50+
-----BEGIN PRIVATE KEY-----
51+
MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQg...
52+
[multiple lines of base64 encoded data]
53+
...
54+
-----END PRIVATE KEY-----
55+
```
56+
57+
**Important:** Make sure to include the `-----BEGIN PRIVATE KEY-----` and `-----END PRIVATE KEY-----` lines!
58+
59+
## Verification Checklist
60+
61+
After adding secrets, verify:
62+
63+
- [ ] Secret `CLOUDKIT_KEY_ID` exists in repository secrets
64+
- [ ] Secret `CLOUDKIT_PRIVATE_KEY` exists in repository secrets
65+
- [ ] PEM content includes header and footer lines
66+
- [ ] No extra whitespace or line breaks in key ID
67+
- [ ] Original `.pem` file stored securely (not in git!)
68+
69+
## Testing the Setup
70+
71+
1. Go to: **Actions → Scheduled CloudKit Sync**
72+
2. Click **Run workflow**
73+
3. Select branch: `main` (or current branch)
74+
4. Click **Run workflow**
75+
5. Monitor the run - it should complete successfully
76+
77+
If you see authentication errors, double-check:
78+
- Key ID matches exactly (no typos)
79+
- PEM content is complete (including headers)
80+
- No extra spaces or formatting issues
81+
82+
## Important Notes
83+
84+
- **One key for both environments:** The same key works for development AND production CloudKit environments
85+
- **Environment selection:** The environment (dev/prod) is controlled by `CLOUDKIT_ENVIRONMENT` variable in the workflow file, not by the key
86+
- **Security:** Never commit the `.pem` file to git
87+
- **Rotation:** Rotate keys every 90 days for security
88+
89+
## Quick Reference
90+
91+
| Secret Name | Where to Get It | Format |
92+
|-------------|-----------------|--------|
93+
| `CLOUDKIT_KEY_ID` | CloudKit Dashboard → API Access → Server-to-Server Keys | 32-character hex (e.g., `a1b2c3...`) |
94+
| `CLOUDKIT_PRIVATE_KEY` | Downloaded `.pem` file | Multi-line PEM format with headers |
95+
96+
## Next Steps
97+
98+
After secrets are configured:
99+
- Workflow will run automatically every 12 hours
100+
- You can trigger manually anytime via Actions tab
101+
- Check workflow logs to verify successful syncs
102+
- Monitor CloudKit Dashboard to see synced data
103+
104+
## Need Help?
105+
106+
See the full setup guide: [CLOUDKIT_SYNC_SETUP.md](CLOUDKIT_SYNC_SETUP.md)

0 commit comments

Comments
 (0)