Skip to content

Commit aed361a

Browse files
taigrrtaigrr
committed
add migration plan
1 parent 06cb0b9 commit aed361a

File tree

1 file changed

+184
-0
lines changed

1 file changed

+184
-0
lines changed

‎docs/MIGRATION.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
# DJAFS Storage Migration Guide
2+
3+
This guide covers migrating from a traditional filesystem to djafs-backed storage for queue-based data ingestion systems.
4+
5+
## Prerequisites
6+
7+
- Ubuntu 22.04+ LTS
8+
- ext4 volume with sufficient storage (current data + 30% headroom)
9+
- FUSE3 support
10+
- Ability to pause/resume queue subscribers
11+
12+
## Available Commands
13+
14+
| Command | Usage | Description |
15+
|---------|-------|-------------|
16+
| `djafs mount` | `djafs mount STORAGE_PATH MOUNTPOINT` | Mount filesystem |
17+
| `djafs convert` | `djafs convert -i INPUT -o OUTPUT [-v] [--dry-run] [--legacy]` | Convert existing data |
18+
| `djafs validate` | `djafs validate -p PATH [-v] [-r]` | Validate archives, optionally repair |
19+
| `djafs count` | `djafs count [PATH] [--progress]` | Count files in directory |
20+
21+
## Migration Phases
22+
23+
### Phase 1: Infrastructure Setup
24+
25+
| Step | Task | Verification |
26+
|------|------|--------------|
27+
| 1.1 | Provision Ubuntu 22.04+ LTS | SSH access works |
28+
| 1.2 | Provision ext4 volume, mount at `/mnt/djafs-storage` | `df -Th` shows ext4 |
29+
| 1.3 | `apt install fuse3 libfuse3-dev` | `fusermount3 --version` |
30+
| 1.4 | Build/install djafs binary | `djafs --version` |
31+
| 1.5 | `mkdir -p /mnt/djafs-mount` | Directory exists |
32+
| 1.6 | `djafs mount /mnt/djafs-storage /mnt/djafs-mount` | `mount \| grep fuse` |
33+
34+
```bash
35+
# Infrastructure setup
36+
sudo apt update && sudo apt install -y fuse3 libfuse3-dev
37+
sudo mkfs.ext4 /dev/sdX
38+
sudo mount /dev/sdX /mnt/djafs-storage
39+
mkdir -p /mnt/djafs-mount
40+
41+
# Mount djafs
42+
djafs mount /mnt/djafs-storage /mnt/djafs-mount
43+
```
44+
45+
### Phase 2: Initial Data Migration
46+
47+
| Step | Task | Verification |
48+
|------|------|--------------|
49+
| 2.1 | `rsync -avP --progress /source/ /mnt/djafs-mount/live/` | Completes (errors on new files OK) |
50+
| 2.2 | Record baseline: `djafs count /mnt/djafs-mount/live --progress` | File count noted |
51+
52+
The first rsync will error on files being added during sync - this is expected. We'll do a final sync during the cutover window.
53+
54+
```bash
55+
# Initial bulk sync (will have errors for new files - OK)
56+
rsync -avP --progress /path/to/source/ /mnt/djafs-mount/live/
57+
58+
# Record baseline count
59+
djafs count /mnt/djafs-mount/live --progress
60+
```
61+
62+
### Phase 3: Queue Subscriber Setup
63+
64+
| Step | Task | Verification |
65+
|------|------|--------------|
66+
| 3.1 | Deploy new subscriber pointing to djafs mount | Service starts |
67+
| 3.2 | Register consumer state in queue tracking | Consumer visible |
68+
| 3.3 | Test write to djafs | Test message processed |
69+
70+
Configure your new subscriber to write to `/mnt/djafs-mount/live/` instead of the original storage location.
71+
72+
### Phase 4: Cutover
73+
74+
| Step | Task | Verification |
75+
|------|------|--------------|
76+
| 4.1 | Pause prod data sink queue | Messages accumulating |
77+
| 4.2 | Pause test data sink queue | Messages accumulating |
78+
| 4.3 | Wait for in-flight to drain | No active processing |
79+
| 4.4 | `rsync -avP --delete /source/ /mnt/djafs-mount/live/` | "0 files transferred" |
80+
| 4.5 | Unpause test queue | Test messages flowing |
81+
| 4.6 | Monitor 15-30 min | No errors |
82+
| 4.7 | Unpause prod queue | Prod messages flowing |
83+
84+
```bash
85+
# Final sync (should show minimal or zero changes)
86+
rsync -avP --delete /path/to/source/ /mnt/djafs-mount/live/
87+
88+
# Dry-run first to verify
89+
rsync -avP --delete --dry-run /path/to/source/ /mnt/djafs-mount/live/
90+
```
91+
92+
### Phase 5: Validation
93+
94+
| Step | Task | Verification |
95+
|------|------|--------------|
96+
| 5.1 | `djafs count /mnt/djafs-mount/live` vs source | Counts match |
97+
| 5.2 | Sample 100 files, compare checksums | All match |
98+
| 5.3 | Check `/mnt/djafs-mount/snapshots/` | Date hierarchy exists |
99+
| 5.4 | `djafs validate -p /mnt/djafs-storage -v` | 0 errors |
100+
101+
```bash
102+
# Validate archives
103+
djafs validate -p /mnt/djafs-storage -v
104+
105+
# If issues found, attempt repair
106+
djafs validate -p /mnt/djafs-storage -v -r
107+
108+
# Compare file counts
109+
echo "Source: $(find /path/to/source -type f | wc -l)"
110+
echo "DJAFS: $(djafs count /mnt/djafs-mount/live)"
111+
112+
# Sample checksum comparison
113+
for f in $(find /path/to/source -type f | shuf -n 100); do
114+
rel="${f#/path/to/source/}"
115+
src_hash=$(sha256sum "$f" | cut -d' ' -f1)
116+
dst_hash=$(sha256sum "/mnt/djafs-mount/live/$rel" | cut -d' ' -f1)
117+
if [ "$src_hash" != "$dst_hash" ]; then
118+
echo "MISMATCH: $rel"
119+
fi
120+
done
121+
```
122+
123+
## Rollback Plan
124+
125+
| Trigger | Action |
126+
|---------|--------|
127+
| djafs mount fails | Fall back to direct ext4 writes |
128+
| Data corruption detected | Pause queues, restore from rsync source |
129+
| Performance issues | Increase GC interval, tune hot cache |
130+
131+
To unmount and rollback:
132+
133+
```bash
134+
# Graceful unmount
135+
fusermount3 -u /mnt/djafs-mount
136+
137+
# Or force unmount if needed
138+
fusermount3 -uz /mnt/djafs-mount
139+
```
140+
141+
## Estimated Timeline
142+
143+
| Phase | Duration |
144+
|-------|----------|
145+
| Infrastructure setup | 1-2 hours |
146+
| Initial rsync | Depends on data size |
147+
| Subscriber setup | 30 min |
148+
| Cutover window | 1-2 hours |
149+
| Validation | 1 hour |
150+
151+
**Total: ~4-6 hours + initial rsync time**
152+
153+
## Post-Migration
154+
155+
### Monitoring
156+
157+
- Monitor queue lag to ensure backlog clears
158+
- Watch djafs process for memory/CPU usage
159+
- Check disk space on storage volume
160+
- Verify snapshots are being created (check `/mnt/djafs-mount/snapshots/YYYY/MM/DD/`)
161+
162+
### Maintenance
163+
164+
```bash
165+
# Periodic validation
166+
djafs validate -p /mnt/djafs-storage -v
167+
168+
# Check storage health
169+
df -h /mnt/djafs-storage
170+
```
171+
172+
### Snapshot Access
173+
174+
Files are accessible at multiple points in time:
175+
176+
```
177+
/mnt/djafs-mount/
178+
├── live/ # Current state
179+
└── snapshots/
180+
├── latest/ # Most recent state
181+
└── YYYY/
182+
└── MM/
183+
└── DD/ # State at end of that day
184+
```

0 commit comments

Comments
 (0)