Skip to content

Commit c069881

Browse files
evgeny-simplexevgeny-simplex
committed
xftp topic
1 parent 73d12aa commit c069881

File tree

1 file changed

+394
-0
lines changed

1 file changed

+394
-0
lines changed

‎spec/topics/xftp.md‎

Lines changed: 394 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,394 @@
1+
# XFTP
2+
3+
File transfer protocol for large files: router storage architecture, protocol commands, agent upload/download pipelines, chunk management, and encryption. XFTP enables secure file sharing by splitting files into encrypted chunks stored across multiple routers.
4+
5+
For XFTP transport handshake details, see [transport.md](transport.md). For the XFTP protocol specification, see [xftp.md](../../protocol/xftp.md).
6+
7+
- [Protocol overview](#protocol-overview)
8+
- [Router storage](#router-storage)
9+
- [Protocol commands](#protocol-commands)
10+
- [Agent upload pipeline](#agent-upload-pipeline)
11+
- [Agent download pipeline](#agent-download-pipeline)
12+
- [Chunk encryption](#chunk-encryption)
13+
- [Chunk management](#chunk-management)
14+
15+
---
16+
17+
## Protocol overview
18+
19+
**Source**: [FileTransfer/Protocol.hs](../../src/Simplex/FileTransfer/Protocol.hs)
20+
21+
XFTP separates file metadata from file content. A sender uploads encrypted chunks to one or more routers, then shares a file description (containing chunk locations, keys, and digests) with recipients via SMP messaging.
22+
23+
Key properties:
24+
- File encrypted as a single stream with XSalsa20-Poly1305, then split into chunks
25+
- Chunks are byte ranges of the encrypted file (not independently encrypted)
26+
- Chunks can be replicated across multiple routers
27+
- Recipients download chunks directly from routers
28+
- Router never sees plaintext or file metadata
29+
30+
### Parties
31+
32+
| Party | Role | Authentication |
33+
|-------|------|----------------|
34+
| Sender | Creates file, uploads chunks, manages recipients | Per-file sender key |
35+
| Recipient | Downloads chunks, acknowledges receipt | Per-recipient key (created by sender) |
36+
37+
### File description
38+
39+
The sender generates a `ValidFileDescription` containing:
40+
- Chunk specifications: server address, recipient ID, recipient key, size, digest
41+
- Encryption key and nonce for the full file
42+
- File size and SHA-512 digest
43+
- Optional redirect to another file description
44+
45+
---
46+
47+
## Router storage
48+
49+
**Source**: [FileTransfer/Server/Store.hs](../../src/Simplex/FileTransfer/Server/Store.hs)
50+
51+
### In-memory store
52+
53+
```haskell
54+
data FileStore = FileStore
55+
{ files :: TMap SenderId FileRec,
56+
recipients :: TMap RecipientId (SenderId, RcvPublicAuthKey),
57+
usedStorage :: TVar Int64
58+
}
59+
```
60+
61+
- `files` maps sender IDs to file records
62+
- `recipients` maps recipient IDs to (sender, auth key) for download authorization
63+
- `usedStorage` tracks total bytes for quota enforcement
64+
65+
### File record
66+
67+
```haskell
68+
data FileRec = FileRec
69+
{ senderId :: SenderId,
70+
fileInfo :: FileInfo, -- sndKey, size, digest
71+
filePath :: TVar (Maybe FilePath), -- set after upload
72+
recipientIds :: TVar (Set RecipientId),
73+
createdAt :: RoundedFileTime, -- truncated to 1-hour precision
74+
fileStatus :: TVar ServerEntityStatus
75+
}
76+
```
77+
78+
The `filePath` is `Nothing` until FPUT completes. The file is stored at `filesPath/<base64url(senderId)>`.
79+
80+
### Quota management
81+
82+
File size is reserved atomically when FNEW is processed. If `usedStorage + fileSize > fileSizeQuota`, the request is rejected with QUOTA error. Storage is released when files are deleted or expire.
83+
84+
### File expiration
85+
86+
Files expire based on `ttl` configuration (default 48 hours). The expiration thread periodically scans files where `createdAt + fileTimePrecision < threshold`. Expired files are deleted from disk and removed from the store.
87+
88+
`fileTimePrecision` is 3600 seconds (1 hour), providing k-anonymity for file creation times.
89+
90+
---
91+
92+
## Protocol commands
93+
94+
**Source**: [FileTransfer/Protocol.hs](../../src/Simplex/FileTransfer/Protocol.hs), [FileTransfer/Server.hs](../../src/Simplex/FileTransfer/Server.hs)
95+
96+
### Command summary
97+
98+
| Command | Party | Purpose |
99+
|---------|-------|---------|
100+
| FNEW | Sender | Create file with metadata and initial recipient keys |
101+
| FADD | Sender | Add recipient auth keys to existing file |
102+
| FPUT | Sender | Upload encrypted chunk data |
103+
| FDEL | Sender | Delete file from router |
104+
| FGET | Recipient | Download file (initiates DH key exchange) |
105+
| FACK | Recipient | Acknowledge download, remove recipient from file |
106+
| PING | Recipient | Keepalive |
107+
108+
### FNEW - create file
109+
110+
Request: `FNEW FileInfo (NonEmpty RcvPublicAuthKey) (Maybe BasicAuth)`
111+
112+
- `FileInfo`: sender's auth key, file size (Word32), SHA-512 digest
113+
- Recipient keys: one per intended recipient
114+
- Optional basic auth for servers requiring authorization
115+
116+
Response: `FRSndIds SenderId (NonEmpty RecipientId)`
117+
118+
The router generates random sender ID and recipient IDs. The sender uses `SenderId` for subsequent commands; recipients receive their `RecipientId` via file description.
119+
120+
### FPUT - upload chunk
121+
122+
Request: `FPUT` with chunk data in HTTP/2 body
123+
124+
The router:
125+
1. Validates sender authorization
126+
2. Reserves storage quota
127+
3. Receives encrypted chunk with timeout
128+
4. Writes to `filesPath/<base64url(senderId)>`
129+
5. Updates `filePath` in file record
130+
131+
If the file already has a `filePath` (re-upload), the body is discarded and `FROk` returned immediately.
132+
133+
### FGET - download chunk
134+
135+
Request: `FGET RcvPublicDhKey`
136+
137+
The recipient provides an ephemeral X25519 public key for DH agreement.
138+
139+
Response: `FRFile SrvPublicDhKey C.CbNonce` (server's ephemeral DH key and nonce)
140+
141+
The router:
142+
1. Generates ephemeral DH key pair
143+
2. Computes shared secret: `dh'(recipientDhKey, serverPrivKey)`
144+
3. Initializes encryption state with shared secret and nonce
145+
4. Streams encrypted file in HTTP/2 response body
146+
147+
The recipient uses the returned server DH key and nonce to decrypt the stream.
148+
149+
### FACK - acknowledge receipt
150+
151+
Request: `FACK`
152+
153+
Removes the recipient from the file's recipient set. Once all recipients have acknowledged, only the sender can access the file (until FDEL or expiration).
154+
155+
### FDEL - delete file
156+
157+
Request: `FDEL`
158+
159+
Deletes the file from disk and store, releases quota. All recipient IDs become invalid.
160+
161+
---
162+
163+
## Agent upload pipeline
164+
165+
**Source**: [FileTransfer/Agent.hs](../../src/Simplex/FileTransfer/Agent.hs), [FileTransfer/Chunks.hs](../../src/Simplex/FileTransfer/Chunks.hs)
166+
167+
### Upload state machine
168+
169+
```
170+
SFSNew -> SFSEncrypting -> SFSEncrypted -> SFSUploading -> SFSComplete
171+
\-> SFSError
172+
```
173+
174+
### Phase 1: File preparation (SFSNew -> SFSEncrypted)
175+
176+
`prepareFile` encrypts the source file:
177+
178+
1. Generate random `SbKey` and `CbNonce`
179+
2. Create encrypted file structure:
180+
- 8 bytes: encoded content length
181+
- FileHeader: filename and optional metadata (SMP-encoded)
182+
- File content: encrypted in 64KB streaming chunks
183+
- Padding: `'#'` characters to multiple of 16384 bytes
184+
- Auth tag: 16 bytes (Poly1305)
185+
3. Compute SHA-512 digest of encrypted file
186+
4. Calculate chunk boundaries via `prepareChunkSizes`
187+
188+
### Chunk size selection
189+
190+
`prepareChunkSizes` selects chunk sizes based on total file size:
191+
192+
| File size | Chunk size used |
193+
|-----------|-----------------|
194+
| > 3/4 of 4MB (~3.0MB) | 4MB chunks |
195+
| > 3/4 of 1MB (768KB) | 1MB chunks |
196+
| Otherwise | 64KB or 256KB |
197+
198+
The last chunk may be smaller than the standard size.
199+
200+
### Phase 2: Chunk registration
201+
202+
For each chunk:
203+
1. Select XFTP server (different server per chunk recommended)
204+
2. Send FNEW with chunk's digest and recipient keys
205+
3. Store `SndFileChunkReplica` with server-assigned IDs
206+
4. Status: `SFRSCreated`
207+
208+
### Phase 3: Upload (SFSUploading -> SFSComplete)
209+
210+
`uploadFileChunk` for each replica:
211+
1. If not all recipients added: send FADD
212+
2. Read chunk from encrypted file at (offset, size)
213+
3. Send FPUT with chunk data
214+
4. Update replica status to `SFRSUploaded`
215+
5. Report progress to agent client
216+
217+
When all chunks uploaded: mark file `SFSComplete`, generate file description.
218+
219+
### Error handling
220+
221+
- Retry with exponential backoff per `reconnectInterval`
222+
- Track consecutive retries per replica
223+
- After `xftpConsecutiveRetries` failures: mark `SFSError`
224+
- Delay and retry count stored in DB for resumption
225+
226+
---
227+
228+
## Agent download pipeline
229+
230+
**Source**: [FileTransfer/Agent.hs](../../src/Simplex/FileTransfer/Agent.hs)
231+
232+
### Download state machine
233+
234+
```
235+
RFSReceiving -> RFSReceived -> RFSDecrypting -> RFSComplete
236+
\-> RFSError
237+
```
238+
239+
### Phase 1: Chunk download (RFSReceiving -> RFSReceived)
240+
241+
`downloadFileChunk` for each chunk:
242+
1. Verify server is in approved relays (if relay approval required)
243+
2. Generate ephemeral DH key pair
244+
3. Send FGET with public DH key
245+
4. Receive `FRFile` with server's DH key and nonce
246+
5. Compute shared secret, initialize decryption
247+
6. Stream-decrypt chunk to `tmpPath/chunkNo`
248+
7. Verify chunk's SHA-256 digest matches specification
249+
8. Mark replica as `received`
250+
251+
Replicas are tried in order; if first fails, try next replica of same chunk.
252+
253+
### Phase 2: Reassembly (RFSReceived -> RFSComplete)
254+
255+
`decryptFile` reassembles and decrypts:
256+
1. Concatenate all chunk files in order
257+
2. Validate total size matches file digest
258+
3. Decrypt with file's `SbKey` and `CbNonce`:
259+
- Parse length prefix and FileHeader
260+
- Stream-decrypt content
261+
- Verify auth tag
262+
4. Write to final destination (`savePath`)
263+
5. Delete temporary chunk files
264+
6. Mark `RFSComplete`
265+
266+
### Redirect files
267+
268+
If the file description has a `redirect` field:
269+
1. Decrypt the downloaded content
270+
2. Parse as YAML file description
271+
3. Validate size/digest match redirect specification
272+
4. Register actual chunks from redirect description
273+
5. Download from redirected sources
274+
275+
This enables indirection for large file descriptions or server migration.
276+
277+
---
278+
279+
## Chunk encryption
280+
281+
**Source**: [FileTransfer/Crypto.hs](../../src/Simplex/FileTransfer/Crypto.hs), [Messaging/Crypto/File.hs](../../src/Simplex/Messaging/Crypto/File.hs)
282+
283+
### File encryption (sender side)
284+
285+
```
286+
[8-byte length][FileHeader][file content][padding][16-byte auth tag]
287+
```
288+
289+
- Algorithm: XSalsa20-Poly1305 (NaCl secret_box)
290+
- Key: random 32-byte `SbKey`
291+
- Nonce: random 24-byte `CbNonce`
292+
- Streaming: 64KB chunks encrypted incrementally
293+
- Padding: `'#'` characters to align to 16384-byte boundary
294+
295+
### Chunk transport encryption (FGET)
296+
297+
Each FGET establishes a fresh DH shared secret:
298+
1. Recipient generates ephemeral X25519 key pair
299+
2. Sends public key in FGET request
300+
3. Router generates ephemeral key pair
301+
4. Both compute: `secret = dh(peerPubKey, ownPrivKey)`
302+
5. Router streams chunk encrypted with `cbInit(secret, nonce)`
303+
6. Recipient decrypts with same parameters
304+
305+
This provides forward secrecy per-download - compromising the file encryption key does not reveal transport keys.
306+
307+
### Auth tag verification
308+
309+
The 16-byte Poly1305 auth tag is verified after receiving all chunks:
310+
- Single chunk: tag appended at end
311+
- Multiple chunks: tag in final chunk, verified after concatenation
312+
313+
Failed auth tag verification produces `CRYPTO` error.
314+
315+
---
316+
317+
## Chunk management
318+
319+
**Source**: [FileTransfer/Types.hs](../../src/Simplex/FileTransfer/Types.hs)
320+
321+
### Sender chunk state
322+
323+
```haskell
324+
data SndFileChunkReplica = SndFileChunkReplica
325+
{ sndChunkReplicaId :: Int64,
326+
server :: XFTPServer,
327+
replicaId :: ChunkReplicaId,
328+
replicaKey :: C.APrivateAuthKey,
329+
rcvIdsKeys :: [(ChunkReplicaId, C.APrivateAuthKey)],
330+
replicaStatus :: SndFileReplicaStatus,
331+
delay :: Maybe Int64,
332+
retries :: Int
333+
}
334+
335+
data SndFileReplicaStatus = SFRSCreated | SFRSUploaded
336+
```
337+
338+
- `SFRSCreated`: FNEW sent, replica registered on server
339+
- `SFRSUploaded`: FPUT complete, chunk data stored
340+
- `rcvIdsKeys`: recipient IDs and keys for this replica
341+
342+
### Recipient chunk state
343+
344+
```haskell
345+
data RcvFileChunk = RcvFileChunk
346+
{ rcvFileChunkId :: Int64,
347+
chunkNo :: Int,
348+
chunkSize :: Word32,
349+
digest :: ByteString,
350+
replicas :: [RcvFileChunkReplica],
351+
fileTmpPath :: FilePath,
352+
chunkTmpPath :: Maybe FilePath
353+
}
354+
355+
data RcvFileChunkReplica = RcvFileChunkReplica
356+
{ rcvChunkReplicaId :: Int64,
357+
server :: XFTPServer,
358+
replicaId :: ChunkReplicaId,
359+
replicaKey :: C.APrivateAuthKey,
360+
received :: Bool,
361+
delay :: Maybe Int64,
362+
retries :: Int
363+
}
364+
```
365+
366+
### Replica selection
367+
368+
Each chunk can have multiple replicas on different servers. The file description includes all replicas; the recipient:
369+
1. Tries first replica
370+
2. On failure, tries next replica
371+
3. Continues until success or all replicas exhausted
372+
373+
This provides redundancy against server unavailability.
374+
375+
### Retry handling
376+
377+
Retry state is stored per-replica with two fields:
378+
- `delay :: Maybe Int64` - milliseconds until next retry
379+
- `retries :: Int` - consecutive failure count
380+
381+
On failure, delay increases with exponential backoff. State persists in DB for resumption after agent restart.
382+
383+
### Chunk sizes
384+
385+
```haskell
386+
chunkSize0 = kb 64 -- 65536 bytes
387+
chunkSize1 = kb 256 -- 262144 bytes
388+
chunkSize2 = mb 1 -- 1048576 bytes
389+
chunkSize3 = mb 4 -- 4194304 bytes
390+
391+
serverChunkSizes = [chunkSize0, chunkSize1, chunkSize2, chunkSize3]
392+
```
393+
394+
Routers validate that uploaded chunks match one of the allowed sizes. This prevents fingerprinting based on exact file sizes.

0 commit comments

Comments
 (0)