Skip to content

Commit 82cc0b5

Browse files
committed
chore: reademe updates
1 parent 4450b16 commit 82cc0b5

File tree

1 file changed

+221
-17
lines changed
  • lib/lib-storage/src/s3-transfer-manager

1 file changed

+221
-17
lines changed

lib/lib-storage/src/s3-transfer-manager/README.md

Lines changed: 221 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ The S3TransferManager constructor accepts an optional `S3TransferManagerConfig`
8484
**Example:**
8585
8686
```js
87-
const transferManager = new S3TransferManager({
87+
const tm = new S3TransferManager({
8888
s3ClientInstance: new S3Client({ region: "us-west-2" }),
8989
targetPartSizeBytes: 10 * 1024 * 1024, // 10MB
9090
multipartUploadThresholdBytes: 20 * 1024 * 1024, // 20MB
@@ -102,6 +102,10 @@ const transferManager = new S3TransferManager({
102102
103103
### upload()
104104
105+
> 🚧 **Under Development**
106+
>
107+
> Documentation will be available when this feature is implemented.
108+
105109
### download()
106110
107111
Downloads objects from S3 using multipart download with two modes:
@@ -151,20 +155,71 @@ We do not recommend updating the object you're downloading mid-download as this
151155
152156
#### uploadAll()
153157
158+
> 🚧 **Under Development**
159+
>
160+
> Documentation will be available when this feature is implemented.
161+
154162
#### downloadAll()
155163
164+
> 🚧 **Under Development**
165+
>
166+
> Documentation will be available when this feature is implemented.
167+
156168
## Event Handling
157169
158-
### addEventListener()
170+
**Event Types and Data:**
159171
160-
Registers event listeners for transfer lifecycle monitoring. It uses familiar EventTarget API patterns.
172+
Event listeners receive a single event object with the following properties:
173+
174+
- **`transferInitiated`** - Fired once when transfer begins
175+
176+
- `request` - Original transfer request (Bucket, Key, etc.)
177+
- `snapshot` - Initial progress state (`transferredBytes: 0`, `totalBytes` if known)
178+
179+
- **`bytesTransferred`** - Fired during transfer progress with each chunk
180+
181+
- `request` - Original transfer request
182+
- `snapshot` - Current progress (`transferredBytes`, `totalBytes`, `transferredFiles` for directory transfers)
183+
184+
- **`transferComplete`** - Fired once when transfer succeeds
185+
186+
- `request` - Original transfer request
187+
- `snapshot` - Final progress state
188+
- `response` - Complete S3 response with metadata
189+
190+
- **`transferFailed`** - Fired once when transfer fails
191+
- `request` - Original transfer request
192+
- `snapshot` - Progress state at time of failure
193+
194+
**Creating Callback Functions:**
195+
196+
Event callbacks receive a single event object. Use destructuring to access specific properties:
197+
198+
```js
199+
// Basic function - access specific properties
200+
function transferComplete({ request, snapshot, response }) {
201+
console.log(`Transfer completed: ${request.Key}`);
202+
console.log(`Total bytes: ${snapshot.transferredBytes}`);
203+
console.log(`Response status: ${response.$metadata?.httpStatusCode}`);
204+
}
161205
162-
**Event Types:**
206+
// Arrow function - inline usage
207+
const progressHandler = ({ snapshot }) => {
208+
const percent = snapshot.totalBytes ? (snapshot.transferredBytes / snapshot.totalBytes) * 100 : 0;
209+
console.log(`Progress: ${percent.toFixed(1)}%`);
210+
};
163211
164-
- `transferInitiated` - Fired when transfer begins
165-
- `bytesTransferred` - Fired during transfer progress with each byte chunk transfer
166-
- `transferComplete` - Fired when transfer succeeds
167-
- `transferFailed` - Fired when transfer fails
212+
// Object with handleEvent method
213+
const transferLogger = {
214+
handleEvent: ({ request, snapshot }) => {
215+
console.log(`${request.Key}: ${snapshot.transferredBytes} bytes transferred`);
216+
},
217+
};
218+
```
219+
220+
### addEventListener()
221+
222+
Registers event listeners for transfer lifecycle monitoring. It uses familiar EventTarget API patterns.
168223
169224
**Parameters:**
170225
@@ -196,10 +251,10 @@ function progressBar({ request, snapshot }) {
196251
process.stdout.write(`Downloading... ${progressBar} ${percent.toFixed(0)}%`);
197252
}
198253
199-
transferManager.addEventListener("bytesTransferred", progressBar);
254+
tm.addEventListener("bytesTransferred", progressBar);
200255
201256
// One-time listener
202-
transferManager.addEventListener(
257+
tm.addEventListener(
203258
"transferComplete",
204259
(event) => {
205260
console.log(`\nTransfer completed: ${event.request.Key}`);
@@ -210,21 +265,28 @@ transferManager.addEventListener(
210265
211266
### removeEventListener()
212267
213-
Removes a previously registered event listener from the specified event type.
268+
Removes a previously registered event listener from the specified event type. You must pass the exact same function reference that was used when adding the listener.
269+
270+
**Important:** If you plan to remove event listeners during transfer lifecycle, define your callback as a named function or variable - you cannot remove anonymous functions.
214271
215272
**Parameters:**
216273
217274
- `type` - Event type to stop listening for
218-
- `callback` - The exact function that was previously registered
275+
- `callback` - The exact function reference that was previously registered
219276
- `options` - Optional configuration (currently unused)
220277
221278
**Example:**
222279
223280
```js
281+
// Can be removed
224282
const progressHandler = (event) => console.log("Progress:", event.snapshot);
225283
226-
transferManager.addEventListener("bytesTransferred", progressHandler);
227-
transferManager.removeEventListener("bytesTransferred", progressHandler);
284+
tm.addEventListener("bytesTransferred", progressHandler);
285+
tm.removeEventListener("bytesTransferred", progressHandler); // Works
286+
287+
// Cannot be removed
288+
tm.addEventListener("bytesTransferred", (event) => console.log("Progress:", event.snapshot));
289+
tm.removeEventListener("bytesTransferred", (event) => console.log("Progress:", event.snapshot)); // Won't work - different function reference
228290
```
229291
230292
### dispatchEvent()
@@ -251,10 +313,152 @@ transferManager.dispatchEvent(customEvent);
251313
252314
### AbortSignal
253315
254-
TODO: Include practical examples of using abortcontroller to cancel downloads
316+
Use the standard AbortController (included in AWS SDK JS V3's HttpHandlerOptions) to cancel downloads at any time during transfer.
317+
318+
**Timeout-Based Cancellation:**
319+
320+
```js
321+
const controller = new AbortController();
322+
323+
// Auto-cancel after 30 seconds
324+
const timeoutId = setTimeout(() => {
325+
controller.abort();
326+
console.log("Download timed out");
327+
}, 30000);
328+
329+
try {
330+
const download = await tm.download({ Bucket: "my-bucket", Key: "data.json" }, { abortSignal: controller.signal });
331+
332+
clearTimeout(timeoutId); // Cancel timeout on success
333+
const data = await download.Body?.transformToByteArray();
334+
} catch (error) {
335+
clearTimeout(timeoutId);
336+
if (error.name === "AbortError") {
337+
console.log("Operation was aborted");
338+
}
339+
}
340+
```
341+
342+
**User-Triggered Cancellation:**
343+
344+
```js
345+
const controller = new AbortController();
346+
347+
// UI cancel button
348+
document.getElementById("cancelBtn").onclick = () => {
349+
controller.abort();
350+
console.log("Download cancelled by user");
351+
};
352+
353+
// Start download
354+
try {
355+
const download = await tm.download({ Bucket: "my-bucket", Key: "video.mp4" }, { abortSignal: controller.signal });
356+
357+
const data = await download.Body?.transformToByteArray();
358+
console.log("Download completed");
359+
} catch (error) {
360+
if (error.name === "AbortError") {
361+
console.log("Download was cancelled");
362+
}
363+
}
364+
```
255365
256366
### Event Listeners
257367
258-
TODO: Include examples of eventListeners are client level and request level
368+
Event listeners can be configured at two levels: **client-level** (applies to all transfers) and **request-level** (applies to specific transfers). (see [Event Handling](#event-handling))
369+
370+
**Client-Level Event Listeners:**
371+
372+
You can configure the event listeners when creating your Transfer Manager instance. These listeners apply to all transfers made with this instance.
373+
374+
```js
375+
const tm = new S3TransferManager({
376+
s3ClientInstance: s3Client,
377+
multipartDownloadType: "RANGE",
378+
checksumValidationEnabled: true,
379+
eventListeners: {
380+
transferInitiated: [downloadingKey],
381+
bytesTransferred: [progressBar],
382+
transferComplete: [
383+
{
384+
handleEvent: ({ request, snapshot, response }) => {
385+
console.log(`Transfer completed: ${request.Key}`);
386+
console.log(`Total bytes: ${snapshot.transferredBytes}`);
387+
},
388+
},
389+
],
390+
transferFailed: [transferFailed],
391+
},
392+
});
393+
394+
// All downloads will use these event listeners
395+
const download1 = await tm.download({ Bucket: "my-bucket", Key: "file1.txt" });
396+
const download2 = await tm.download({ Bucket: "my-bucket", Key: "file2.txt" });
397+
```
398+
399+
**Request-Level Event Listeners:**
259400
260-
## Performance
401+
You can add event listeners for individual requests like this. Note adding event listeners at request-level will supplement any event listeners defined at client-level. So if you add the same callback at client and request level they will duplicate when the respective event occurs.
402+
403+
```js
404+
const download = await tm.download(
405+
{
406+
Bucket: "my-bucket",
407+
Key: "large-file.zip",
408+
Range: `bytes=0-${5 * 1024 * 1024 - 1}`,
409+
},
410+
{
411+
eventListeners: {
412+
transferInitiated: [downloadingKey],
413+
bytesTransferred: [
414+
{
415+
handleEvent: ({ request, snapshot }) => {
416+
const percent = snapshot.totalBytes ? (snapshot.transferredBytes / snapshot.totalBytes) * 100 : 0;
417+
console.log(`Progress: ${percent.toFixed(1)}%`);
418+
},
419+
},
420+
],
421+
transferComplete: [transferComplete],
422+
transferFailed: [transferFailed],
423+
},
424+
}
425+
);
426+
```
427+
428+
**Practical Example of Combining Both Levels:**
429+
430+
Because request-level listeners are added to client-level listeners (not replaced), it allows for global logging plus request-specific handling.
431+
432+
```js
433+
// Client-level: global logging
434+
const tm = new S3TransferManager({
435+
s3ClientInstance: s3Client,
436+
eventListeners: {
437+
transferInitiated: [
438+
{
439+
handleEvent: ({ request }) => {
440+
console.log(`Global: Started ${request.Key}`);
441+
},
442+
},
443+
],
444+
transferFailed: [globalErrorHandler],
445+
},
446+
});
447+
448+
// Request-level: specific progress tracking
449+
const download = await tm.download(
450+
{ Bucket: "my-bucket", Key: "video.mp4" },
451+
{
452+
eventListeners: {
453+
bytesTransferred: [videoProgressBar], // Added to global listeners
454+
transferComplete: [
455+
{
456+
handleEvent: ({ request, response }) => {
457+
console.log(`Video ${request.Key} completed with status ${response.$metadata?.httpStatusCode}`);
458+
},
459+
},
460+
],
461+
},
462+
}
463+
);
464+
```

0 commit comments

Comments
 (0)