Migration Tooling - 10th April - Storacha Migration Tool - PLDG 2

[leny62]

UI Progress Update & Upcoming Features

Current Progress

Space Management

We're implementing space listing and management using the existing library functions:

async listSpaces(): Promise<SpaceResponse[]> {
  // Returns array of spaces with { success, did, name }
}

Planned UI Features:

• Space selector dropdown in the migration form
• Space management dashboard
• Create/delete space functionality
• Space usage statistics

File Browser

Implementing file browsing capabilities using:

async listFilesInSpace(did: string): Promise<UploadListItem[]> {
  // Returns array of uploaded files in a space
}

Planned Features:

• File list view with sorting and filtering
• File metadata display (CID, URL, size, upload date)
• Quick actions (copy CID, open URL)
• Batch operations support

UI Evolution

We've made significant progress on the migration tool's UI, evolving from separate download/upload components to a more integrated solution. Here's what we've accomplished:

1. Initial Version (Separated Components)

   • S3Downloader: Handled S3 file downloads with progress tracking
   • StorachaUploader: Managed Storacha uploads independently
   • Issue Identified: This separation led to incorrect file data handling, resulting in empty files being uploaded (missing CID/URLs)

2. Current Version (Integrated Flow)

   • Combined workflow in StorachaMigrator component
   • Real-time progress tracking with detailed status updates
   • Improved error handling and user feedback
   • Storacha branding integration with custom color scheme

UI Features

• ✅ S3 Configuration with secure credential handling
• ✅ Real-time progress tracking with percentage, speed, and ETA
• ✅ Error handling with user-friendly messages
• ✅ Success states with CID and URL display
• ✅ Shard progress tracking for large files

Demo

Link

Upcoming Features

TBD

Current Issues & Solutions

1. Empty File Data Issue

Problem: The initial separated component approach resulted in incorrect file data being sent to Storacha.

Solution:

• Implemented proper file metadata handling
• Added validation checks before upload
• Integrated S3 and Storacha operations for seamless data transfer

2. Progress Tracking Improvements

• Added detailed phase tracking (initializing, authenticating, preparing, uploading, finalizing)
• Implemented shard-level progress tracking
• Added speed and time remaining calculations

Example Implementation

The library provides a robust implementation that we're integrating into the UI:

const migrator = new StorachaMigrator({
  s3: {
    bucketName: 'my-bucket',
    region: 'us-east-1',
    credentials: {
      accessKeyId: '...',
      secretAccessKey: '...'
    }
  },
  storacha: {
    email: '[email protected]'
  },
  retry: {
    maxAttempts: 3,
    backoffMs: 1000,
    maxBackoffMs: 10000
  },
  batch: {
    concurrency: 5,
    size: 10
  }
});

// Space listing
const spaces = await migrator.listSpaces();

// File listing
const files = await migrator.listFilesInSpace(spaceId);

Next Steps

1. Create a 100% working UI example and can add more features afterwards

Looking forward to your feedback and suggestions on these upcoming features!

[leny62]

Github repo: https://github.com/[HarshS1611/storacha-migration-tool](https://github.com/HarshS1611/storacha-migration-tool)