Skip to content

SFTP: Stream file transfers instead of buffering entire file in memory #239

@BrianLeishman

Description

@BrianLeishman

Summary

SFTP file operations (download_file_from_sftp, upload_file_to_sftp, copy) read entire files into a Vec<u8>, which will OOM on large files.

Current State

  • src-tauri/src/locations/sftp/mod.rs:385sftp.read(remote_path) loads full file
  • src-tauri/src/locations/sftp/mod.rs:408tokio::fs::read(local_path) loads full file
  • src-tauri/src/locations/sftp/mod.rs:268copy() reads full file into memory
  • Same pattern exists in SMB provider (via sidecar)

Proposed Changes

Use russh-sftp's open() + chunked read()/write() instead of the convenience read()/write() methods:

let file = sftp.open(path).await?;
let mut buf = vec![0u8; 64 * 1024]; // 64KB chunks
loop {
    let n = file.read(&mut buf).await?;
    if n == 0 { break; }
    dest.write_all(&buf[..n]).await?;
}

Acceptance Criteria

  • File transfers use constant memory regardless of file size
  • Downloads stream to disk in chunks
  • Uploads stream from disk in chunks
  • Copy streams through a temp file without full buffering
  • Progress reporting for large transfers (nice-to-have)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions