Skip to content

Feature/add thumbnail support#133

Merged
tonyantony300 merged 23 commits intotonyantony300:mainfrom
WangJie-Mant:feature/add-thumbnail-support
Mar 7, 2026
Merged

Feature/add thumbnail support#133
tonyantony300 merged 23 commits intotonyantony300:mainfrom
WangJie-Mant:feature/add-thumbnail-support

Conversation

@WangJie-Mant
Copy link
Contributor

Description

This PR implements File Preview metadata fetching through a dedicated metadata protocol channel, without increasing ticket length.
To keep tickets compact, preview metadata (thumbnail/description/mime) is not encoded into the ticket itself.
Instead, the receiver parses the existing ticket for address resolution, then opens a metadata-specific connection to fetch length-prefixed JSON metadata.

metadata Transport (Core)

  • Added/used a dedicated ALPN channel (sendme/metadata/1) for preview metadata exchange.
  • Sender metadata handler now:
    • waits for a 1-byte metadata request marker (any bytes)
    • returns [4-byte length prefix] [JSON payload]
    • enforces metadata size bounds for safety
  • Receiver metadata flow now:
    • parses existing ticket
    • resolves sender endpoint address from ticket
    • connects via metadata ALPN
    • requests and parses metadata without starting file download

Tauri Commend Integration

  • fetch_ticket_metadata command now provides a direct metadata-only fetch path for frontend preview UI
  • Error/timeout paths return explicit command errors for UI handling

File Preview UX (Web)

  • Receiver preview now requests metadata from backend command and renders file preview card
  • Added loading state while metadata is being fetched
  • Keeps receive flow unchanged; preview is fetched before download starts.

Testing

  • Ticket generation remains address/hash/format based
  • No metadata is embedded into ticket payload, so ticket size not expanded by preview data.

Checklist

  • I have run npm run lint before raising this PR
  • I have run npm run format before raising this PR

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 4, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR introduces metadata exchange functionality to the file sharing system, enabling clients to fetch file metadata (name, size, thumbnail, MIME type) via a separate ALPN protocol channel, and adds platform-specific video thumbnail extraction capabilities with MIME-type-based routing across Windows, macOS, and Linux.

Changes

Cohort / File(s) Summary
Core metadata protocol
sendme/src/core/types.rs, sendme/src/core/receive.rs, sendme/src/core/send.rs, sendme/src/lib.rs
Added FileMetadata struct with serde support and integrated metadata fetch pathway using a dedicated ALPN channel. Introduced fetch_metadata() public function and metadata protocol handler; updated download() signature to accept AppHandle and emit metadata events.
Tauri dependencies & commands
src-tauri/Cargo.toml, src-tauri/src/commands.rs, src-tauri/src/lib.rs
Added dependencies for thumbnail generation (mime_guess, base64, image) and platform-specific video support (gstreamer on Linux, windows crate, objc on macOS). Enhanced start_sharing to compute and pass metadata; added new fetch_ticket_metadata command.
Thumbnail generation infrastructure
src-tauri/src/features/mod.rs, src-tauri/src/features/thumbnail/mod.rs, src-tauri/src/features/thumbnail/image.rs, src-tauri/src/features/thumbnail/mime.rs
Created thumbnail module with MIME-type detection routing to media-specific generators. Image thumbnail function generates 128x128 JPEG from image files with base64 encoding.
Platform-specific video thumbnail extraction
src-tauri/src/platform/mod.rs, src-tauri/src/platform/video_thumbnail/mod.rs, src-tauri/src/platform/video_thumbnail/linux.rs, src-tauri/src/platform/video_thumbnail/macos.rs, src-tauri/src/platform/video_thumbnail/windows.rs
Implemented platform-abstracted video frame extraction with GStreamer on Linux (fallback to FFmpeg), AVFoundation on macOS (fallback to FFmpeg), and FFmpeg on Windows. All paths generate 128x128 JPEG thumbnails at quality 70.
Frontend types & state
web-app/src/types/transfer.ts, web-app/src/types/receiver.ts, web-app/src/types/sender.ts, web-app/src/store/sender-store.ts
Added TicketPreviewMetadata type and thumbnailUrl field to transfer metadata. Extended receiver props with preview metadata and loading state; removed onFileSelect prop from sender ShareActionProps.
Frontend hooks
web-app/src/hooks/useReceiver.ts, web-app/src/hooks/useSender.ts, web-app/src/hooks/useDragDrop.ts
Added debounced metadata fetch with race-guard logic in useReceiver; updated handleFileSelect in useSender to async with optional pathType parameter; broadened onFileSelect callback signature in useDragDrop to support Promise-returning handlers.
Frontend components & UI
web-app/src/components/receiver/Receiver.tsx, web-app/src/components/receiver/TicketInput.tsx, web-app/src/components/sender/Sender.tsx, web-app/src/components/sender/ShareActionCard.tsx, web-app/src/components/icons.tsx, web-app/src/components/ui/menu.tsx
Enhanced TicketInput with preview panel displaying thumbnail, file name, and formatted size; added file icon rendering and thumbnail load fallback. Updated DragDrop callback signature. Refactored icons registry for explicit icon mapping. Minor layout and CSS selector adjustments.
Localization
web-app/src/locales/en/common.json, web-app/src/locales/zh-CN/common.json
Added optional "description" field to sender section in English and Chinese locale files.

Possibly related PRs

  • Feat/setting #121: Modifies the same web-app/src/components/icons.tsx module to introduce an explicit icon registry mapping, sharing the same refactoring pattern for icon resolution.

Suggested reviewers

  • tonyantony300
🚥 Pre-merge checks | ✅ 1
✅ Passed checks (1 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tonyantony300
Copy link
Owner

🔥

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 19

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
web-app/src/hooks/useSender.ts (1)

51-63: ⚠️ Potential issue | 🟠 Major

Reset description when selection is cleared to prevent stale metadata reuse.

With persistent fileDescription added, clearing a selected file currently leaves the old description in state, so a later share can inherit unintended text.

🧹 Proposed fix (outside this changed hunk)
 const clearSelectedPath = () => {
 	setSelectedPath(null)
 	setPathType(null)
+	setFileDescription('')
 }

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 321b976b-8648-4dce-a030-04d6ca75ea75

📥 Commits

Reviewing files that changed from the base of the PR and between da41438 and 13f69c9.

⛔ Files ignored due to path filters (1)
  • src-tauri/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (31)
  • sendme/src/core/receive.rs
  • sendme/src/core/send.rs
  • sendme/src/core/types.rs
  • sendme/src/lib.rs
  • src-tauri/Cargo.toml
  • src-tauri/src/commands.rs
  • src-tauri/src/features/mod.rs
  • src-tauri/src/features/thumbnail/image.rs
  • src-tauri/src/features/thumbnail/mime.rs
  • src-tauri/src/features/thumbnail/mod.rs
  • src-tauri/src/features/thumbnail/video.rs
  • src-tauri/src/main.backup.final.rs
  • src-tauri/src/platform/mod.rs
  • src-tauri/src/platform/video_thumbnail/linux.rs
  • src-tauri/src/platform/video_thumbnail/macos.rs
  • src-tauri/src/platform/video_thumbnail/mod.rs
  • src-tauri/src/platform/video_thumbnail/windows.rs
  • src-tauri/src/utils/base64.rs
  • web-app/src/components/receiver/Receiver.tsx
  • web-app/src/components/receiver/TicketInput.tsx
  • web-app/src/components/sender/Sender.tsx
  • web-app/src/components/sender/ShareActionCard.tsx
  • web-app/src/hooks/useReceiver.ts
  • web-app/src/hooks/useSender.ts
  • web-app/src/lib/tauri.ts
  • web-app/src/locales/en/common.json
  • web-app/src/locales/zh-CN/common.json
  • web-app/src/store/sender-store.ts
  • web-app/src/types/receiver.ts
  • web-app/src/types/sender.ts
  • web-app/src/types/transfer.ts

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (3)
web-app/src/hooks/useReceiver.ts (1)

144-194: ⚠️ Potential issue | 🟠 Major

Invalidate in-flight preview requests on receive/reset to prevent stale overwrites.

The sequence guard is good, but it is not invalidated when transitioning into receive/reset flows. A pending metadata request can still resolve and overwrite cleared preview state.

Suggested fix
 useEffect(() => {
+  const seq = ++previewRequestSeqRef.current
+
   if (isReceiving) {
+    setIsPreviewLoading(false)
     return
   }

   const trimmed = ticket.trim()
   if (!trimmed) {
     setPreviewMetadata(null)
     setIsPreviewLoading(false)
     return
   }

-  const seq = ++previewRequestSeqRef.current
   setIsPreviewLoading(true)
@@
 const handleReceive = async () => {
   if (!ticket.trim()) return

   try {
     setIsReceiving(true)
@@
+    previewRequestSeqRef.current += 1
     setPreviewMetadata(null)
+    setIsPreviewLoading(false)
     folderOpenTriggeredRef.current = false
@@
 const resetForNewTransfer = async () => {
+  previewRequestSeqRef.current += 1
   setIsReceiving(false)
@@
   setPreviewMetadata(null)
   setIsPreviewLoading(false)

Also applies to: 398-399, 414-425

src-tauri/src/commands.rs (2)

93-100: ⚠️ Potential issue | 🟠 Major

Offload metadata prep from async path and stop coercing size failures to 0.

Line 99 currently masks metadata size failures, and the directory walk + thumbnail generation run synchronously in the async command path.

🛠️ Proposed fix
-    let size = get_total_size(&path).unwrap_or(0);
-    let thumbnail = generate_thumbnail(&path);
+    let path_for_metadata = path.clone();
+    let (size_opt, thumbnail) = tokio::task::spawn_blocking(move || {
+        (get_total_size(&path_for_metadata), generate_thumbnail(&path_for_metadata))
+    })
+    .await
+    .map_err(|e| format!("Failed to prepare metadata: {e}"))?;
+    let size = size_opt
+        .ok_or_else(|| format!("Failed to calculate size for {}", path.display()))?;
#!/bin/bash
# Verify whether blocking metadata calls and silent fallback are still present.
rg -n --type=rust -C2 'get_total_size\(&path\)\.unwrap_or\(0\)|generate_thumbnail\(&path\)|spawn_blocking' src-tauri/src/commands.rs

120-126: ⚠️ Potential issue | 🟠 Major

Do not log raw user description text.

Line 125 and Line 174 still emit description content directly, which can leak sensitive user-provided text.

🔒 Safer logging pattern
     tracing::info!(
         path = %path.display(),
         file_name = %metadata.file_name,
         size = metadata.size,
         has_thumbnail = metadata.thumbnail.is_some(),
-        description = ?metadata.description,
+        has_description = metadata.description.as_ref().map(|d| !d.is_empty()).unwrap_or(false),
         "share metadata prepared"
     );
@@
             tracing::info!(
                 file_name = %metadata.file_name,
                 size = metadata.size,
                 has_thumbnail = metadata.thumbnail.is_some(),
-                description = ?metadata.description,
+                has_description = metadata.description.as_ref().map(|d| !d.is_empty()).unwrap_or(false),
                 "fetch_ticket_metadata succeeded"
             );

Also applies to: 170-175


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7a3e9135-c349-403a-a866-cab92176085e

📥 Commits

Reviewing files that changed from the base of the PR and between 13f69c9 and 67c0623.

⛔ Files ignored due to path filters (1)
  • src-tauri/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (4)
  • src-tauri/Cargo.toml
  • src-tauri/src/commands.rs
  • web-app/src/hooks/useReceiver.ts
  • web-app/src/hooks/useSender.ts

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (5)
src-tauri/src/commands.rs (1)

92-99: ⚠️ Potential issue | 🟠 Major

Move metadata preparation off the async command path, and fail instead of publishing 0 bytes.

get_total_size() walks the filesystem and generate_thumbnail() does synchronous media work. Running both inline here can stall the Tauri runtime, and unwrap_or(0) turns size failures into incorrect preview metadata. Prefer doing this work in spawn_blocking before the state mutex is held.

Suggested fix
-    let size = get_total_size(&path).unwrap_or(0);
-    let thumbnail = generate_thumbnail(&path);
+    let (size, thumbnail) = tokio::task::spawn_blocking({
+        let path = path.clone();
+        move || {
+            let size = get_total_size(&path)
+                .ok_or_else(|| format!("Failed to calculate size for {}", path.display()))?;
+            Ok::<_, String>((size, generate_thumbnail(&path)))
+        }
+    })
+    .await
+    .map_err(|e| e.to_string())??;
web-app/src/hooks/useSender.ts (1)

30-30: 🛠️ Refactor suggestion | 🟠 Major

Keep handleFileSelect’s exported signature aligned with the returned function.

The implementation still accepts an optional pathType and returns a Promise. Narrowing the public type to (path: string) => void hides that capability and forces an extra check_path_type round-trip when the caller already knows the type.

Suggested fix
-	handleFileSelect: (path: string) => void
+	handleFileSelect: (
+		path: string,
+		pathType?: 'file' | 'directory'
+	) => Promise<void>
sendme/src/core/send.rs (1)

59-70: ⚠️ Potential issue | 🟡 Minor

Reject invalid metadata request markers.

The handler reads one byte but never checks its value, so any client currently receives metadata without speaking the intended protocol.

Suggested fix
         let mut req = [0u8; 1];
         tokio::time::timeout(Duration::from_secs(10), recv_stream.read_exact(&mut req))
             .await
             .map_err(|_| {
                 AcceptError::from_err(std::io::Error::new(
                     ErrorKind::TimedOut,
                     "metadata request read timeout",
                 ))
             })?
             .map_err(AcceptError::from_err)?;
+        if req[0] != 1 {
+            return Err(AcceptError::from_err(std::io::Error::new(
+                ErrorKind::InvalidData,
+                format!("invalid metadata request marker: {}", req[0]),
+            )));
+        }
 
         tracing::debug!("metadata request marker received");
sendme/src/core/receive.rs (1)

351-362: ⚠️ Potential issue | 🟡 Minor

Plan a real third retry for direct-only tickets.

fetch_metadata says it retries up to 3 times, but attempt_plan only has two default attempts. Tickets without relay addresses therefore lose the last retry entirely.

Suggested fix
-    let mut attempt_plan: Vec<(usize, &'static str, iroh::EndpointAddr)> =
-        vec![(1, "default", addr.clone()), (2, "default", addr.clone())];
+    let mut attempt_plan: Vec<(usize, &'static str, iroh::EndpointAddr)> = vec![
+        (1, "default", addr.clone()),
+        (2, "default", addr.clone()),
+        (3, "default", addr.clone()),
+    ];
@@
-    if !relay_only_addr.addrs.is_empty() {
-        attempt_plan.push((3, "relay-only", relay_only_addr));
+    if !relay_only_addr.addrs.is_empty() {
+        attempt_plan[2] = (3, "relay-only", relay_only_addr);
     }
web-app/src/hooks/useReceiver.ts (1)

143-156: ⚠️ Potential issue | 🟠 Major

Invalidate in-flight preview requests when the ticket is cleared or receive starts.

previewRequestSeqRef only advances when a non-empty, non-receiving fetch begins. Once the timeout has fired, clearing the ticket, starting handleReceive, or resetting the transfer leaves the old sequence active, so a stale response can still repopulate previewMetadata after the UI has been cleared. Please bump the sequence and clear isPreviewLoading on those transitions.

Also applies to: 167-191, 389-397, 412-422


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 544d0835-d230-4bb8-ad9d-13cae0b43eb2

📥 Commits

Reviewing files that changed from the base of the PR and between 67c0623 and 0084c17.

📒 Files selected for processing (11)
  • sendme/src/core/receive.rs
  • sendme/src/core/send.rs
  • sendme/src/core/types.rs
  • src-tauri/src/commands.rs
  • web-app/src/components/receiver/TicketInput.tsx
  • web-app/src/components/sender/Sender.tsx
  • web-app/src/hooks/useReceiver.ts
  • web-app/src/hooks/useSender.ts
  • web-app/src/store/sender-store.ts
  • web-app/src/types/sender.ts
  • web-app/src/types/transfer.ts
💤 Files with no reviewable changes (2)
  • web-app/src/types/sender.ts
  • web-app/src/components/sender/Sender.tsx

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
web-app/src/components/sender/DragDrop.tsx (1)

35-39: 🧹 Nitpick | 🔵 Trivial

Consider memoizing checkPathType to avoid effect re-runs.

The checkPathType function is included in the dependency array, but since it's recreated on each render of useDragDrop, this effect may run more often than intended. If this causes issues, consider wrapping checkPathType in useCallback within the hook.

♻️ Duplicate comments (1)
src-tauri/src/commands.rs (1)

305-323: ⚠️ Potential issue | 🟠 Major

Don't return partial directory sizes as success.

WalkDir and entry.metadata() failures are skipped here, so unreadable descendants silently shrink the size that gets sent in FileMetadata. Please fail fast on the first traversal/stat error instead of returning a partial total.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f853b7a8-28d0-44bc-963f-76ee47ae2a0c

📥 Commits

Reviewing files that changed from the base of the PR and between 72b72b4 and 6f6af38.

📒 Files selected for processing (21)
  • sendme/src/core/receive.rs
  • sendme/src/core/send.rs
  • src-tauri/Cargo.toml
  • src-tauri/src/commands.rs
  • src-tauri/src/features/thumbnail/image.rs
  • src-tauri/src/features/thumbnail/mod.rs
  • src-tauri/src/features/thumbnail/video.rs
  • src-tauri/src/lib.rs
  • src-tauri/src/platform/video_thumbnail/linux.rs
  • src-tauri/src/platform/video_thumbnail/macos.rs
  • src-tauri/src/platform/video_thumbnail/mod.rs
  • src-tauri/src/platform/video_thumbnail/windows.rs
  • web-app/src/components/icons.tsx
  • web-app/src/components/receiver/TicketInput.tsx
  • web-app/src/components/sender/DragDrop.tsx
  • web-app/src/components/ui/menu.tsx
  • web-app/src/hooks/useDragDrop.ts
  • web-app/src/hooks/useReceiver.ts
  • web-app/src/hooks/useSender.ts
  • web-app/src/locales/en/common.json
  • web-app/src/types/transfer.ts

Copy link
Owner

@tonyantony300 tonyantony300 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please handle warnings

@tonyantony300 tonyantony300 merged commit 536e024 into tonyantony300:main Mar 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants