MosaicKit

A high-performance Swift package for generating video mosaics with Metal-accelerated image processing. Extract frames from videos and arrange them into beautiful, customizable mosaic layouts with optional metadata headers.

Features

🚀 Metal-Accelerated Processing - Hardware-accelerated mosaic generation for maximum performance
🎨 Multiple Layout Algorithms - Classic, custom, auto-screen, dynamic, and iPhone-optimized layouts
⚙️ Configurable Density Levels - From XXL (minimal) to XXS (maximal) frame extraction
📦 Multiple Output Formats - JPEG, PNG, and HEIF with configurable compression
🔄 Batch Processing - Intelligent concurrency management for processing multiple videos
🎯 Hardware-Accelerated Frame Extraction - Uses VideoToolbox for optimal performance
📊 Metadata Headers - Optional metadata overlay with video information
🎬 Video Preview Generation - Create short highlight reels from any video, either exported to file or as a live AVPlayerItem composition

Requirements

macOS 15.0+ or iOS 15.0+
Xcode 16.0+
Swift 6.2+
Metal-capable device

Installation

Swift Package Manager

Add MosaicKit to your Package.swift:

dependencies: [
    .package(url: "https://github.com/fdenis75/MosaicKit.git", from: "1.0.0")
]

Then add it to your target dependencies:

targets: [
    .target(
        name: "YourTarget",
        dependencies: ["MosaicKit"]
    )
]

Xcode

Go to File → Add Package Dependencies...
Enter the repository URL
Select the version you want to use
Click Add Package

Quick Start

Simple API (Recommended)

import MosaicKit

// Simple one-step generation
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let outputDir = URL(fileURLWithPath: "/path/to/output")

let generator = try MosaicGenerator()
let config = MosaicConfiguration.default

let mosaicURL = try await generator.generate(
    from: videoURL,
    config: config,
    outputDirectory: outputDir
)

print("Mosaic saved to: \(mosaicURL.path)")

Advanced Usage (Direct Access)

For more control, use MetalMosaicGenerator directly:

import MosaicKit

// Create a video input from a file URL
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let video = try await VideoInput(from: videoURL)

// Configure mosaic settings
var config = MosaicConfiguration.default
config.width = 5000
config.density = .m
config.format = .heif
config.outputdirectory = URL(fileURLWithPath: "/path/to/output")

// Generate the mosaic
let generator = try MetalMosaicGenerator()
let mosaicURL = try await generator.generate(
    for: video,
    config: config
)

print("Mosaic saved to: \(mosaicURL.path)")

Configuration Options

MosaicConfiguration

public struct MosaicConfiguration {
    var width: Int                      // Output width (default: 5120)
    var density: DensityConfig          // Frame density (default: .m)
    var format: OutputFormat            // JPEG, PNG, or HEIF
    var layout: LayoutConfiguration     // Layout settings
    var includeMetadata: Bool           // Add metadata header
    var useAccurateTimestamps: Bool     // Precise frame extraction
    var compressionQuality: Double      // 0.0 to 1.0 (default: 0.4)
    var outputdirectory: URL?           // Output directory
}

Density Levels

Control the number of frames extracted from your video:

// Fewer frames (faster processing, smaller file)
config.density = .xxl  // Minimal - ~25% of base calculation
config.density = .xl   // Low - ~50% of base calculation
config.density = .l    // Medium - ~75% of base calculation

// More frames (slower processing, larger file, more detail)
config.density = .m    // High (default) - 100% of base calculation
config.density = .s    // Very high - 200% of base calculation
config.density = .xs   // Super high - 300% of base calculation
config.density = .xxs  // Maximal - 400% of base calculation

Layout Options

Choose from multiple layout algorithms:

var layout = LayoutConfiguration()

// Aspect ratios
layout.aspectRatio = .widescreen  // 16:9
layout.aspectRatio = .standard    // 4:3
layout.aspectRatio = .square      // 1:1
layout.aspectRatio = .ultrawide   // 21:9
layout.aspectRatio = .vertical    // 9:16 (portrait)

// Layout modes
layout.useCustomLayout = true     // Three-zone layout with large center thumbnails
layout.useAutoLayout = true       // Adapt to screen size
// Or use classic grid layout (default)

// Visual settings
layout.visual.addBorder = true
layout.visual.borderColor = .white
layout.visual.borderWidth = 2.0
layout.visual.addShadow = true

Output Formats

// HEIF - Best compression, smaller file size (recommended)
config.format = .heif
config.compressionQuality = 0.4

// JPEG - Good compression, universal compatibility
config.format = .jpeg
config.compressionQuality = 0.8

// PNG - Lossless, larger file size
config.format = .png

Advanced Usage

Batch Processing

Process multiple videos with intelligent concurrency:

let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 4
)

let videos: [VideoInput] = [video1, video2, video3]

let results = try await coordinator.generateMosaicsforbatch(
    videos: videos,
    config: config
) { progress in
    print("Video: \(progress.video.title)")
    print("Progress: \(Int(progress.progress * 100))%")
    print("Status: \(progress.status)")
}

// Check results
for result in results {
    if result.isSuccess {
        print("✅ Success: \(result.outputURL?.path ?? "unknown")")
    } else {
        print("❌ Failed: \(result.error?.localizedDescription ?? "unknown")")
    }
}

Progress Tracking

Monitor generation progress in real-time using MosaicGeneratorCoordinator:

let result = try await coordinator.generateMosaic(
    for: video,
    config: config
) { progress in
    // progress.progress is 0.0–1.0
    // progress.status indicates the current phase
    print("Progress: \(Int(progress.progress * 100))% - \(progress.status)")
}

Custom Video Input

Create VideoInput manually with specific metadata:

let video = VideoInput(
    url: videoURL,
    title: "My Video",
    duration: 120.0,
    width: 1920,
    height: 1080,
    frameRate: 30.0,
    fileSize: 50_000_000,
    metadata: VideoMetadata(
        codec: "H.264",
        bitrate: 5_000_000
    )
)

Performance Metrics

Track generator performance:

let metrics = await generator.getPerformanceMetrics()
print("Average generation time: \(metrics["averageGenerationTime"] ?? 0)")
print("Total generations: \(metrics["generationCount"] ?? 0)")

Cancellation

Cancel ongoing operations:

// Cancel specific video
await generator.cancel(for: video)

// Cancel all operations
await generator.cancelAll()

// Or cancel batch operations
await coordinator.cancelAllGenerations()

Video Preview Generation

MosaicKit can generate short highlight-reel previews from any video. A preview stitches together evenly-distributed clips from across the video into a single condensed output.

Two delivery modes are available:

Mode	API	Use case
Export to file	`PreviewVideoGenerator.generate(for:config:)`	Share, upload, or store the preview
Composition	`PreviewVideoGenerator.generateComposition(for:config:)`	Instant playback in `AVPlayer` — no file written

Basic Preview Export

import MosaicKit

let video = try await VideoInput(from: URL(fileURLWithPath: "/path/to/video.mp4"))

let config = PreviewConfiguration(
    targetDuration: 60,          // ~60-second preview
    density: .m,                 // 16 clips
    format: .mp4,
    includeAudio: true,
    outputDirectory: URL(fileURLWithPath: "/path/to/output"),
    compressionQuality: 0.8
)

let generator = PreviewVideoGenerator()
let previewURL = try await generator.generate(for: video, config: config)
print("Preview saved to: \(previewURL.path)")

Instant Composition (No Export)

Generate a ready-to-play AVPlayerItem without writing any file — significantly faster than exporting:

let playerItem = try await generator.generateComposition(for: video, config: config)

let player = AVPlayer(playerItem: playerItem)
player.play()

Preview Configuration

public struct PreviewConfiguration {
    var targetDuration: TimeInterval  // Target preview length in seconds
    var density: DensityConfig        // Number of clips (same levels as mosaic)
    var format: VideoFormat           // .mp4, .mov, .hevc, etc.
    var includeAudio: Bool            // Include audio track in preview
    var outputDirectory: URL?         // Output folder (nil = video's parent directory)
    var fullPathInName: Bool          // Embed full source path in filename
    var compressionQuality: Double    // 0.0–1.0
    var useNativeExport: Bool         // AVAssetExportSession vs SJSAssetExportSession
}

Clip count scales automatically with video duration. Use extractCount(forVideoDuration:) to inspect the calculated value:

let count = config.extractCount(forVideoDuration: video.duration)
print("Clips to extract: \(count)")

Resolution Cap (macOS 26+ / iOS 26+)

On macOS 26 and iOS 26, you can cap the output resolution to reduce file size:

if #available(macOS 26, iOS 26, *) {
    config.exportMaxResolution = ._1080p   // or ._720p, ._4K, etc.
}

On earlier OS versions the setting is silently ignored and the full source resolution is used.

Preview with Progress Tracking

let coordinator = PreviewGeneratorCoordinator()

let playerItem = try await coordinator.generatePreviewComposition(
    for: video,
    config: config
) { progress in
    print("\(Int(progress.progress * 100))% — \(progress.status.displayLabel)")
}

Batch Preview Generation

let coordinator = PreviewGeneratorCoordinator(concurrencyLimit: 2)

let results = try await coordinator.generatePreviewCompositionsForBatch(
    videos: [video1, video2, video3],
    config: config
) { progress in
    print("\(progress.video.filename): \(progress.status.displayLabel)")
}

let succeeded = results.filter(\.isSuccess)
print("Generated \(succeeded.count)/\(results.count) previews")

if let first = succeeded.first, let playerItem = first.playerItem {
    AVPlayer(playerItem: playerItem).play()
}

Layout Algorithm Details

Custom Layout (Recommended)

Three-zone layout with small thumbnails at top/bottom and large thumbnails in the center:

config.layout.useCustomLayout = true
// Automatically calculates optimal grid based on:
// - Target aspect ratio
// - Video aspect ratio
// - Thumbnail count
// - Density settings

Classic Layout

Traditional grid layout with uniform thumbnail sizes:

config.layout.useCustomLayout = false
config.layout.useAutoLayout = false
// Simple rows × columns grid

Auto Layout

Adapts to your display size for optimal viewing:

config.layout.useAutoLayout = true
// Calculates based on:
// - Screen resolution
// - DPI/scaling factor
// - Minimum readable thumbnail size

Dynamic Layout

Center-emphasized layout with variable thumbnail sizes:

config.layout.useDynamicLayout = true
// Larger thumbnails in center, smaller at edges

Examples

Example 1: High-Quality Mosaic

var config = MosaicConfiguration(
    width: 10000,
    density: .xs,
    format: .heif,
    layout: .default,
    includeMetadata: true,
    useAccurateTimestamps: true,
    compressionQuality: 0.6
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 2: Fast Preview Mosaic

var config = MosaicConfiguration(
    width: 2000,
    density: .xl,
    format: .jpeg,
    layout: .default,
    includeMetadata: false,
    useAccurateTimestamps: false,
    compressionQuality: 0.4
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 3: Square Social Media Mosaic

var config = MosaicConfiguration.default
config.width = 3000
config.layout.aspectRatio = .square
config.density = .m
config.format = .jpeg
config.compressionQuality = 0.8
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Performance Tips

Use HEIF format - Best compression with good quality
Start with medium density - Adjust based on video length
Disable accurate timestamps for faster processing when precision isn't critical
Use batch processing for multiple videos to leverage concurrency
Consider screen size - Match output width to your display for optimal viewing
Monitor memory usage - Very high densities or large widths can use significant memory

Frame Extraction Strategy

MosaicKit uses an intelligent frame extraction strategy:

Skips first 5% and last 5% of video (avoid fade in/out)
First third: 20% of frames (opening scenes)
Middle third: 60% of frames (main content)
Last third: 20% of frames (ending)
Hardware accelerated using VideoToolbox
Concurrent extraction based on available CPU cores

Error Handling

do {
    let mosaicURL = try await generator.generate(for: video, config: config)
} catch MosaicError.metalNotSupported {
    print("Metal is not available on this device")
} catch MosaicError.invalidVideo(let message) {
    print("Invalid video: \(message)")
} catch MosaicError.layoutCreationFailed(let error) {
    print("Layout creation failed: \(error)")
} catch MosaicError.saveFailed(let url, let error) {
    print("Failed to save mosaic to \(url): \(error)")
} catch {
    print("Unexpected error: \(error)")
}

System Requirements for Best Performance

Apple Silicon (M1/M2/M3) - Optimal performance with unified memory
16GB+ RAM - For processing large videos or high densities
Intel Mac with dedicated GPU - Good performance with AMD/NVIDIA GPUs
Fast SSD - For quick frame extraction and mosaic saving

Concurrency Management

Batch processing automatically adjusts concurrency based on:

CPU cores: max(2, processorCount - 1)
Available memory: max(2, physicalMemory / 4GB)
Final limit: min(cpu_limit, memory_limit, configured_limit)

// Configure custom concurrency limit
let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 8  // Max 8 videos processed simultaneously
)

Troubleshooting

"Metal is not supported"

Ensure you're running on a Metal-capable device
Check minimum OS requirements (macOS 15+ / iOS 15+)

Out of memory errors

Reduce mosaic width
Lower density setting
Process videos in smaller batches
Close other applications

Slow processing

Check if accurate timestamps are needed (slower but more precise)
Verify Metal is being used (check device capabilities)
Consider reducing frame count for long videos
Use batch processing for multiple videos

Quality issues

Increase compression quality (0.6-0.8 for HEIF/JPEG)
Use higher density settings
Increase output width
Try PNG format for lossless output

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Built with Swift 6's modern concurrency features
Uses Metal for GPU-accelerated processing
VideoToolbox for hardware-accelerated frame extraction
swift-log for structured logging
DominantColors for color analysis

Support

For issues, questions, or feature requests, please open an issue on GitHub.

FilesExpand file tree

README.md

Latest commit

History