Skip to content

fdenis75/MosaicKit

BranchesTags

Repository files navigation

MosaicKit

A high-performance Swift package for generating video mosaics with Metal-accelerated image processing. Extract frames from videos and arrange them into beautiful, customizable mosaic layouts with optional metadata headers.

Platform Swift License

Features

  • 🚀 Metal-Accelerated Processing - Hardware-accelerated mosaic generation for maximum performance
  • 🎨 Multiple Layout Algorithms - Classic, custom, auto-screen, dynamic, and iPhone-optimized layouts
  • ⚙️ Configurable Density Levels - From XXL (minimal) to XXS (maximal) frame extraction
  • 📦 Multiple Output Formats - JPEG, PNG, and HEIF with configurable compression
  • 🔄 Batch Processing - Intelligent concurrency management for processing multiple videos
  • 🎯 Hardware-Accelerated Frame Extraction - Uses VideoToolbox for optimal performance
  • 📊 Metadata Headers - Optional metadata overlay with video information
  • 🎬 Video Preview Generation - Create short highlight reels from any video, either exported to file or as a live AVPlayerItem composition

Requirements

  • macOS 15.0+ or iOS 15.0+
  • Xcode 16.0+
  • Swift 6.2+
  • Metal-capable device

Installation

Swift Package Manager

Add MosaicKit to your Package.swift:

dependencies: [
    .package(url: "https://github.com/fdenis75/MosaicKit.git", from: "1.0.0")
]

Then add it to your target dependencies:

targets: [
    .target(
        name: "YourTarget",
        dependencies: ["MosaicKit"]
    )
]

Xcode

  1. Go to File → Add Package Dependencies...
  2. Enter the repository URL
  3. Select the version you want to use
  4. Click Add Package

Quick Start

Simple API (Recommended)

import MosaicKit

// Simple one-step generation
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let outputDir = URL(fileURLWithPath: "/path/to/output")

let generator = try MosaicGenerator()
let config = MosaicConfiguration.default

let mosaicURL = try await generator.generate(
    from: videoURL,
    config: config,
    outputDirectory: outputDir
)

print("Mosaic saved to: \(mosaicURL.path)")

Advanced Usage (Direct Access)

For more control, use MetalMosaicGenerator directly:

import MosaicKit

// Create a video input from a file URL
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let video = try await VideoInput(from: videoURL)

// Configure mosaic settings
var config = MosaicConfiguration.default
config.width = 5000
config.density = .m
config.format = .heif
config.outputdirectory = URL(fileURLWithPath: "/path/to/output")

// Generate the mosaic
let generator = try MetalMosaicGenerator()
let mosaicURL = try await generator.generate(
    for: video,
    config: config
)

print("Mosaic saved to: \(mosaicURL.path)")

Configuration Options

MosaicConfiguration

public struct MosaicConfiguration {
    var width: Int                      // Output width (default: 5120)
    var density: DensityConfig          // Frame density (default: .m)
    var format: OutputFormat            // JPEG, PNG, or HEIF
    var layout: LayoutConfiguration     // Layout settings
    var includeMetadata: Bool           // Add metadata header
    var useAccurateTimestamps: Bool     // Precise frame extraction
    var compressionQuality: Double      // 0.0 to 1.0 (default: 0.4)
    var outputdirectory: URL?           // Output directory
}

Density Levels

Control the number of frames extracted from your video:

// Fewer frames (faster processing, smaller file)
config.density = .xxl  // Minimal - ~25% of base calculation
config.density = .xl   // Low - ~50% of base calculation
config.density = .l    // Medium - ~75% of base calculation

// More frames (slower processing, larger file, more detail)
config.density = .m    // High (default) - 100% of base calculation
config.density = .s    // Very high - 200% of base calculation
config.density = .xs   // Super high - 300% of base calculation
config.density = .xxs  // Maximal - 400% of base calculation

Layout Options

Choose from multiple layout algorithms:

var layout = LayoutConfiguration()

// Aspect ratios
layout.aspectRatio = .widescreen  // 16:9
layout.aspectRatio = .standard    // 4:3
layout.aspectRatio = .square      // 1:1
layout.aspectRatio = .ultrawide   // 21:9
layout.aspectRatio = .vertical    // 9:16 (portrait)

// Layout modes
layout.useCustomLayout = true     // Three-zone layout with large center thumbnails
layout.useAutoLayout = true       // Adapt to screen size
// Or use classic grid layout (default)

// Visual settings
layout.visual.addBorder = true
layout.visual.borderColor = .white
layout.visual.borderWidth = 2.0
layout.visual.addShadow = true

Output Formats

// HEIF - Best compression, smaller file size (recommended)
config.format = .heif
config.compressionQuality = 0.4

// JPEG - Good compression, universal compatibility
config.format = .jpeg
config.compressionQuality = 0.8

// PNG - Lossless, larger file size
config.format = .png

Advanced Usage

Batch Processing

Process multiple videos with intelligent concurrency:

let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 4
)

let videos: [VideoInput] = [video1, video2, video3]

let results = try await coordinator.generateMosaicsforbatch(
    videos: videos,
    config: config
) { progress in
    print("Video: \(progress.video.title)")
    print("Progress: \(Int(progress.progress * 100))%")
    print("Status: \(progress.status)")
}

// Check results
for result in results {
    if result.isSuccess {
        print("✅ Success: \(result.outputURL?.path ?? "unknown")")
    } else {
        print("❌ Failed: \(result.error?.localizedDescription ?? "unknown")")
    }
}

Progress Tracking

Monitor generation progress in real-time using MosaicGeneratorCoordinator:

let result = try await coordinator.generateMosaic(
    for: video,
    config: config
) { progress in
    // progress.progress is 0.0–1.0
    // progress.status indicates the current phase
    print("Progress: \(Int(progress.progress * 100))% - \(progress.status)")
}

Custom Video Input

Create VideoInput manually with specific metadata:

let video = VideoInput(
    url: videoURL,
    title: "My Video",
    duration: 120.0,
    width: 1920,
    height: 1080,
    frameRate: 30.0,
    fileSize: 50_000_000,
    metadata: VideoMetadata(
        codec: "H.264",
        bitrate: 5_000_000
    )
)

Performance Metrics

Track generator performance:

let metrics = await generator.getPerformanceMetrics()
print("Average generation time: \(metrics["averageGenerationTime"] ?? 0)")
print("Total generations: \(metrics["generationCount"] ?? 0)")

Cancellation

Cancel ongoing operations:

// Cancel specific video
await generator.cancel(for: video)

// Cancel all operations
await generator.cancelAll()

// Or cancel batch operations
await coordinator.cancelAllGenerations()

Video Preview Generation

MosaicKit can generate short highlight-reel previews from any video. A preview stitches together evenly-distributed clips from across the video into a single condensed output.

Two delivery modes are available:

Mode API Use case
Export to file PreviewVideoGenerator.generate(for:config:) Share, upload, or store the preview
Composition PreviewVideoGenerator.generateComposition(for:config:) Instant playback in AVPlayer — no file written

Basic Preview Export

import MosaicKit

let video = try await VideoInput(from: URL(fileURLWithPath: "/path/to/video.mp4"))

let config = PreviewConfiguration(
    targetDuration: 60,          // ~60-second preview
    density: .m,                 // 16 clips
    format: .mp4,
    includeAudio: true,
    outputDirectory: URL(fileURLWithPath: "/path/to/output"),
    compressionQuality: 0.8
)

let generator = PreviewVideoGenerator()
let previewURL = try await generator.generate(for: video, config: config)
print("Preview saved to: \(previewURL.path)")

Instant Composition (No Export)

Generate a ready-to-play AVPlayerItem without writing any file — significantly faster than exporting:

let playerItem = try await generator.generateComposition(for: video, config: config)

let player = AVPlayer(playerItem: playerItem)
player.play()

Preview Configuration

public struct PreviewConfiguration {
    var targetDuration: TimeInterval  // Target preview length in seconds
    var density: DensityConfig        // Number of clips (same levels as mosaic)
    var format: VideoFormat           // .mp4, .mov, .hevc, etc.
    var includeAudio: Bool            // Include audio track in preview
    var outputDirectory: URL?         // Output folder (nil = video's parent directory)
    var fullPathInName: Bool          // Embed full source path in filename
    var compressionQuality: Double    // 0.0–1.0
    var useNativeExport: Bool         // AVAssetExportSession vs SJSAssetExportSession
}

Clip count scales automatically with video duration. Use extractCount(forVideoDuration:) to inspect the calculated value:

let count = config.extractCount(forVideoDuration: video.duration)
print("Clips to extract: \(count)")

Resolution Cap (macOS 26+ / iOS 26+)

On macOS 26 and iOS 26, you can cap the output resolution to reduce file size:

if #available(macOS 26, iOS 26, *) {
    config.exportMaxResolution = ._1080p   // or ._720p, ._4K, etc.
}

On earlier OS versions the setting is silently ignored and the full source resolution is used.

Preview with Progress Tracking

let coordinator = PreviewGeneratorCoordinator()

let playerItem = try await coordinator.generatePreviewComposition(
    for: video,
    config: config
) { progress in
    print("\(Int(progress.progress * 100))% — \(progress.status.displayLabel)")
}

Batch Preview Generation

let coordinator = PreviewGeneratorCoordinator(concurrencyLimit: 2)

let results = try await coordinator.generatePreviewCompositionsForBatch(
    videos: [video1, video2, video3],
    config: config
) { progress in
    print("\(progress.video.filename): \(progress.status.displayLabel)")
}

let succeeded = results.filter(\.isSuccess)
print("Generated \(succeeded.count)/\(results.count) previews")

if let first = succeeded.first, let playerItem = first.playerItem {
    AVPlayer(playerItem: playerItem).play()
}

Layout Algorithm Details

Custom Layout (Recommended)

Three-zone layout with small thumbnails at top/bottom and large thumbnails in the center:

config.layout.useCustomLayout = true
// Automatically calculates optimal grid based on:
// - Target aspect ratio
// - Video aspect ratio
// - Thumbnail count
// - Density settings

Classic Layout

Traditional grid layout with uniform thumbnail sizes:

config.layout.useCustomLayout = false
config.layout.useAutoLayout = false
// Simple rows × columns grid

Auto Layout

Adapts to your display size for optimal viewing:

config.layout.useAutoLayout = true
// Calculates based on:
// - Screen resolution
// - DPI/scaling factor
// - Minimum readable thumbnail size

Dynamic Layout

Center-emphasized layout with variable thumbnail sizes:

config.layout.useDynamicLayout = true
// Larger thumbnails in center, smaller at edges

Examples

Example 1: High-Quality Mosaic

var config = MosaicConfiguration(
    width: 10000,
    density: .xs,
    format: .heif,
    layout: .default,
    includeMetadata: true,
    useAccurateTimestamps: true,
    compressionQuality: 0.6
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 2: Fast Preview Mosaic

var config = MosaicConfiguration(
    width: 2000,
    density: .xl,
    format: .jpeg,
    layout: .default,
    includeMetadata: false,
    useAccurateTimestamps: false,
    compressionQuality: 0.4
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 3: Square Social Media Mosaic

var config = MosaicConfiguration.default
config.width = 3000
config.layout.aspectRatio = .square
config.density = .m
config.format = .jpeg
config.compressionQuality = 0.8
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Performance Tips

  1. Use HEIF format - Best compression with good quality
  2. Start with medium density - Adjust based on video length
  3. Disable accurate timestamps for faster processing when precision isn't critical
  4. Use batch processing for multiple videos to leverage concurrency
  5. Consider screen size - Match output width to your display for optimal viewing
  6. Monitor memory usage - Very high densities or large widths can use significant memory

Frame Extraction Strategy

MosaicKit uses an intelligent frame extraction strategy:

  • Skips first 5% and last 5% of video (avoid fade in/out)
  • First third: 20% of frames (opening scenes)
  • Middle third: 60% of frames (main content)
  • Last third: 20% of frames (ending)
  • Hardware accelerated using VideoToolbox
  • Concurrent extraction based on available CPU cores

Error Handling

do {
    let mosaicURL = try await generator.generate(for: video, config: config)
} catch MosaicError.metalNotSupported {
    print("Metal is not available on this device")
} catch MosaicError.invalidVideo(let message) {
    print("Invalid video: \(message)")
} catch MosaicError.layoutCreationFailed(let error) {
    print("Layout creation failed: \(error)")
} catch MosaicError.saveFailed(let url, let error) {
    print("Failed to save mosaic to \(url): \(error)")
} catch {
    print("Unexpected error: \(error)")
}

System Requirements for Best Performance

  • Apple Silicon (M1/M2/M3) - Optimal performance with unified memory
  • 16GB+ RAM - For processing large videos or high densities
  • Intel Mac with dedicated GPU - Good performance with AMD/NVIDIA GPUs
  • Fast SSD - For quick frame extraction and mosaic saving

Concurrency Management

Batch processing automatically adjusts concurrency based on:

  • CPU cores: max(2, processorCount - 1)
  • Available memory: max(2, physicalMemory / 4GB)
  • Final limit: min(cpu_limit, memory_limit, configured_limit)
// Configure custom concurrency limit
let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 8  // Max 8 videos processed simultaneously
)

Troubleshooting

"Metal is not supported"

  • Ensure you're running on a Metal-capable device
  • Check minimum OS requirements (macOS 15+ / iOS 15+)

Out of memory errors

  • Reduce mosaic width
  • Lower density setting
  • Process videos in smaller batches
  • Close other applications

Slow processing

  • Check if accurate timestamps are needed (slower but more precise)
  • Verify Metal is being used (check device capabilities)
  • Consider reducing frame count for long videos
  • Use batch processing for multiple videos

Quality issues

  • Increase compression quality (0.6-0.8 for HEIF/JPEG)
  • Use higher density settings
  • Increase output width
  • Try PNG format for lossless output

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

  • Built with Swift 6's modern concurrency features
  • Uses Metal for GPU-accelerated processing
  • VideoToolbox for hardware-accelerated frame extraction
  • swift-log for structured logging
  • DominantColors for color analysis

Support

For issues, questions, or feature requests, please open an issue on GitHub.

About

No description, website, or topics provided.

Resources

Readme

License

Apache-2.0 license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

1.0.0 Latest
Feb 23, 2026

Packages

 
 
 

Contributors

Languages