Skip to content

Latest commit

 

History

History
338 lines (249 loc) · 8.96 KB

hub_integration.md

File metadata and controls

338 lines (249 loc) · 8.96 KB

HuggingFace Hub Integration for RuvLTRA

This document describes the HuggingFace Hub integration for publishing and downloading RuvLTRA models.

Overview

The ruvllm::hub module provides comprehensive functionality for:

  1. Model Download: Pull GGUF files from HuggingFace Hub with progress tracking and resume support
  2. Model Upload: Push models to HuggingFace Hub with automatic model card generation
  3. Model Registry: Pre-configured RuvLTRA model collection with hardware requirements
  4. Progress Tracking: Visual progress bars with ETA and speed indicators
  5. Checksum Verification: SHA256 validation for downloaded files

Module Structure

crates/ruvllm/src/hub/
├── mod.rs              # Main module with exports and common types
├── download.rs         # Model download functionality
├── upload.rs           # Model upload functionality
├── registry.rs         # RuvLTRA model registry
├── model_card.rs       # HuggingFace model card generation
└── progress.rs         # Progress tracking utilities

Model Registry

The registry includes pre-configured RuvLTRA models:

Base Models

Model ID Size Params Quantization Use Case
ruvltra-small 662MB 0.5B Q4_K_M Edge devices, includes SONA weights
ruvltra-small-q8 1.3GB 0.5B Q8_0 High quality, small model
ruvltra-medium 2.1GB 3B Q4_K_M General purpose, extended context
ruvltra-medium-q8 4.2GB 3B Q8_0 High quality, balanced model

LoRA Adapters

Adapter ID Size Base Model Purpose
ruvltra-small-coder 50MB ruvltra-small Code completion specialization

Usage

1. Model Download

Using the CLI Example

# Download RuvLTRA Small
cargo run -p ruvllm --example hub_cli -- pull ruvltra-small

# Download to custom directory
cargo run -p ruvllm --example hub_cli -- pull ruvltra-medium --output ./models

# List all available models
cargo run -p ruvllm --example hub_cli -- list

# Show detailed model info
cargo run -p ruvllm --example hub_cli -- info ruvltra-small

Using the API

use ruvllm::hub::{ModelDownloader, RuvLtraRegistry};

// Download by model ID
let downloader = ModelDownloader::new();
let path = downloader.download_by_id("ruvltra-small")?;

// Or download with custom config
let registry = RuvLtraRegistry::new();
let model_info = registry.get("ruvltra-small").unwrap();

let config = DownloadConfig {
    cache_dir: PathBuf::from("./models"),
    resume: true,
    show_progress: true,
    verify_checksum: true,
    ..Default::default()
};

let downloader = ModelDownloader::with_config(config);
let path = downloader.download(model_info, None)?;

2. Model Upload

Using the CLI Example

# Upload a custom model (requires HF_TOKEN)
export HF_TOKEN=your_huggingface_token

cargo run -p ruvllm --example hub_cli -- push \
  --model ./my-ruvltra-custom.gguf \
  --repo username/my-ruvltra-custom \
  --description "My custom RuvLTRA model" \
  --params 0.5 \
  --architecture llama \
  --context 4096 \
  --quant Q4_K_M

Using the API

use ruvllm::hub::{ModelUploader, ModelMetadata, UploadConfig};

// Create metadata
let metadata = ModelMetadata {
    name: "My RuvLTRA Model".to_string(),
    description: Some("A custom RuvLTRA variant".to_string()),
    architecture: "llama".to_string(),
    params_b: 0.5,
    context_length: 4096,
    quantization: Some("Q4_K_M".to_string()),
    license: Some("MIT".to_string()),
    datasets: vec!["custom-dataset".to_string()],
    tags: vec!["ruvltra".to_string(), "custom".to_string()],
};

// Configure uploader
let config = UploadConfig::new(hf_token)
    .private(false)
    .commit_message("Upload custom RuvLTRA model");

let uploader = ModelUploader::with_config(config);
let url = uploader.upload(
    "./my-model.gguf",
    "username/my-ruvltra-custom",
    Some(metadata),
)?;

println!("Model uploaded to: {}", url);

3. Model Registry

use ruvllm::hub::{RuvLtraRegistry, ModelSize};

let registry = RuvLtraRegistry::new();

// Get a specific model
let model = registry.get("ruvltra-small").unwrap();
println!("Model: {}", model.name);
println!("Size: {} MB", model.size_bytes / (1024 * 1024));

// List all models
for model in registry.list_all() {
    println!("{}: {}", model.id, model.description);
}

// List by size category
for model in registry.list_by_size(ModelSize::Small) {
    println!("Small model: {}", model.id);
}

// Get adapters for a base model
for adapter in registry.list_adapters("ruvltra-small") {
    println!("Adapter: {}", adapter.id);
}

// Recommend model based on available RAM
let model = registry.recommend_for_ram(4.0).unwrap();
println!("Recommended for 4GB RAM: {}", model.id);

4. Model Card Generation

use ruvllm::hub::{
    ModelCardBuilder, TaskType, Framework, License
};

let card = ModelCardBuilder::new("RuvLTRA Custom")
    .description("A custom RuvLTRA variant")
    .task(TaskType::TextGeneration)
    .framework(Framework::Gguf)
    .architecture("llama")
    .parameters(500_000_000)
    .context_length(4096)
    .license(License::Mit)
    .add_dataset("training-data", Some("Custom dataset".to_string()))
    .add_metric("perplexity", 5.2, Some("test-set".to_string()))
    .add_tag("ruvltra")
    .add_tag("custom")
    .build();

// Generate markdown for HuggingFace
let markdown = card.to_markdown();

5. Progress Tracking

use ruvllm::hub::{ProgressBar, ProgressStyle};

let mut pb = ProgressBar::new(total_bytes)
    .with_style(ProgressStyle::Detailed)
    .with_width(50);

// Update progress
pb.update(downloaded_bytes);

// Finish
pb.finish();

Hardware Requirements

Each model in the registry includes hardware requirements:

let model = registry.get("ruvltra-small").unwrap();

println!("Minimum RAM: {:.1} GB", model.hardware.min_ram_gb);
println!("Recommended RAM: {:.1} GB", model.hardware.recommended_ram_gb);
println!("Apple Neural Engine: {}", model.hardware.supports_ane);
println!("Metal GPU: {}", model.hardware.supports_metal);
println!("CUDA: {}", model.hardware.supports_cuda);

Environment Variables

  • HF_TOKEN: HuggingFace API token (required for uploads and private repos)
  • HUGGING_FACE_HUB_TOKEN: Alternative name for HF token
  • RUVLLM_MODELS_DIR: Default cache directory for downloaded models

Dependencies

The hub integration requires:

  • curl or wget for downloads (uses system tools for efficiency)
  • huggingface-cli for uploads (install with pip install huggingface_hub[cli])
  • SHA256 for checksum verification (built-in via sha2 crate)

Features

Download Features

  • ✅ Resume interrupted downloads
  • ✅ Progress bar with ETA
  • ✅ SHA256 checksum verification
  • ✅ Automatic retry on failure
  • ✅ HuggingFace token authentication
  • ✅ Cache directory management

Upload Features

  • ✅ Automatic repository creation
  • ✅ Model card generation
  • ✅ Public/private repository support
  • ✅ SONA weights upload
  • ✅ Custom metadata
  • ✅ Commit message customization

Registry Features

  • ✅ Pre-configured model catalog
  • ✅ Hardware requirement tracking
  • ✅ Quantization level support
  • ✅ LoRA adapter registry
  • ✅ RAM-based recommendations
  • ✅ Download time estimation

Error Handling

All hub operations return Result<T, HubError>:

use ruvllm::hub::{HubError, ModelDownloader};

match downloader.download_by_id("ruvltra-small") {
    Ok(path) => println!("Downloaded to: {}", path.display()),
    Err(HubError::NotFound(id)) => eprintln!("Model {} not found", id),
    Err(HubError::ChecksumMismatch { expected, actual }) => {
        eprintln!("Checksum mismatch: expected {}, got {}", expected, actual);
    }
    Err(HubError::Network(msg)) => eprintln!("Network error: {}", msg),
    Err(e) => eprintln!("Error: {}", e),
}

Testing

Run the hub integration tests:

# Test model registry
cargo test -p ruvllm --lib hub::registry

# Test download (requires network)
cargo test -p ruvllm --lib hub::download

# Test model card generation
cargo test -p ruvllm --lib hub::model_card

# Run all hub tests
cargo test -p ruvllm --lib hub

Examples

See the examples for complete usage:

  1. examples/download_test_model.rs - Legacy downloader with hub integration
  2. examples/hub_cli.rs - Full CLI with pull/push/list/info commands

Future Enhancements

Planned improvements:

  • Direct API uploads (without huggingface-cli dependency)
  • Parallel chunk downloads for faster transfers
  • Delta updates for model weights
  • Model versioning support
  • Automatic quantization variant selection
  • Multi-repo synchronization
  • Offline model registry cache

Contributing

To add a new model to the registry:

  1. Add model definition to registry.rs in RuvLtraRegistry::new()
  2. Include hardware requirements
  3. Set checksum after first upload
  4. Update this documentation

License

MIT License - See LICENSE file for details