Skip to content

vantagecompute/slurm-factory

BranchesTags

Repository files navigation

Vantage Compute Logo

Slurm Factory

License Python PyPI Docker

Build relocatable, GPG-signed Slurm packages using Docker and Spack.

Documentation | Buildcache

Quick Start

Option 1: Install Pre-built Slurm from Buildcache (Fastest!)

Use Spack to install GPG-signed pre-built binaries (no slurm-factory tool needed):

# Install Spack
git clone --depth 1 --branch v1.0.0 https://github.com/spack/spack.git
source spack/share/spack/setup-env.sh

# Add mirrors for dependencies and Slurm
SLURM_VERSION=25.11
TOOLCHAIN=noble  # or: jammy, resolute, rockylinux9, rockylinux10, rockylinux8
CLOUDFRONT_URL=https://slurm-factory-spack-binary-cache.vantagecompute.ai

spack mirror add slurm-factory-deps "${CLOUDFRONT_URL}/${TOOLCHAIN}/slurm/deps/"
spack mirror add slurm-factory-slurm "${CLOUDFRONT_URL}/${TOOLCHAIN}/slurm/${SLURM_VERSION}/"

# Import GPG keys and install Slurm (5-15 min!)
spack buildcache keys --install --trust
spack install slurm@${SLURM_VERSION}

→ Full guide: Installing Slurm from Buildcache

Option 2: Build Custom Slurm with slurm-factory Tool

Install the slurm-factory tool to build custom packages:

# Install Docker and the slurm-factory build tool
pip install slurm-factory

# Build Slurm with default compiler (GCC 13.4.0)
slurm-factory build-slurm --slurm-version 25.11

→ Full guide: Installing slurm-factory Tool

Supported Versions

Slurm × Operating System Matrix

All combinations use OS-provided compilers and are available in the public buildcache:

Slurm Version Status Supported Operating Systems
25.11 Latest Rocky 10, Rocky 9, Rocky 8, Ubuntu 24.04, Ubuntu 22.04, Ubuntu 25.04
24.11 LTS Rocky 10, Rocky 9, Rocky 8, Ubuntu 24.04, Ubuntu 22.04, Ubuntu 25.04
23.11 Stable Rocky 10, Rocky 9, Rocky 8, Ubuntu 24.04, Ubuntu 22.04, Ubuntu 25.04

Default: Ubuntu 24.04 (Noble) - recommended for most users

Operating System Compiler Toolchains

Slurm is built using the default system compiler from each OS distribution:

OS Codename GCC Version GLIBC Use Case
Ubuntu 24.04 Noble 13.3.0 2.39 Recommended - Modern stable
Ubuntu 26.04 Resolute 15.2.0 2.42 Latest features (development)
Ubuntu 22.04 Jammy 11.4.0 2.35 LTS - Wide compatibility
Rocky Linux 10 - 14.3.1 2.39 RHEL 10 compatible
Rocky Linux 9 - 11.5.0 2.34 RHEL 9 compatible
Rocky Linux 8 - 8.5.0 2.28 RHEL 8 compatible

GPG Package Signing

All packages are cryptographically signed with GPG for security and integrity.

Why GPG Signing?

  • ✅ Authenticity: Verify packages were built by Vantage Compute
  • ✅ Integrity: Detect tampering or corruption during download
  • ✅ Security: Prevent man-in-the-middle attacks
  • ✅ Trust Chain: Establish provenance for production deployments

GPG Key Information

Key ID: DFB92630BCA5AB71
Owner: Vantage Compute Corporation (Slurm Factory Spack Cache Signing Key)
Email: info@vantagecompute.ai

Importing GPG Keys

Keys are automatically imported when using the buildcache:

# Automatic import and trust
spack buildcache keys --install --trust

# Packages are verified during installation
spack install slurm@25.11%gcc@13.4.0

Features

  • 🔐 GPG-Signed Packages - All Slurm packages cryptographically signed
  • ⚡ 10-15x Faster - Pre-built packages install in 5-15 minutes vs 45-90 minutes
  • 📦 Relocatable - Deploy to any path, no host dependencies
  • 🌍 CDN Distribution - CloudFront-distributed buildcache for fast global access
  • 🐧 6 OS Platforms - Rocky Linux 8/9/10, Ubuntu 22.04/24.04/25.04
  • 🎯 3 Slurm Versions - 25.11, 24.11, 23.11
  • 🏭 OS-Native Compilers - Uses system-provided GCC for maximum compatibility
  • 🚀 Optimized - Architecture-specific compilation (x86_64_v3)
  • 🐳 Clean Builds - Docker isolation, no system pollution

Build Options

Build from Source with slurm-factory

# Default build (CPU-only, Ubuntu 24.04 toolchain)
slurm-factory build-slurm --slurm-version 25.11

# GPU support (CUDA/ROCm)
slurm-factory build-slurm --slurm-version 25.11 --gpu

# Different OS toolchain
slurm-factory build-slurm --slurm-version 25.11 --toolchain jammy

# Build and publish to buildcache with GPG signing
slurm-factory build-slurm --slurm-version 25.11

Install Pre-built from Buildcache (Fastest!)

# Install Spack
git clone --depth 1 --branch v1.0.0 https://github.com/spack/spack.git
source spack/share/spack/setup-env.sh

# Configure mirrors for buildcache
SLURM_VERSION=25.11
TOOLCHAIN=noble  # or: jammy, resolute, rockylinux9, rockylinux10, rockylinux8
CLOUDFRONT_URL=https://slurm-factory-spack-binary-cache.vantagecompute.ai

spack mirror add slurm-factory-deps "${CLOUDFRONT_URL}/${TOOLCHAIN}/slurm/deps/"
spack mirror add slurm-factory-slurm "${CLOUDFRONT_URL}/${TOOLCHAIN}/slurm/${SLURM_VERSION}/"

# Import GPG signing keys and trust
spack buildcache keys --install --trust

# Install signed package (5-15 minutes!)
spack install slurm@${SLURM_VERSION} target=x86_64_v3

# Deploy
spack load slurm@${SLURM_VERSION}

Download Pre-built Tarball (Alternative)

Download complete Slurm installation as a tarball with GPG signature verification:

# Set versions
SLURM_VERSION=25.11
TOOLCHAIN=noble  # or: jammy, rockylinux9, etc.
CLOUDFRONT_URL=https://slurm-factory-spack-binary-cache.vantagecompute.ai

# Download tarball and signature
wget "${CLOUDFRONT_URL}/builds/${SLURM_VERSION}/${TOOLCHAIN}/slurm-${SLURM_VERSION}-${TOOLCHAIN}-software.tar.gz"
wget "${CLOUDFRONT_URL}/builds/${SLURM_VERSION}/${TOOLCHAIN}/slurm-${SLURM_VERSION}-${TOOLCHAIN}-software.tar.gz.asc"

# Import GPG key
gpg --keyserver keyserver.ubuntu.com --recv-keys DFB92630BCA5AB71

# Verify signature
gpg --verify slurm-${SLURM_VERSION}-${TOOLCHAIN}-software.tar.gz.asc \
             slurm-${SLURM_VERSION}-${TOOLCHAIN}-software.tar.gz

# Extract and install
sudo tar -xzf slurm-${SLURM_VERSION}-${TOOLCHAIN}-software.tar.gz -C /opt/
cd /opt && sudo ./data/slurm_assets/slurm_install.sh --full-init

Package Structure

All builds produce GPG-signed relocatable tarballs:

slurm-25.11-noble-software.tar.gz
├── view/                    # Slurm binaries & libraries
├── modules/slurm/25.11.lua  # Lmod module (relocatable)
└── data/slurm_assets/       # Config templates & install script

Deployment

# Extract (from tarball build)
sudo tar -xzf slurm-25.11-noble-software.tar.gz -C /opt/

# Install (creates users, configs, services)
cd /opt && sudo ./data/slurm_assets/slurm_install.sh --full-init

# Use
module load slurm/25.11

Buildcache Structure

The public buildcache is organized by OS toolchain for optimal performance:

https://slurm-factory-spack-binary-cache.vantagecompute.ai/
├── <toolchain>/
│   └── slurm/
│       ├── deps/            # Slurm dependencies (GPG-signed)
│       ├── 25.11/           # Slurm 25.11 packages (GPG-signed)
│       ├── 24.11/           # Slurm 24.11 packages (GPG-signed)
│       └── 23.11/           # Slurm 23.11 packages (GPG-signed)
├── noble/                   # Ubuntu 24.04
│   └── slurm/
│       ├── deps/
│       ├── 25.11/
│       ├── 24.11/
│       └── 23.11/
├── jammy/                   # Ubuntu 22.04
├── resolute/                # Ubuntu 25.04
├── rockylinux10/            # Rocky Linux 10
├── rockylinux9/             # Rocky Linux 9
├── rockylinux8/             # Rocky Linux 8
└── builds/
    └── <slurm_version>/
        └── <toolchain>/
            ├── slurm-<version>-<toolchain>-software.tar.gz      # Complete tarball
            └── slurm-<version>-<toolchain>-software.tar.gz.asc  # GPG signature


### Mirror Architecture

The buildcache uses a **toolchain-first structure** organized by OS for efficient caching:

1. **`<toolchain>/slurm/deps/`** - Slurm runtime dependencies (OpenMPI, PMIx, Munge, etc.)
2. **`<toolchain>/slurm/<version>/`** - Slurm packages for each version

This separation allows:

- ✅ **Better caching** - Dependencies shared across Slurm versions within a toolchain
- ✅ **OS compatibility** - Packages built with native system compilers
- ✅ **Reduced storage** - No duplication of common packages per toolchain
- ✅ **Parallel downloads** - Spack can fetch from multiple mirrors simultaneously

## Requirements

- Python 3.12+
- Docker 24.0+ (for building from source)
- 50GB disk space
- 4+ CPU cores (8+ recommended)
- 16GB RAM (32GB+ recommended)

For buildcache installs: only Spack required (no Docker needed)

## Documentation

**[vantagecompute.github.io/slurm-factory](https://vantagecompute.github.io/slurm-factory)**

- [Overview](https://vantagecompute.github.io/slurm-factory/overview) - Architecture and features
- [Installation](https://vantagecompute.github.io/slurm-factory/installation) - Setup and quick start
- [Examples](https://vantagecompute.github.io/slurm-factory/examples) - Common use cases
- [Buildcache Guide](https://vantagecompute.github.io/slurm-factory/slurm-factory-spack-build-cache) - GPG signing and distribution
- [API Reference](https://vantagecompute.github.io/slurm-factory/api-reference) - Python API

## Development

```bash
# Install from source
git clone https://github.com/vantagecompute/slurm-factory.git
cd slurm-factory
pip install -e .

# Run tests
pytest

# Build docs
cd docusaurus && npm run build

License

Apache License 2.0 - See LICENSE file.

Support

Built with ❤️ by Vantage Compute

About

Slurm build factory using Spack + Docker.

vantagecompute.github.io/slurm-factory/

Topics

docker hpc slurm build-tool spack cicd

Resources

Readme

License

View license
Activity
Custom properties

Stars

3 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases 11

Release 0.1.38 Latest
Feb 9, 2026
+ 10 releases

Packages

No packages published

Contributors 3

  •  
  •  
  •  

Languages