FABRIC Generic Cluster - Notebooks

Jupyter notebooks and example topology files for the fabric-generic-cluster framework.

Overview

This repository contains Jupyter notebooks and YAML topology examples for creating and managing FABRIC testbed slices using the fabric-generic-cluster Python package.

Prerequisites

1. Install the fabric-generic-cluster package

pip install fabric-generic-cluster

2. Configure FABRIC credentials

Ensure you have FABRIC testbed access configured. See FABRIC Learn for setup instructions.

3. Install Jupyter

pip install jupyter

Repository Structure

fabric-generic-cluster-notebooks/
├── notebooks/              # Jupyter notebooks
│   ├── notebook-create-slice.ipynb                    # Main workflow: create and configure slices
│   ├── notebook-configure-slice-bluefield_dpu.ipynb   # BlueField DPU configuration
│   ├── notebook-aux-setup-environment.ipynb           # Environment setup and verification
│   ├── notebook-aux-topology-builder.ipynb            # Interactive topology builder
│   ├── notebook-aux-topology-summary-generator.ipynb  # Generate topology summaries
│   ├── notebook-aux-converter.ipynb                   # Convert notebooks to Python scripts
│   └── notebook-aux-resource-finder.ipynb             # Find available FABRIC resources
│
└── model/                  # Example YAML topology files
    ├── _slice_topology.yaml           # Basic topology example
    ├── _slice_topology_1.yaml         # Simple 2-node cluster
    ├── _slice_topology_2.yaml         # Multi-site deployment
    ├── _slice_topology_3.yaml         # Storage cluster
    ├── _slice_topology_4.yaml         # Network-intensive topology
    ├── _slice_topology_5.yaml         # Mixed workload cluster
    ├── _slice_topology_6.yaml         # High-performance computing setup
    ├── _slice_topology_7*.yaml        # OpenStack deployment variants
    ├── _slice_topology_8*.yaml        # DPU/SmartNIC configurations
    └── _slice_topology_9.yaml         # FPGA-enabled topology

Quick Start

1. Clone this repository

git clone https://github.com/mcevik0/fabric-generic-cluster-notebooks.git
cd fabric-generic-cluster-notebooks

2. Start Jupyter

jupyter notebook

3. Run the setup notebook

Open and run notebooks/notebook-aux-setup-environment.ipynb to verify your environment is correctly configured.

4. Create your first slice

Open notebooks/notebook-create-slice.ipynb and follow the workflow to deploy a FABRIC slice.

Notebooks Guide

Core Notebooks

📘 notebook-create-slice.ipynb

Main workflow for creating and configuring FABRIC slices

Complete workflow including:

Loading topology from YAML
Deploying slice to FABRIC
Configuring L3 networks
Setting up network interfaces
Configuring SSH access
Testing connectivity

Usage:

from fabric_generic_cluster import (
    load_topology_from_yaml_file,
    deploy_topology_to_fabric,
    configure_node_interfaces,
    setup_passwordless_ssh,
)

topology = load_topology_from_yaml_file("../model/_slice_topology.yaml")
slice = deploy_topology_to_fabric(topology, "my-cluster")
configure_node_interfaces(slice, topology)
setup_passwordless_ssh(slice)

📗 notebook-configure-slice-bluefield_dpu.ipynb

Configure slices with NVIDIA BlueField SmartNICs

Specialized notebook for:

BlueField DPU deployment
DPU configuration and management
High-performance networking with SmartNICs
DOCA software installation

Auxiliary Notebooks

🔧 notebook-aux-setup-environment.ipynb

Verify and setup your Python environment

Check installed packages
Install dependencies
Verify FABRIC configuration
Test module imports

Run this first to ensure everything is configured correctly.

🏗️ notebook-aux-topology-builder.ipynb

Interactive topology builder

Create topologies programmatically
Add nodes and networks interactively
Validate topology structure
Export to YAML

📊 notebook-aux-topology-summary-generator.ipynb

Generate comprehensive topology summaries

Analyze topology files
Generate markdown summaries
Create network diagrams
Export topology metadata

🔄 notebook-aux-converter.ipynb

Convert notebooks to Python scripts

Single or batch conversion
Remove magic commands
Clean output for automation
CI/CD integration

🔍 notebook-aux-resource-finder.ipynb

Find available FABRIC resources

Query available sites
Check resource availability
Find nodes with specific hardware (DPUs, FPGAs, GPUs)
Plan resource allocation

Example Topologies

The model/ directory contains example YAML topology files demonstrating various configurations:

File	Description	Use Case
`_slice_topology.yaml`	Basic example	Learning the topology format
`_slice_topology_1.yaml`	2-node cluster	Simple experimentation
`_slice_topology_2.yaml`	Multi-site	Geographically distributed systems
`_slice_topology_3.yaml`	Storage cluster	Persistent storage testing
`_slice_topology_4.yaml`	Network-heavy	Network protocol research
`_slice_topology_7*.yaml`	OpenStack	Cloud infrastructure deployment
`_slice_topology_8*.yaml`	DPU/SmartNIC	Hardware acceleration
`_slice_topology_9.yaml`	FPGA	Custom hardware logic

Using Example Topologies

from fabric_generic_cluster import load_topology_from_yaml_file

# Load any example topology
topology = load_topology_from_yaml_file("../model/_slice_topology_8.yaml")

# Customize for your needs
# ... modify topology ...

# Deploy
slice = deploy_topology_to_fabric(topology, "my-slice-name")

Common Workflows

Workflow 1: Deploy a Basic Cluster

1. Open notebook-create-slice.ipynb
2. Set slice_name and site_topology_yaml path
3. Run all cells
4. Wait for deployment (~5-10 minutes)
5. SSH into nodes or run experiments

Workflow 2: Create Custom Topology

1. Open notebook-aux-topology-builder.ipynb
2. Interactively build your topology
3. Export to YAML in model/ directory
4. Use in notebook-create-slice.ipynb

Workflow 3: BlueField DPU Deployment

1. Open notebook-configure-slice-bluefield_dpu.ipynb
2. Configure DPU settings
3. Deploy and configure BlueField
4. Access DPU via SSH or console

Package Usage in Notebooks

All notebooks use the installed fabric-generic-cluster package:

# Import from installed package
from fabric_generic_cluster import (
    load_topology_from_yaml_file,
    deploy_topology_to_fabric,
    configure_node_interfaces,
    setup_passwordless_ssh,
    print_topology_summary,
)

# Or import modules
from fabric_generic_cluster import deployment as sd
from fabric_generic_cluster import network_config as snc

No sys.path manipulation needed - everything works via pip install!

Tips and Best Practices

💡 Naming Conventions

Use descriptive slice names: my-project-experiment-1
YAML files start with _slice_topology_ for consistency

💡 Resource Management

Always delete slices when done: slice.delete()
Check existing slices before creating new ones
Monitor lease expiration times

💡 Topology Design

Start with example topologies
Validate before deploying
Use topology viewer to visualize
Document custom topologies

💡 Debugging

Use slice.show() to see slice details
Check slice.list_nodes() and slice.list_interfaces()
Verify network configuration with ping tests
Check SSH connectivity before running commands

Troubleshooting

Issue: Package not found

pip install fabric-generic-cluster
# Restart Jupyter kernel

Issue: FABRIC authentication failed

# Reconfigure FABRIC credentials
# See: https://learn.fabric-testbed.net/

Issue: Slice deployment fails

Check resource availability at target site
Verify topology YAML is valid
Ensure slice name is unique
Check FABRIC system status

Issue: Network configuration fails

Verify L3 networks are configured first
Check interface names match topology
Ensure nodes are active before configuring

Contributing

Found an issue or have a useful notebook to share?

Framework issues: Report at fabric-generic-cluster
Notebook issues: Report at this repository's issues
Pull requests: Contributions welcome!

Related Resources

Main Package: fabric-generic-cluster
FABRIC Testbed: https://fabric-testbed.net/
FABRIC Learn: https://learn.fabric-testbed.net/
PyPI Package: fabric-generic-cluster

License

MIT License - See LICENSE file for details

Authors

Mert Cevik (@mcevik0)

Citation

If you use this framework in your research, please cite:

@software{fabric_generic_cluster,
  author = {Cevik, Mert},
  title = {FABRIC Generic Cluster Framework},
  year = {2024},
  url = {https://github.com/mcevik0/fabric-generic-cluster}
}

Questions? Open an issue or contact via GitHub!

FilesExpand file tree

README.md

Latest commit

History