Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ cluster provisioning, component installation, and test execution.
┌──────────────▼──────────────────────────────────────┐
│ 5. Execute Tests (run-spread-tests) │
│ • Identify components with use-spread: true │
│ • Identify components with test: true │
│ • Clone component repos │
│ • Run spread tests from component spread.yaml │
│ • Collect and report results │
Expand Down Expand Up @@ -123,14 +123,14 @@ Components use this to download/build for the correct architecture.

**Steps**:
1. Install spread testing framework
2. Scan manifest for use-spread components
2. Scan manifest for test components
3. Clone component repos
4. Execute spread tests
5. Collect artifacts

**Key Features**:
- Tests come from components and local tests/
- Scans for use-spread: true in components
- Scans for test: true in components
- Reports results and failures
- Preserves test artifacts

Expand Down
33 changes: 17 additions & 16 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ Scalable Kubernetes testing infrastructure that validates custom-built component

**Manifest-Driven Design** - YAML cluster configurations:
- Define K8s version, node counts, networking, and component versions/sources
- Components specify repo URL, release tag, format (Binary|Container|Binary+Container), and `use-spread` flag (component provides tests)
- Components specify repo URL, release tag, format (Binary|Container|Binary+Container), and `test` flag (component provides tests)
- 4 baseline manifests targeting K8s 1.33, 1.34, 1.35, 1.36 with 30+ components each (containerd, etcd, coredns, kube-* etc.)

**Python CLI Framework** - Cross-platform testing:
- `kube-galaxy` CLI with command routing and automatic manifest discovery
- `pkg/cluster/setup.py`: Manifest-based provisioning via kubeadm (not container shortcuts)
- `pkg/testing/spread.py`: Executes spread tests from components marked `use-spread: true`
- `pkg/testing/spread.py`: Executes spread tests from components marked `test: true`
- `pkg/utils/logs.py`: Log collection and debugging utilities
- Library utilities for arch detection, YAML manifest parsing, component installation

Expand Down Expand Up @@ -42,25 +42,28 @@ components:
- name: containerd # Component identifier
category: containerd # Organizational
release: "2.2.1" # Git tag or branch
repo: "https://github.com/..." # Fetch source
repo: # Repository info object
base-url: "https://github.com/..." # Required: fetch source
subdir: "path/to/component" # Optional: for monorepo components
ref: "feature-branch" # Optional: override release with git ref
format: Binary|Container|Binary+Container # Install method
use-spread: false/true # Component provides spread tests
test: false/true # Component provides spread tests
networking:
- name: calico
service-cidr: "10.96.0.0/12"
pod-cidr: "192.168.0.0/16"
```
Key insight: `use-spread: true` means component repo has spread.yaml tests that `kube-galaxy test spread` will execute.
Key insight: `test: true` means component repo has spread.yaml tests that `kube-galaxy test` will execute.

### Local Development Workflow
```bash
# Validate manifest YAML syntax
kube-galaxy validate manifests
kube-galaxy validate

# Provision real cluster with kubeadm (no container shortcuts)
kube-galaxy setup

# Run spread tests from components with use-spread: true
# Run spread tests from components with test: true
kube-galaxy test spread
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CLI has been changed so kube-galaxy test takes a manifest path directly (and test spread subcommand was removed), but this doc still instructs kube-galaxy test spread and describes that flow in the “Test Execution Model” section. Please update these examples/descriptions to match the new CLI syntax (e.g., kube-galaxy test manifests/baseline-k8s-1.35.yaml).

Suggested change
kube-galaxy test spread
kube-galaxy test manifests/baseline-k8s-1.35.yaml

Copilot uses AI. Check for mistakes.

# Clean cluster and artifacts
Expand All @@ -83,7 +86,7 @@ kube-galaxy cleanup all
5. Components specify format: Binary (install to /usr/local/bin), Container (pull image), or both

### Test Execution Model
- **Test discovery**: Only components with `use-spread: true` are tested
- **Test discovery**: Only components with `test: true` are tested
- **Test location**: Component repos contain `spread.yaml` at root
- **Spread execution**: `kube-galaxy test spread` clones each component, finds spread.yaml, runs spread test suite
- **Parallelism**: Spread tests run concurrently if specified in spread.yaml
Expand Down Expand Up @@ -142,7 +145,7 @@ Architecture detection happens at runtime in `pkg/cluster/setup.py`:

**When Adding Components**:
1. Add entry to all 4 `manifests/baseline-k8s-*.yaml` files (don't skip versions)
2. Set `use-spread: true` only if component repo has `spread.yaml` with test definitions
2. Set `test: true` only if component repo has `spread.yaml` with test definitions
3. Use canonical GitHub repos where available; verify release tag exists
4. Set `format` correctly based on component's build/distribution (Binary, Container, or both)

Expand All @@ -163,17 +166,15 @@ Architecture detection happens at runtime in `pkg/cluster/setup.py`:

```bash
# Validation
kube-galaxy validate all
kube-galaxy validate manifests
kube-galaxy test-manifest manifests/baseline-k8s-1.35.yaml
kube-galaxy validate
kube-galaxy validate --manifest manifests/baseline-k8s-1.35.yaml

# Testing
kube-galaxy setup
kube-galaxy test local
kube-galaxy test spread
kube-galaxy setup manifests/baseline-k8s-1.35.yaml
kube-galaxy test manifests/baseline-k8s-1.35.yaml

# Management
kube-galaxy cleanup all
kube-galaxy cleanup manifests/baseline-k8s-1.35.yaml
kube-galaxy status
```

Expand Down
29 changes: 13 additions & 16 deletions .github/workflows/test-baseline-clusters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,17 @@ jobs:
with:
python-version: "3.12"

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: ">=1.22"

- name: Install Spread
run: go install github.com/snapcore/spread/cmd/spread@latest

- name: Setup LXD
uses: canonical/setup-lxd@main

- name: Install kube-galaxy
shell: bash
run: |
Expand Down Expand Up @@ -151,28 +162,14 @@ jobs:
id: run-tests
shell: bash
run: |
kube-galaxy test spread

- name: Capture Test Results
if: always()
shell: bash
run: |
echo "## Test Execution Summary"
echo "- **Manifest**: ${{ matrix.manifest-file }}"
echo "- **Status**: ${{ steps.run-tests.outputs.test-status }}"
echo "- **Results Path**: ${{ steps.run-tests.outputs.test-results }}"

if [ -f "test-results/summary.md" ]; then
echo ""
cat test-results/summary.md
fi
kube-galaxy test ${{ matrix.manifest-file }}

- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ steps.setup-cluster.outputs.artifact-name }}
path: test-results/
path: logs/
retention-days: 30
if-no-files-found: warn

Expand Down
58 changes: 47 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A scalable, multi-architecture testing infrastructure for Kubernetes using funct
- **Per-Component Definition**: Each component specifies its repo, release, and whether it provides tests
- **GitHub Actions**: Automatic provisioning, testing, and cleanup workflows
- **Multiarch Support**: Runtime architecture detection (amd64, arm64, riscv64, etc.)
- **Component-Driven Tests**: Tests live in component repos, referenced via `use-spread` flag
- **Component-Driven Tests**: Tests live in component repos, referenced via `test` flag
- **Kubeadm-Based Provisioning**: Real cluster setup without container-based shortcuts

## 🏃 Quick Start
Expand Down Expand Up @@ -135,16 +135,25 @@ kubernetes-version: "v1.29.0"
components:
- name: containerd
release: "v1.7.0"
repo: "https://github.com/containerd/containerd"
use-spread: false
repo:
base-url: "https://github.com/containerd/containerd"
test: false
- name: kubeadm
release: "v1.29.0"
repo: "https://github.com/kubernetes/kubernetes"
use-spread: false
repo:
base-url: "https://github.com/kubernetes/kubernetes"
test: false
- name: my-custom-cni
release: "main"
repo: "https://github.com/myorg/custom-cni"
use-spread: true
repo:
base-url: "https://github.com/myorg/custom-cni"
test: true
- name: monorepo-component
repo:
base-url: "https://github.com/myorg/monorepo"
subdir: "components/my-component"
ref: "feature-branch"
test: true
networking:
- name: "calico"
service-cidr: "10.96.0.0/12"
Expand All @@ -158,17 +167,44 @@ networking:
- **kubernetes-version**: Kubernetes version (informational)
- **components**: List of components to install
- **name**: Component identifier
- **release**: Git tag/branch to checkout
- **repo**: Git repository URL
- **use-spread**: If `true`, component repository provides spread tests
- **release**: Git tag/branch to checkout (defaults to repo.ref if not specified)
- **repo**: Repository information object
- **base-url**: Git repository URL (required)
- **subdir**: Optional subdirectory path for monorepo components
- **ref**: Optional git reference (branch/tag/commit), defaults to release
- **test**: If `true`, component repository provides spread tests
- **networking**: CNI and network settings

### Repository Configuration

The `repo` field is an object that provides flexible repository configuration:

- **Simple component**: Only needs `base-url`
```yaml
repo:
base-url: "https://github.com/containerd/containerd"
```

- **Monorepo component**: Add `subdir` to specify component location
```yaml
repo:
base-url: "https://github.com/kubernetes/kubernetes"
subdir: "staging/src/k8s.io/kubectl"
```

- **Development branch**: Use `ref` to override the release tag
```yaml
repo:
base-url: "https://github.com/myorg/component"
ref: "feature-branch"
```

## 🔧 Component Repositories

Each component repository must contain a `spread.yaml` file that defines:

1. **Install instructions** (required if component is used)
2. **Test definitions** (required if `use-spread: true`)
2. **Test definitions** (required if `test: true`)

### Example Component spread.yaml

Expand Down
18 changes: 0 additions & 18 deletions docs/class-based-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ The class-based approach provides several advantages:
- Pythonic, simple, no special methods needed

3. **Configuration**: Easy access to manifest configuration via properties:
- `self.hook_config` - dict of hook-specific configuration
- `self.custom_binary_url` - custom binary URL (if provided)
- `self.install_method` - installation method

Expand Down Expand Up @@ -118,23 +117,6 @@ def install_hook(self):
install_binary(self.binary_path, 'mycomponent')
```

### Hook Configuration

Access hook-specific configuration from manifest:

```python
def bootstrap_hook(self):
# Get hook configuration from manifest (via property)
config = self.hook_config.get('bootstrap', {})
pod_cidr = config.get('pod_network_cidr', '10.244.0.0/16')
service_cidr = config.get('service_cidr', '10.96.0.0/12')

# Use configuration
run(['kubeadm', 'init', f'--pod-network-cidr={pod_cidr}'])
```

**Note**: Hook skipping is automatic! If you don't override a hook method, it simply won't run (the base class provides an empty default implementation).

## Complete Example

```python
Expand Down
2 changes: 1 addition & 1 deletion docs/component-lifecycle-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ The `@register_component_class` decorator registers the component at module impo

2. **Use instance attributes for state**: Share data between hooks using `self.attribute_name`

3. **Use properties for configuration**: Access manifest config via `self.custom_binary_url`, `self.hook_config`, etc.
3. **Use properties for configuration**: Access manifest config via `self.custom_binary_url`, etc.

4. **Don't override hooks you don't need**: The base class provides empty defaults

Expand Down
Loading
Loading