Skip to content

Commit c5affaa

Browse files
committed
Organize multi-arch tracking tools into dedicated directory
Move architecture tracking script and configuration into a dedicated script/multi-arch-tracking/ directory with improved organization: - Rename generate_arch_table.py → generate-table.py - Rename arch_exceptions.toml → exceptions.toml - Rename README_arch_table.md → README.md - Add auto-detection of repository root and config file location - Add --branch flag to read PipelineRun files from git branches - Update all documentation with new paths and usage examples The script now works seamlessly from both the repository root and its own directory, automatically detecting the correct paths. Assisted-By: Claude <noreply@anthropic.com>
1 parent 74bfb6a commit c5affaa

File tree

4 files changed

+1313
-0
lines changed

4 files changed

+1313
-0
lines changed

CLAUDE.md

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Repository Overview
6+
7+
This repository (`konflux-central`) serves as the central configuration repository for Red Hat OpenShift AI (RHOAI) Konflux CI/CD pipelines. It manages Tekton pipelines and PipelineRuns for building container images across multiple RHOAI components.
8+
9+
## Repository Structure
10+
11+
- `pipelines/` - Reusable Tekton pipeline definitions
12+
- `container-build.yaml` - Single-arch container build pipeline
13+
- `multi-arch-container-build.yaml` - Multi-platform container build pipeline
14+
- `fbc-fragment-build.yaml` - File-Based Catalog fragment build pipeline
15+
16+
- `pipelineruns/` - Component-specific PipelineRun definitions organized by component name
17+
- Each component directory contains a `.tekton/` subdirectory
18+
- PipelineRun files follow the naming pattern: `{component}-{version}-{trigger}.yaml`
19+
- Example: `pipelineruns/odh-dashboard/.tekton/odh-dashboard-v3-2-push.yaml`
20+
21+
- `.tekton/` - Pull request pipeline definitions for this repository itself
22+
- `container-build-pull-request.yaml` - Triggered on PRs to validate pipeline changes
23+
- `multi-arch-container-build-pull-request.yaml` - Multi-arch validation for PRs
24+
25+
- `canary-build/` - Canary build resources for testing pipeline changes
26+
- `Dockerfile.konflux` - Minimal Dockerfile used for canary builds
27+
28+
## Branch Structure
29+
30+
- Branches follow the pattern `rhoai-X.Y` (e.g., `rhoai-3.2`)
31+
- Each branch corresponds to a specific RHOAI release version
32+
- The `main` branch is the primary development branch
33+
- PipelineRuns are typically configured to reference pipelines from the same target branch
34+
35+
## Pipeline Architecture
36+
37+
### Single-Architecture Pipeline (`container-build.yaml`)
38+
39+
Used for building container images on a single platform. The pipeline:
40+
1. Clones the source repository using `git-clone-oci-ta` task
41+
2. Prefetches dependencies with `prefetch-dependencies-oci-ta` task (supports npm, gomod, rpm via Cachi2)
42+
3. Builds the container image with `buildah-oci-ta` task
43+
4. Creates an image index with `build-image-index` task
44+
5. Builds a source image with `source-build-oci-ta` task
45+
6. Runs security and quality checks:
46+
- `sast-shell-check-oci-ta` - Shell script static analysis
47+
- `sast-unicode-check-oci-ta` - Unicode character validation
48+
- `deprecated-base-image-check` - Base image deprecation check
49+
- `clair-scan` - Vulnerability scanning
50+
- `ecosystem-cert-preflight-checks` - Red Hat certification checks
51+
- `sast-snyk-check-oci-ta` - Snyk security scanning
52+
- `clamav-scan` - Malware scanning
53+
- `rpms-signature-scan` - RPM signature validation
54+
7. Applies additional tags with `apply-tags` task
55+
8. Pushes the Dockerfile with `push-dockerfile-oci-ta` task
56+
9. Sends Slack notifications on failure via `send-slack-notification` task
57+
58+
### Multi-Architecture Pipeline (`multi-arch-container-build.yaml`)
59+
60+
Extends the single-arch pipeline to support multiple platforms (x86_64, ppc64le, s390x, arm64). Uses the `build-platforms` parameter to specify target architectures.
61+
62+
### PipelineRun Configuration
63+
64+
PipelineRuns are configured with:
65+
- **Triggers**: Defined via `pipelinesascode.tekton.dev/on-cel-expression` annotations
66+
- Push events: `event == "push" && target_branch == "rhoai-X.Y"`
67+
- Pull request events: `event == "pull_request" && target_branch.matches("^rhoai-\\d+\\.\\d+$")`
68+
- **Parameters**:
69+
- `git-url` - Source repository URL (templated with `{{source_url}}`)
70+
- `revision` - Git commit SHA (templated with `{{revision}}`)
71+
- `output-image` - Target image registry path (typically `quay.io/rhoai/...`)
72+
- `dockerfile` - Path to Dockerfile (often `Dockerfile.konflux`)
73+
- `path-context` - Build context directory
74+
- `hermetic` - Network isolation flag (usually `true`)
75+
- `prefetch-input` - Dependency prefetch configuration (JSON array)
76+
- `build-platforms` - Array of target platforms for multi-arch builds
77+
- `additional-tags` - Extra image tags beyond the default
78+
- `additional-labels` - Extra OCI image labels
79+
80+
### Custom RHOAI Tasks
81+
82+
The pipelines use custom RHOAI-specific tasks from the `rhoai-konflux-tasks` repository:
83+
- `rhoai-init` - Initialization task that validates cluster environment and prepares Slack notifications
84+
- Located at: `https://github.com/red-hat-data-services/rhoai-konflux-tasks.git`
85+
- Referenced by commit SHA in pipeline definitions
86+
- Sets `expected-cluster` parameter to ensure builds run on the correct cluster
87+
88+
## GitHub Actions Workflows
89+
90+
### Sync Pipelineruns (`.github/workflows/sync-pipelineruns.yml`)
91+
92+
Automates the synchronization of PipelineRun definitions to component repositories:
93+
- Triggered on pushes to `rhoai-*` branches when files in `pipelineruns/` change
94+
- Can be manually triggered with `workflow_dispatch` to select specific repositories
95+
- Supports dry-run mode for testing without committing changes
96+
- Manages synchronization for all RHOAI components
97+
98+
### Update Repository List (`.github/workflows/update-repository-list.yml`)
99+
100+
Maintains the list of repositories available in the sync workflow.
101+
102+
## Working with PipelineRuns
103+
104+
### Adding a New Component
105+
106+
1. Create a new directory under `pipelineruns/{component-name}/`
107+
2. Create a `.tekton/` subdirectory
108+
3. Add PipelineRun YAML files following the naming convention
109+
4. Reference the appropriate pipeline from `pipelines/` directory
110+
5. Configure trigger expressions for push/PR events
111+
6. Set component-specific parameters (image name, dockerfile path, etc.)
112+
113+
### Modifying Existing Pipelines
114+
115+
- Pipeline changes in `pipelines/` directory affect all components using those pipelines
116+
- Test changes using the canary-build pipeline triggers in `.tekton/` before merging
117+
- Pipeline definitions reference specific task bundle digests for reproducibility
118+
- Update task bundle references when updating to newer task versions
119+
120+
### Testing Pipeline Changes
121+
122+
- Pull requests to `rhoai-*` branches trigger validation pipelines in `.tekton/`
123+
- Canary builds validate pipeline modifications before they affect component builds
124+
- Use `[skip-sync]` in commit messages to prevent automatic PipelineRun synchronization
125+
126+
## Common Parameters and Conventions
127+
128+
- **Image Registry**: Most images are pushed to `quay.io/rhoai/` or `quay.io/redhat-user-workloads/rhoai-tenant/`
129+
- **Image Tags**: Follow the pattern `{branch}-{commit-sha}` for additional tags
130+
- **Version Labels**: Set via `additional-labels` parameter (e.g., `version=v3.2.0`)
131+
- **Timeouts**: Multi-arch builds typically use 8-hour pipeline timeouts
132+
- **Service Accounts**: Each component has a dedicated service account (e.g., `build-pipeline-odh-dashboard-v3-2`)
133+
- **Workspaces**: Git authentication is provided via the `git-auth` workspace using secrets
134+
135+
## Hermetic Builds and Dependency Prefetching
136+
137+
- Most builds use hermetic mode (`hermetic: true`) for network isolation
138+
- Dependencies are prefetched using Cachi2 via the `prefetch-input` parameter
139+
- Supported package managers: npm, gomod, rpm, pip
140+
- Prefetch configuration is passed as JSON array in the PipelineRun spec
141+
142+
## Security and Compliance
143+
144+
- All builds include comprehensive security scanning (Clair, Snyk, ClamAV)
145+
- RPM signatures are validated for Red Hat package integrity
146+
- Base images are checked for deprecation
147+
- Static analysis is performed on shell scripts
148+
- Red Hat ecosystem certification checks ensure compliance
149+
- Build failures trigger Slack notifications to the RHOAI team
150+
151+
## Architecture Support Table Generator
152+
153+
A Python script (`script/multi-arch-tracking/generate-table.py`) that generates tables showing which CPU architectures each RHOAI component supports.
154+
155+
### Usage
156+
157+
```bash
158+
# Generate architecture support table
159+
./script/multi-arch-tracking/generate-table.py --format markdown
160+
161+
# Output formats: markdown, csv, jira, text
162+
./script/multi-arch-tracking/generate-table.py --format csv --output arch_support.csv
163+
```
164+
165+
### Google Sheets Integration
166+
167+
When using CSV format, the output includes `=HYPERLINK()` formulas for Jira issue references:
168+
- The CSV is formatted with proper escaping for Google Sheets import
169+
- After importing the CSV into Google Sheets (File → Import):
170+
1. Select all cells containing formulas
171+
2. Go to **Format → Number → Automatic**
172+
3. Formulas will activate and become clickable hyperlinks
173+
- Note: Formulas are imported as plain text initially and require manual activation
174+
175+
### Configuration
176+
177+
The `script/multi-arch-tracking/exceptions.toml` file tracks:
178+
- **Accelerator incompatibility rules**: Defines which architectures are incompatible with each accelerator type (CUDA, ROCm, Gaudi, Spyre)
179+
- **Specific exceptions**: Tracks component/architecture combinations that cannot be built, with Jira issue references
180+
181+
### Cell Values in Tables
182+
183+
- **Y** - Component is built for this architecture
184+
- **N/A** - Accelerator incompatibility (e.g., CUDA not available on ppc64le)
185+
- **Issue Key** (e.g., RHOAIENG-12345) - Specific exception tracked in Jira
186+
- **XXX** - Exception without assigned Jira issue
187+
- **(empty)** - Not currently built, but architecturally possible
188+
189+
### Common Architecture Patterns
190+
191+
- **Core platform components**: Built for all 4 architectures (amd64, arm64, ppc64le, s390x)
192+
- **CUDA components**: Only amd64 and sometimes arm64 (NVIDIA GPU requirement)
193+
- **ROCm components**: Only amd64 (AMD GPU requirement)
194+
- **Gaudi/Spyre components**: Only amd64 (Intel accelerator requirement)
195+
- **Data Science Pipelines**: amd64, arm64, ppc64le (no s390x)
196+
197+
See `script/multi-arch-tracking/README.md` for complete documentation.

0 commit comments

Comments
 (0)