CI/CD Workflow Documentation

This document explains the GitHub Actions workflow for building the ML Dataset Preparation Tool across all three supported platforms.

Overview

The workflow automatically builds standalone executables/bundles for:

Windows 10 - Standalone .exe with Qt DLLs
macOS 10.14+ - Native .app bundle with Qt frameworks
Linux/Ubuntu 18.04+ - Binary with launcher scripts

Workflow Triggers

The workflow runs on:

Push to master, main, or develop branches
Pull requests to master, main, or develop branches
Manual trigger via GitHub Actions UI (workflow_dispatch)

Build Jobs

1. Windows Build (`build-windows`)

Runner: windows-latest (Windows Server 2022)

Steps:

Checkout code - Clone the repository
Install Qt 5.15.2 - Using jurplel/install-qt-action@v3
- Architecture: win64_msvc2019_64
- No modules specified (qtbase included by default)
- Caching enabled for faster subsequent builds
Configure MSVC - Setup Visual Studio 2019 build tools
Clean build directory - Remove any existing build artifacts
Configure CMake - Generate Visual Studio 2019 solution
Build - Compile in Release mode
Verify build - Check that MLDatasetTool.exe exists
Deploy Qt dependencies - Run windeployqt to bundle DLLs
Package - Copy all files to artifact directory
Upload artifact - Upload to GitHub Actions artifacts

Output:

MLDatasetTool.exe - Main executable
Qt DLLs (QtCore5.dll, QtGui5.dll, QtWidgets5.dll, etc.)
Platform plugins (qwindows.dll)
Image format plugins (qjpeg.dll, qpng.dll, etc.)

Artifact Name: MLDatasetTool-Windows

2. Linux Build (`build-linux`)

Runner: ubuntu-latest (Ubuntu 22.04)

Steps:

Checkout code - Clone the repository
Install dependencies - Install build tools and Qt5
- build-essential, cmake
- qtbase5-dev, qtbase5-dev-tools, qt5-qmake
Verify Qt installation - Check qmake and cmake versions
Clean build directory - Remove any existing build artifacts
Configure CMake - Generate Unix Makefiles
Build - Compile using make with all CPU cores
Verify build - Check that ImageClassificationTool exists
Package - Copy binary, launcher scripts, and README
Upload artifact - Upload to GitHub Actions artifacts

Output:

ImageClassificationTool - Main executable
run_app.sh - Recommended launcher script
launch_clean.sh - Alternative launcher with maximum isolation
README.md - Documentation

Artifact Name: MLDatasetTool-Linux

3. macOS Build (`build-macos`)

Runner: macos-latest (macOS 12 Monterey)

Steps:

Checkout code - Clone the repository
Install Qt 5.15.2 - Using jurplel/install-qt-action@v3
- No modules specified (qtbase included by default)
- Caching enabled for faster subsequent builds
Verify Qt installation - Check qmake version
Clean build directory - Remove any existing build artifacts
Configure CMake - Generate Unix Makefiles with macOS deployment target
Build - Compile using make with all CPU cores
Verify build - Check that MLDatasetTool.app exists
Deploy Qt dependencies - Run macdeployqt to bundle frameworks
Verify Qt frameworks - Check that frameworks are bundled
Package - Copy .app bundle to artifact directory
Create DMG (optional) - Create disk image for distribution
Upload artifacts - Upload .app and DMG to GitHub Actions

Output:

MLDatasetTool.app - Application bundle with Qt frameworks
MLDatasetTool.dmg - Disk image (optional)

Artifact Names:

MLDatasetTool-macOS (app bundle)
MLDatasetTool-macOS-DMG (disk image)

4. Release Summary (`create-release`)

Runner: ubuntu-latest

Dependencies: Requires all three build jobs to complete successfully

Condition: Only runs on push to master or main branches

Steps:

Download all artifacts - Download from all build jobs
Display structure - Show artifact contents and sizes
Create release info - Generate markdown summary with:
- Build metadata (commit, branch, date)
- Artifact descriptions for each platform
- Usage instructions
- Feature highlights
- Documentation links
Upload release info - Upload summary as artifact

Output:

release-info.md - Comprehensive build summary

Artifact Name: release-info

Key Fixes Applied

1. Qt Installation Fix (Windows & macOS)

Problem:

The packages ['qtbase'] were not found while parsing XML of package information!

Root Cause: The --modules qtbase flag was incorrect. In Qt5, qtbase is not a separate module but the base installation that's included by default.

Solution: Removed the modules: 'qtbase' line from the Qt installation step. The base Qt5 installation includes all necessary components (QtCore, QtGui, QtWidgets) by default.

Before:

- name: Install Qt
  uses: jurplel/install-qt-action@v3
  with:
    version: '5.15.2'
    modules: 'qtbase'  # ❌ Incorrect

After:

- name: Install Qt
  uses: jurplel/install-qt-action@v3
  with:
    version: '5.15.2'
    # ✓ No modules specified - qtbase included by default

2. Build Directory Conflict Fix (All Platforms)

Problem:

mkdir: cannot create directory 'build': File exists

Root Cause: Build directory persisted from previous runs or caching, causing conflicts.

Solution: Added explicit build directory cleanup before creating new one.

Windows:

if (Test-Path build) { Remove-Item -Recurse -Force build }

Linux/macOS:

rm -rf build

3. CMake Generator Fix (Windows)

Problem: Using Ninja generator which may not be available or properly configured.

Solution: Changed to Visual Studio 2019 generator which is guaranteed to be available on windows-latest runners.

Before:

cmake .. -G "Ninja" -DCMAKE_BUILD_TYPE=Release

After:

cmake .. -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Release

4. Output Verification (All Platforms)

Problem: Builds could fail silently without clear error messages.

Solution: Added explicit verification steps that check for expected output files and provide clear success/failure messages.

Example (Windows):

if (Test-Path build\Release\MLDatasetTool.exe) {
  Write-Host "✓ MLDatasetTool.exe built successfully"
} else {
  Write-Error "✗ MLDatasetTool.exe not found!"
  exit 1
}

5. macOS Deployment Target

Problem: No minimum macOS version specified, potentially causing compatibility issues.

Solution: Added CMAKE_OSX_DEPLOYMENT_TARGET=10.14 to ensure compatibility with macOS 10.14 (Mojave) and later.

cmake .. -DCMAKE_OSX_DEPLOYMENT_TARGET=10.14

Parallel Execution

All three build jobs run in parallel for maximum efficiency:

No dependencies between platform builds
Each job has isolated environment
Build directory cleanup prevents conflicts
Caching is platform-specific

Benefits:

Faster total build time (~10-15 minutes vs 30-45 minutes sequential)
Independent failure isolation
Efficient resource utilization

Artifact Retention

All artifacts are retained for 30 days by default. This can be adjusted in the workflow file:

retention-days: 30

Downloading Artifacts

Via GitHub UI

Go to the repository's "Actions" tab
Click on a workflow run
Scroll to "Artifacts" section
Click on artifact name to download

Via GitHub CLI

# List artifacts for a run
gh run view <run-id> --log

# Download specific artifact
gh run download <run-id> -n MLDatasetTool-Windows

# Download all artifacts
gh run download <run-id>

Local Testing

To test the workflow locally before pushing:

Using act (GitHub Actions local runner)

# Install act
brew install act  # macOS
# or
curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash  # Linux

# Run workflow locally
act push

# Run specific job
act -j build-windows
act -j build-linux
act -j build-macos

Note: Local testing has limitations:

May not perfectly replicate GitHub's environment
Some actions may not work locally
Useful for catching obvious errors

Troubleshooting

Build Fails on Windows

Check:

Qt installation succeeded
MSVC tools are available
CMake can find Qt (CMAKE_PREFIX_PATH set correctly)
windeployqt is in PATH

Common Issues:

Qt installation timeout (increase timeout or retry)
MSVC version mismatch (ensure Visual Studio 2019)
Path issues (check $env:Qt5_DIR)

Build Fails on Linux

Check:

All dependencies installed (qtbase5-dev, etc.)
Qt version is 5.x (not Qt6)
Build directory cleaned properly
Sufficient disk space

Common Issues:

Missing dependencies (add to apt-get install)
Qt4 vs Qt5 conflicts (ensure Qt5 packages)
Permission issues (shouldn't occur on GitHub runners)

Build Fails on macOS

Check:

Qt installation succeeded
Xcode Command Line Tools available
macdeployqt runs successfully
Frameworks bundled in .app

Common Issues:

Qt installation timeout (increase timeout or retry)
macdeployqt not found (check Qt installation)
Framework bundling fails (check disk space)

Artifacts Not Uploaded

Check:

Build step completed successfully
Artifact path exists
Artifact size within limits (GitHub has size limits)
Upload action version is current

Common Issues:

Path typos in upload step
Artifact too large (>2GB limit per artifact)
Network issues (retry workflow)

Workflow Optimization

Current Optimizations

Caching: Qt installations are cached to speed up subsequent runs
Parallel execution: All platforms build simultaneously
Minimal dependencies: Only install what's needed
Efficient packaging: Only include necessary files

Future Improvements

Matrix builds: Build multiple Qt versions or configurations
Conditional builds: Skip unchanged platforms
Incremental builds: Cache build artifacts between runs
Automated testing: Run tests before packaging
Code signing: Sign executables for distribution
Release automation: Auto-create GitHub releases

Security Considerations

No secrets required: Workflow uses only public actions
Read-only checkout: Code is checked out read-only
Isolated environments: Each job runs in fresh VM
Artifact encryption: GitHub encrypts artifacts at rest
Limited retention: Artifacts auto-delete after 30 days

Monitoring

Workflow Status

Check workflow status:

Repository "Actions" tab
Commit status checks
Email notifications (if enabled)
GitHub mobile app

Build Metrics

Monitor:

Build duration (target: <15 minutes total)
Success rate (target: >95%)
Artifact sizes (Windows: ~20MB, macOS: ~25MB, Linux: ~5MB)
Cache hit rate (target: >80%)

Support

For workflow issues:

Check workflow logs in GitHub Actions
Review this documentation
Check platform-specific build guides:
- Windows: BUILD_WINDOWS.md
- macOS: BUILD_MACOS.md
- Linux: README.md
Open an issue with workflow logs attached

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI/CD Workflow Documentation

Overview

Workflow Triggers

Build Jobs

1. Windows Build (`build-windows`)

2. Linux Build (`build-linux`)

3. macOS Build (`build-macos`)

4. Release Summary (`create-release`)

Key Fixes Applied

1. Qt Installation Fix (Windows & macOS)

2. Build Directory Conflict Fix (All Platforms)

3. CMake Generator Fix (Windows)

4. Output Verification (All Platforms)

5. macOS Deployment Target

Parallel Execution

Artifact Retention

Downloading Artifacts

Via GitHub UI

Via GitHub CLI

Local Testing

Using act (GitHub Actions local runner)

Troubleshooting

Build Fails on Windows

Build Fails on Linux

Build Fails on macOS

Artifacts Not Uploaded

Workflow Optimization

Current Optimizations

Future Improvements

Security Considerations

Monitoring

Workflow Status

Build Metrics

Support

References

FilesExpand file tree

CI_CD_WORKFLOW.md

Latest commit

History

CI_CD_WORKFLOW.md

File metadata and controls

CI/CD Workflow Documentation

Overview

Workflow Triggers

Build Jobs

1. Windows Build (build-windows)

2. Linux Build (build-linux)

3. macOS Build (build-macos)

4. Release Summary (create-release)

Key Fixes Applied

1. Qt Installation Fix (Windows & macOS)

2. Build Directory Conflict Fix (All Platforms)

3. CMake Generator Fix (Windows)

4. Output Verification (All Platforms)

5. macOS Deployment Target

Parallel Execution

Artifact Retention

Downloading Artifacts

Via GitHub UI

Via GitHub CLI

Local Testing

Using act (GitHub Actions local runner)

Troubleshooting

Build Fails on Windows

Build Fails on Linux

Build Fails on macOS

Artifacts Not Uploaded

Workflow Optimization

Current Optimizations

Future Improvements

Security Considerations

Monitoring

Workflow Status

Build Metrics

Support

References

1. Windows Build (`build-windows`)

2. Linux Build (`build-linux`)

3. macOS Build (`build-macos`)

4. Release Summary (`create-release`)