Create robust packaging system

[connorgallopo]

Complete Configuration System Overhaul & Build Infrastructure

Overview

This PR represents a significant milestone in modernizing PiTrac's configuration management, build system, and packaging infrastructure. After extensive development and testing, I've successfully removed the dependency on manually building, compiling, or referencing documentation for configuration. The system now provides a clean, maintainable approach to managing PiTrac across different deployment scenarios.

What's Changed

Configuration System Revolution

Before: Configuration was scattered across JSON files, environment variables, and hardcoded values with no clear precedence or validation. Users had to manually edit JSON files and hope they got the structure right.

After: Implemented a hierarchical configuration system with clear precedence:

1. Default values in golf_sim_config.json
2. Environment variables
3. YAML configuration (pitrac.yaml)
4. CLI flags

The new ConfigurationManager class (496 lines of battle-tested C++) provides:

• Full backwards compatibility with existing JSON configs
• Type-safe parameter access with validation
• YAML override support for cleaner configuration
• Automatic environment variable mapping
• Real-time configuration reloading capabilities

Unified CLI Interface

Created a comprehensive pitrac command using Bashly that replaces dozens of scattered shell scripts:

# Before: ./RunScripts/SinglePi/runCam1ExternalStrobe.sh
# After:  pitrac run

# Before: Multiple confusing test scripts
# After:  pitrac test hardware
         pitrac test camera
         pitrac test gspro

The new CLI provides 40+ commands including:

• Core operations: run, stop, status
• Testing suite: test hardware, test quick, test camera, test pulse, test spin, test gspro
• Configuration: config show, config set, config backup, config restore, config migrate
• Service management: service start/stop/restart, tomee, activemq
• System utilities: logs, boot config, version

Production-Ready Packaging

Built a complete APT packaging system that actually works:

# One command to build everything
./packaging/build.sh all

# Create installable .deb package
./packaging/build-apt-package.sh

# Install on any Pi
sudo apt install ./pitrac_*.deb

The package includes:

• All compiled binaries and dependencies
• SystemD service files for automatic startup
• Proper file permissions and locations
• Test images and calibration data
• Complete removal/upgrade handling

Docker Build Infrastructure

Implemented multi-stage Docker builds for:

• Cross-compilation from x86 to ARM64
• Dependency caching (OpenCV, ActiveMQ, lgpio)
• Reproducible builds across environments
• CI/CD pipeline support

Key improvements:

• Build time reduced from 1+ hours to ~1 minutes with caching
• Support for both native Pi builds and cross-compilation
• Artifact management for pre-built dependencies

Comprehensive Documentation

Added developer documentation covering:

• Overview - System architecture and components
• Build System - Complete build instructions
• Configuration - How the new config system works
• Packaging - Creating APT packages and ISOs
• CLI Interface - Full command reference
• Testing - Test framework documentation
• Dependencies - Library management guide
• Docker - Container development workflow
• Services - SystemD integration

🔨 Core Application Improvements

Enhanced the C++ codebase with:

• New configuration_manager.cpp/h for centralized config handling
• Improved parameter validation and error messages
• Better separation of concerns between config and runtime

Technical Details

Configuration Priority System

The new system resolves configuration in this order (highest to lowest priority):

1. CLI Arguments - Direct command-line flags
2. YAML Config - User-friendly pitrac.yaml overrides
3. Environment Variables - PITRAC_* prefixed vars
4. JSON Config - Legacy golf_sim_config.json
5. Defaults - Hardcoded fallback values

Build System Architecture

packaging/
├── build.sh              # Master build orchestrator
├── build-apt-package.sh  # APT package creator
├── generate.sh           # CLI generator
├── scripts/              # Dependency builders
├── deps-artifacts/       # Pre-built dependencies
└── templates/            # Config templates

Key Files Changed

• Added complete packaging infrastructure
• Refactored configuration handling
• Created unified CLI interface
• Extensive documentation additions

Testing

Tested on:

• OrangePi 5 Plus - Native builds + Deb Install
• Raspberry Pi 5 (16GB) - Native builds + Deb Install
• Ubuntu 22.04 x86_64 - Cross-compilation
• Docker containers - CI/CD simulation

All test modes verified:

• pitrac test quick - Basic functionality
• pitrac test hardware - Full hardware test
• pitrac test camera - Camera detection
• pitrac test gspro - Simulator integration

Migration Guide

For existing users:

1. Your existing golf_sim_config.json will continue to work
2. Optionally create /etc/pitrac/pitrac.yaml for cleaner overrides
3. Use new pitrac CLI commands instead of old scripts
4. Run pitrac config migrate to convert JSON to YAML

Breaking Changes

None! Full backwards compatibility maintained. Old scripts and configs will continue to work, though the new methods are strongly recommended.

Future Work

This PR lays the groundwork for:

• ISO image generation for ready-to-flash SD cards
• Web-based configuration interface
• Automatic dependency updates
• Cloud-based build service

Acknowledgments

This was a massive undertaking that touched nearly every aspect of the PiTrac build and deployment process. The result is a much more maintainable, user-friendly system that should significantly lower the barrier to entry for new users while providing power users with the flexibility they need.

Special thanks to the Discord community for testing early versions and providing invaluable feedback throughout this process.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[jeshernandez]

A lot to review without testing. But looks good to me high level :P

[jeshernandez]

What is this Verdant consultants? Should we be adding to all of our code? I been meaning to ask.

[connorgallopo]

Just something I picked up on as a pattern in 180+ files and decided to continue on lol. Def worth shedding some light on

[jeshernandez]

I guess the only other action I would take on this PR is simply squashing some of the commits and summarize the overall feature in one commit?

1. git reset --soft HEAD~23
2. git add .
3. git commit -a -m "feature: bla bla"
4. git push -f // You have to use force.

[connorgallopo]

I'll look to squash and merge - want to keep the timeline for the time being so if I need to debug, drop, or test something I can.

But I'll def squash before it makes it to main

[connorgallopo]

recheck