-
Notifications
You must be signed in to change notification settings - Fork 143
Add CI automation framework to the CLI for open-source database build and test #845
Description
Is your feature request related to a problem? Please describe.
Currently HammerDB has no built-in CI integration for open-source databases hosted on GitHub (e.g., MariaDB, MySQL, PostgreSQL). This makes it difficult to automatically clone, build, install, and benchmark new releases or pull-request branches. Users must manually track upstream changes, run builds, execute tests, and compare benchmark results, which is slow, error-prone, and inconsistent. A dedicated CI automation framework within HammerDB would allow fully repeatable performance testing triggered directly by upstream GitHub events.
Describe the solution you'd like
Introduce a first version of HammerDB CI automation, enabling automatic clone/build/install/test pipelines for MariaDB (initially), with config stored in ci.xml and synchronized to HammerDB jobs/web service.
Describe alternatives you've considered
HammerDB already includes CLI scripts to automate the test process, however build processes are manual. Including a pipeline in HammerDB can enable testing of a tag or commit by automating the entire pipeline.
Additional context
New CI Commands
cilistencistatuscistopcitmpcipushcistepciset
CI Configuration (ci.xml)
- Common/global settings
- MariaDB build / install / test / pipeline definitions
- SQLite-backed mirroring of the CI config
CI Pipeline
- Job watcher (polling JOBTEST table)
- Webhook listener (GitHub-compatible)
- Pipeline executor implementing:
clonebuildpackageinstallstartrun tests- performance profile comparison (
compare_profileid)
tests are user definable and run the scripts in the scripts directory.
cistep allows breaking down the pipeline into different stages to e.g. build and not test.
CLI Enhancements
print ciupdated to show configuration
New Performance Profile Features
jobs diff <id1> <id2>
Compare two performance profiles and show average performance difference.
hammerdb>jobs diff 3333 3334
Profiles compared (unweighted): matched=12, avg_good=498585, avg_bad=493675
-0.01
jobs profileid all
Display all previously used performance profile IDs, including:- Profile ID
- Number of jobs in each profile
- Max NOPM/TPM/AVU per profile
e.g.
hammerdb>jobs profileid all
[
"3333",
"3334"
]
Enhance Web Service for performance profile comparison and diff