Skip to content

Add CI automation framework to the CLI for open-source database build and test #845

@sm-shaw

Description

@sm-shaw

Is your feature request related to a problem? Please describe.
Currently HammerDB has no built-in CI integration for open-source databases hosted on GitHub (e.g., MariaDB, MySQL, PostgreSQL). This makes it difficult to automatically clone, build, install, and benchmark new releases or pull-request branches. Users must manually track upstream changes, run builds, execute tests, and compare benchmark results, which is slow, error-prone, and inconsistent. A dedicated CI automation framework within HammerDB would allow fully repeatable performance testing triggered directly by upstream GitHub events.

Describe the solution you'd like
Introduce a first version of HammerDB CI automation, enabling automatic clone/build/install/test pipelines for MariaDB (initially), with config stored in ci.xml and synchronized to HammerDB jobs/web service.

Describe alternatives you've considered
HammerDB already includes CLI scripts to automate the test process, however build processes are manual. Including a pipeline in HammerDB can enable testing of a tag or commit by automating the entire pipeline.

Additional context

New CI Commands

  • cilisten
  • cistatus
  • cistop
  • citmp
  • cipush
  • cistep
  • ciset

CI Configuration (ci.xml)

  • Common/global settings
  • MariaDB build / install / test / pipeline definitions
  • SQLite-backed mirroring of the CI config

CI Pipeline

  • Job watcher (polling JOBTEST table)
  • Webhook listener (GitHub-compatible)
  • Pipeline executor implementing:
    • clone
    • build
    • package
    • install
    • start
    • run tests
    • performance profile comparison (compare_profileid)

tests are user definable and run the scripts in the scripts directory.
cistep allows breaking down the pipeline into different stages to e.g. build and not test.

CLI Enhancements

  • print ci updated to show configuration

New Performance Profile Features

  • jobs diff <id1> <id2>
    Compare two performance profiles and show average performance difference.
hammerdb>jobs diff 3333 3334
Profiles compared (unweighted): matched=12, avg_good=498585, avg_bad=493675
-0.01
  • jobs profileid all
    Display all previously used performance profile IDs, including:
    • Profile ID
    • Number of jobs in each profile
    • Max NOPM/TPM/AVU per profile
      e.g.
hammerdb>jobs profileid all
[
  "3333",
  "3334"
]

Enhance Web Service for performance profile comparison and diff

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions