ggml : introduce semantic versioning

[danbev]

This commit introduces semantic versioning for the GGML library.

The motivation for this is that the current versioning, using build numbers, makes it difficult to track changes and releases for projects that use ggml.

Suggested release process/steps:

1. Sync the changes from llama.cpp using sync-llama-am.sh and after the PR has been approved and merged move to step 2.
2. Run scripts/release.sh and specify the type of release, major, minor, or patch. This script will handle incrementing the version (major|minor|patch), create a new commit with the version change, create a tag for the version, and prepare for the next development iteration.
3. Inspect the commits/tag and push the release candidate branch created (will be in the format ggml-rc-v1.0.0) and open a pull request.
4. Once the pull request has been merged, the tag can be pushed to master. This will trigger the github release workflow which is triggered for new tags which will then publish a new release on github.
5. The changes in ggml need to be synced back to llama.cpp and to whisper.cpp by running scripts/sync-ggml-am.sh in those repositories.

Example usage:

$ ./scripts/release.sh major --dry-run
[dry-run] - No changes will be made

[dry run] Warning: Not on master branch (currently on: semantic-versioning). Continuing with dry-run...

Step 1: Reading current version...
Current version: 0.9.0-dev
New release version: 1.0.0
Release candidate branch: ggml-rc-v1.0.0

Step 2: Creating release candidate branch...
  [dry-run] Would create branch: ggml-rc-v1.0.0

Step 3: Updating version in CMakeLists.txt...
  [dry-run] Would update GGML_VERSION_MAJOR to 1
  [dry-run] Would update GGML_VERSION_MINOR to 0
  [dry-run] Would update GGML_VERSION_PATCH to 0
  [dry-run] Would remove -dev suffix

Step 4: Committing version bump...
  [dry-run] Would commit: 'ggml : bump version to 1.0.0'

Step 5: Creating signed git tag...
  [dry-run] Would create signed tag: v1.0.0 with message 'Release version 1.0.0'

Step 6: Preparing for next development cycle...
  [dry-run] Would update GGML_VERSION_MINOR to 1
  [dry-run] Would add -dev suffix back

Step 7: Committing development version...
  [dry-run] Would commit: 'ggml : prepare for development of 1.1.0-dev'

[dry-run] Summary (no changes were made):
  • Would have created branch: ggml-rc-v1.0.0
  • Would have created tag: v1.0.0
  • Would have set next development version: 1.1.0-dev

Refs: #1333

For changes to the core ggml library (including to the CMake build system), please open a PR in https://github.com/ggml-org/llama.cpp. Doing so will make your PR more visible, better tested and more likely to be reviewed.

This PR requires adding a file to the scripts directory which is not available in llama.cpp and is the reason for opening this PR here instead of in llama.cpp. It also contains changes to CMakeLists.txt but it would make it more difficult to test and review separating the commits/prs.

[danbev]

@ggerganov @slaren Any thoughts about having a process like this?
Is there anything I've not taken into account?

[ggerganov]

After making a release there is one more step to sync back ggml into llama.cpp and whisper.cpp. Otherwise, they would point to the old ggml version.

[danbev]

A suggestion to handle the CI failure has been opened in ggml-org/llama.cpp#15922.

[danbev]

@ggerganov When get a chance can you give this another review and see what you think?

[ggerganov]

I guess we can give it a try.

I'm hesitating because I'm a bit unsure about how the version is going to be used by downstream projects. We can adapt based on feedback.

[ckastner]

Speaking as a downstream, this is great!

Some additional feedback:

• Any versioning implies tags/releases (I guess?) and most packaging tooling just works better with that, so that's a help
• Best thing about this though is proper package relationships: For distributions with a standalone ggml, the llama.cpp -> ggml dependency can now be expressed via major/minor which is much simpler and cleaner. Right now, we still need to rely on git commits
• If this works out, the semantic major version could eventually become the SOVERSION, which would be fantastic from a distribution POV

[ggerganov]

I spotted 2 problems with the release script:

• When doing a patch release, the version jumps with 2 numbers: 0.9.1 -> 0.9.3 -> 0.9.5` -> etc.
• The tagged commit is the one in the RC branch, not the one merged in master. For example, for the v0.9.1 tag, the commit is ebdf2fb, while the one in the master branch is 8c18eab. Not sure if it really matters, but we might want to update the workflow with instructions to create the tag manually after the merge