Skip to content

Commit 2988311

Browse files
MaciejKarasm1kola
andauthored
CLOUDP-295785 - Calculate next version and release notes script (#193)
# Summary Initial work for calculating next operator version and release notes. This is based on https://docs.google.com/document/d/1eJ8iKsI0libbpcJakGjxcPfbrTn8lmcZDbQH1UqMR_g/edit?tab=t.aov8hxoyxtos#heading=h.6q8mwv95pdu6. **This is not yet integrated with CI nor forces engineers to create change log files. There will be next smaller PRs that will address missing integration. Functionalities added by this PR:** 1. `scripts.release.calculate_next_version.py` -> this will calculate next version based on the changelog files provided. ```sh ➜ mongodb-kubernetes ✗ python3 -m scripts.release.calculate_next_version --help usage: calculate_next_version.py [-h] [-p ] [-c ] [-s ] [-v ] Calculate the next version based on the changes since the previous version tag. options: -h, --help show this help message and exit -p, --path Path to the Git repository. Default is the current directory '.' -c, --changelog-path Path to the changelog directory relative to the repository root. Default is 'changelog/' -s, --initial-commit-sha SHA of the initial commit to start from if no previous version tag is found. -v, --initial-version Version to use if no previous version tag is found. Default is '1.0.0' ``` 2. `scripts.release.create_changelog.py` -> this is a tool for creating changelog files. It makes life easier for engineers and reduces a chance for errors in filename and contents syntax ```sh ➜ mongodb-kubernetes ✗python3 -m scripts.release.create_changelog --help usage: create_changelog.py [-h] [-c ] [-d ] [-e] -k title Utility to easily create a new changelog entry file. positional arguments: title Title for the changelog entry options: -h, --help show this help message and exit -c, --changelog-path Path to the changelog directory relative to a current working directory. Default is 'changelog/' -d, --date Date in 'YYYY-MM-DD' format to use for the changelog entry. Default is today's date -e, --editor Open the created changelog entry in the default editor (if set, otherwise uses 'vi'). Default is True -k, --kind Kind of the changelog entry: - 'prelude' for prelude entries - 'breaking' for breaking change entries - 'feature' for feature entries - 'fix' for bugfix entries - 'other' for other entries ``` 3. `scripts.release.release_notes.py` -> create release notes based on the changes since last release tag ```sh ➜ mongodb-kubernetes ✗ python3 -m scripts.release.release_notes --help usage: release_notes.py [-h] [-p ] [-c ] [-s ] [-v ] [--output ] Generate release notes based on the changes since the previous version tag. options: -h, --help show this help message and exit -p, --path Path to the Git repository. Default is the current directory '.' -c, --changelog-path Path to the changelog directory relative to the repository root. Default is 'changelog/' -s, --initial-commit-sha SHA of the initial commit to start from if no previous version tag is found. -v, --initial-version Version to use if no previous version tag is found. Default is '1.0.0' --output, -o Path to save the release notes. If not provided, prints to stdout. ``` ## Proof of Work Passing unit tests for now is enough. ## Checklist - [x] Have you linked a jira ticket and/or is the ticket in the title? - [x] Have you checked whether your jira ticket required DOCSP changes? - [x] Have you checked for release_note changes? ## Reminder (Please remove this when merging) - Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible - Our Short Guide for PRs: [Link](https://docs.google.com/document/d/1T93KUtdvONq43vfTfUt8l92uo4e4SEEvFbIEKOxGr44/edit?tab=t.0) - Remember the following Communication Standards - use comment prefixes for clarity: * **blocking**: Must be addressed before approval. * **follow-up**: Can be addressed in a later PR or ticket. * **q**: Clarifying question. * **nit**: Non-blocking suggestions. * **note**: Side-note, non-actionable. Example: Praise * --> no prefix is considered a question --------- Co-authored-by: Mikalai Radchuk <[email protected]>
1 parent 07a91dc commit 2988311

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+1341
-0
lines changed

requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ pytest-mock==3.14.1
3333
wrapt==1.17.2
3434
botocore==1.39.4
3535
boto3==1.39.4
36+
python-frontmatter==1.1.0
3637

3738
# from kubeobject
3839
freezegun==1.5.3

scripts/release/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Makes 'release' a Python package.
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
import argparse
2+
import pathlib
3+
4+
from git import Repo
5+
6+
from scripts.release.changelog import (
7+
DEFAULT_CHANGELOG_PATH,
8+
DEFAULT_INITIAL_GIT_TAG_VERSION,
9+
)
10+
from scripts.release.release_notes import calculate_next_version_with_changelog
11+
12+
if __name__ == "__main__":
13+
parser = argparse.ArgumentParser(
14+
description="Calculate the next version based on the changes since the previous version tag.",
15+
formatter_class=argparse.RawTextHelpFormatter,
16+
)
17+
parser.add_argument(
18+
"-p",
19+
"--path",
20+
default=".",
21+
metavar="",
22+
action="store",
23+
type=pathlib.Path,
24+
help="Path to the Git repository. Default is the current directory '.'",
25+
)
26+
parser.add_argument(
27+
"-c",
28+
"--changelog-path",
29+
default=DEFAULT_CHANGELOG_PATH,
30+
metavar="",
31+
action="store",
32+
type=str,
33+
help=f"Path to the changelog directory relative to the repository root. Default is '{DEFAULT_CHANGELOG_PATH}'",
34+
)
35+
parser.add_argument(
36+
"-s",
37+
"--initial-commit-sha",
38+
metavar="",
39+
action="store",
40+
type=str,
41+
help="SHA of the initial commit to start from if no previous version tag is found.",
42+
)
43+
parser.add_argument(
44+
"-v",
45+
"--initial-version",
46+
default=DEFAULT_INITIAL_GIT_TAG_VERSION,
47+
metavar="",
48+
action="store",
49+
type=str,
50+
help=f"Version to use if no previous version tag is found. Default is '{DEFAULT_INITIAL_GIT_TAG_VERSION}'",
51+
)
52+
args = parser.parse_args()
53+
54+
repo = Repo(args.path)
55+
56+
version, _ = calculate_next_version_with_changelog(
57+
repo, args.changelog_path, args.initial_commit_sha, args.initial_version
58+
)
59+
60+
print(version)

scripts/release/changelog.py

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
import datetime
2+
import os
3+
import re
4+
from enum import StrEnum
5+
6+
import frontmatter
7+
from git import Commit, Repo
8+
9+
DEFAULT_CHANGELOG_PATH = "changelog/"
10+
DEFAULT_INITIAL_GIT_TAG_VERSION = "1.0.0"
11+
FILENAME_DATE_FORMAT = "%Y%m%d"
12+
FRONTMATTER_DATE_FORMAT = "%Y-%m-%d"
13+
MAX_TITLE_LENGTH = 50
14+
15+
16+
class ChangeKind(StrEnum):
17+
PRELUDE = "prelude"
18+
BREAKING = "breaking"
19+
FEATURE = "feature"
20+
FIX = "fix"
21+
OTHER = "other"
22+
23+
@staticmethod
24+
def from_str(kind_str: str) -> "ChangeKind":
25+
kind_str_lower = kind_str.lower()
26+
if kind_str_lower == str(ChangeKind.PRELUDE):
27+
return ChangeKind.PRELUDE
28+
if kind_str_lower == str(ChangeKind.BREAKING):
29+
return ChangeKind.BREAKING
30+
elif kind_str_lower == str(ChangeKind.FEATURE):
31+
return ChangeKind.FEATURE
32+
elif kind_str_lower == str(ChangeKind.FIX):
33+
return ChangeKind.FIX
34+
elif kind_str_lower == str(ChangeKind.OTHER):
35+
return ChangeKind.OTHER
36+
raise ValueError(f"unknown change kind: {kind_str}")
37+
38+
39+
class ChangeEntry:
40+
def __init__(self, date: datetime, kind: ChangeKind, title: str, contents: str):
41+
self.date = date
42+
self.kind = kind
43+
self.title = title
44+
self.contents = contents
45+
46+
47+
def get_changelog_entries(
48+
base_commit: Commit,
49+
repo: Repo,
50+
changelog_sub_path: str,
51+
) -> list[ChangeEntry]:
52+
changelog = []
53+
54+
# Compare base commit with current working tree
55+
diff_index = base_commit.diff(other=repo.head.commit, paths=changelog_sub_path)
56+
57+
# No changes since the previous version
58+
if not diff_index:
59+
return changelog
60+
61+
# Traverse added Diff objects only (change type 'A' for added files)
62+
# Because we are traversing back to the most recent version, we only care about files that were added since then
63+
# If a file was added in one commit and then modified in another, we only care about the final version of the file
64+
# Same for deletions - if file was added in one commit but then deleted in another, we don't want to include it
65+
# in the Release Notes
66+
for diff_item in diff_index.iter_change_type("A"):
67+
file_path = diff_item.b_path
68+
69+
change_entry = extract_changelog_entry(repo.working_dir, file_path)
70+
changelog.append(change_entry)
71+
72+
return changelog
73+
74+
75+
def extract_changelog_entry(working_dir: str, file_path: str) -> ChangeEntry:
76+
file_name = os.path.basename(file_path)
77+
date, kind = extract_date_and_kind_from_file_name(file_name)
78+
79+
abs_file_path = os.path.join(working_dir, file_path)
80+
with open(abs_file_path, "r") as file:
81+
file_content = file.read()
82+
83+
change_entry = extract_changelog_entry_from_contents(file_content)
84+
85+
if change_entry.date != date:
86+
raise Exception(
87+
f"{file_name} - date in front matter '{change_entry.date}' does not match date extracted from file name '{date}'"
88+
)
89+
90+
if change_entry.kind != kind:
91+
raise Exception(
92+
f"{file_name} - kind in front matter '{change_entry.kind}' does not match kind extracted from file name '{kind}'"
93+
)
94+
95+
return change_entry
96+
97+
98+
def extract_date_and_kind_from_file_name(file_name: str) -> (datetime, ChangeKind):
99+
match = re.match(r"(\d{8})_([a-zA-Z]+)_(.+)\.md", file_name)
100+
if not match:
101+
raise Exception(f"{file_name} - doesn't match expected pattern")
102+
103+
date_str, kind_str, _ = match.groups()
104+
try:
105+
date = parse_change_date(date_str, FILENAME_DATE_FORMAT)
106+
except Exception as e:
107+
raise Exception(f"{file_name} - {e}")
108+
109+
try:
110+
kind = ChangeKind.from_str(kind_str)
111+
except Exception as e:
112+
raise Exception(f"{file_name} - {e}")
113+
114+
return date, kind
115+
116+
117+
def parse_change_date(date_str: str, date_format: str) -> datetime:
118+
try:
119+
date = datetime.datetime.strptime(date_str, date_format).date()
120+
except Exception:
121+
raise Exception(f"date '{date_str}' is not in the expected format {date_format}")
122+
123+
return date
124+
125+
126+
def extract_changelog_entry_from_contents(file_contents: str) -> ChangeEntry:
127+
data = frontmatter.loads(file_contents)
128+
129+
kind = ChangeKind.from_str(str(data["kind"]))
130+
date = parse_change_date(str(data["date"]), FRONTMATTER_DATE_FORMAT)
131+
## Add newline to contents so the Markdown file also contains a newline at the end
132+
contents = data.content + "\n"
133+
134+
return ChangeEntry(date=date, title=str(data["title"]), kind=kind, contents=contents)
135+
136+
137+
def get_changelog_filename(title: str, kind: ChangeKind, date: datetime) -> str:
138+
sanitized_title = sanitize_title(title)
139+
filename_date = datetime.datetime.strftime(date, FILENAME_DATE_FORMAT)
140+
141+
return f"{filename_date}_{kind}_{sanitized_title}.md"
142+
143+
144+
def sanitize_title(title: str) -> str:
145+
# Only keep alphanumeric characters, dashes, underscores and spaces
146+
regex = re.compile("[^a-zA-Z0-9-_ ]+")
147+
title = regex.sub("", title)
148+
149+
# Replace multiple dashes, underscores and spaces with underscores
150+
regex_underscore = re.compile("[-_ ]+")
151+
title = regex_underscore.sub(" ", title).strip()
152+
153+
# Lowercase and split by space
154+
words = [word.lower() for word in title.split(" ")]
155+
156+
result = words[0]
157+
158+
for word in words[1:]:
159+
if len(result) + len("_") + len(word) <= MAX_TITLE_LENGTH:
160+
result = result + "_" + word
161+
else:
162+
break
163+
164+
return result

scripts/release/changelog_test.py

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
import datetime
2+
import unittest
3+
4+
from scripts.release.changelog import (
5+
MAX_TITLE_LENGTH,
6+
ChangeKind,
7+
extract_changelog_entry_from_contents,
8+
extract_date_and_kind_from_file_name,
9+
sanitize_title,
10+
)
11+
12+
13+
class TestExtractChangelogDataFromFileName(unittest.TestCase):
14+
def test_prelude(self):
15+
date, kind = extract_date_and_kind_from_file_name("20250502_prelude_release_notes.md")
16+
self.assertEqual(date, datetime.date(2025, 5, 2))
17+
self.assertEqual(kind, ChangeKind.PRELUDE)
18+
19+
def test_breaking_changes(self):
20+
date, kind = extract_date_and_kind_from_file_name("20250101_breaking_api_update.md")
21+
self.assertEqual(date, datetime.date(2025, 1, 1))
22+
self.assertEqual(kind, ChangeKind.BREAKING)
23+
24+
def test_features(self):
25+
date, kind = extract_date_and_kind_from_file_name("20250509_feature_new_dashboard.md")
26+
self.assertEqual(date, datetime.date(2025, 5, 9))
27+
self.assertEqual(kind, ChangeKind.FEATURE)
28+
29+
def test_fixes(self):
30+
date, kind = extract_date_and_kind_from_file_name("20251210_fix_olm_missing_images.md")
31+
self.assertEqual(date, datetime.date(2025, 12, 10))
32+
self.assertEqual(kind, ChangeKind.FIX)
33+
34+
def test_other(self):
35+
date, kind = extract_date_and_kind_from_file_name("20250520_other_update_readme.md")
36+
self.assertEqual(date, datetime.date(2025, 5, 20))
37+
self.assertEqual(kind, ChangeKind.OTHER)
38+
39+
def test_invalid_date(self):
40+
with self.assertRaises(Exception) as context:
41+
extract_date_and_kind_from_file_name("20250640_other_codebase.md")
42+
self.assertEqual(
43+
str(context.exception),
44+
"20250640_other_codebase.md - date '20250640' is not in the expected format %Y%m%d",
45+
)
46+
47+
def test_wrong_file_name_format_date(self):
48+
with self.assertRaises(Exception) as context:
49+
extract_date_and_kind_from_file_name("202yas_other_codebase.md")
50+
self.assertEqual(str(context.exception), "202yas_other_codebase.md - doesn't match expected pattern")
51+
52+
def test_wrong_file_name_format_missing_title(self):
53+
with self.assertRaises(Exception) as context:
54+
extract_date_and_kind_from_file_name("20250620_change.md")
55+
self.assertEqual(str(context.exception), "20250620_change.md - doesn't match expected pattern")
56+
57+
58+
def test_strip_changelog_entry_frontmatter():
59+
file_contents = """
60+
---
61+
title: This is my change
62+
kind: feature
63+
date: 2025-07-10
64+
---
65+
66+
* **MongoDB**: public search preview release of MongoDB Search (Community Edition) is now available.
67+
* Added new property [spec.search](https://www.mongodb.com/docs/kubernetes/current/mongodb/specification/#spec-search) to enable MongoDB Search.
68+
"""
69+
70+
change_entry = extract_changelog_entry_from_contents(file_contents)
71+
72+
assert change_entry.title == "This is my change"
73+
assert change_entry.kind == ChangeKind.FEATURE
74+
assert change_entry.date == datetime.date(2025, 7, 10)
75+
assert (
76+
change_entry.contents
77+
== """* **MongoDB**: public search preview release of MongoDB Search (Community Edition) is now available.
78+
* Added new property [spec.search](https://www.mongodb.com/docs/kubernetes/current/mongodb/specification/#spec-search) to enable MongoDB Search.
79+
"""
80+
)
81+
82+
83+
class TestSanitizeTitle(unittest.TestCase):
84+
def test_basic_case(self):
85+
self.assertEqual(sanitize_title("Simple Title"), "simple_title")
86+
87+
def test_non_alphabetic_chars(self):
88+
self.assertEqual(sanitize_title("Title tha@t-_ contain's strange char&s!"), "title_that_contains_strange_chars")
89+
90+
def test_with_numbers_and_dashes(self):
91+
self.assertEqual(sanitize_title("Title with 123 numbers to-go!"), "title_with_123_numbers_to_go")
92+
93+
def test_mixed_case(self):
94+
self.assertEqual(sanitize_title("MiXeD CaSe TiTlE"), "mixed_case_title")
95+
96+
def test_length_limit(self):
97+
long_title = "This is a very long title that should be truncated because it exceeds the maximum length"
98+
sanitized_title = sanitize_title(long_title)
99+
self.assertTrue(len(sanitized_title) <= MAX_TITLE_LENGTH)
100+
self.assertEqual(sanitized_title, "this_is_a_very_long_title_that_should_be_truncated")
101+
102+
def test_leading_trailing_spaces(self):
103+
sanitized_title = sanitize_title(" Title with spaces ")
104+
self.assertEqual(sanitized_title, "title_with_spaces")
105+
106+
def test_empty_title(self):
107+
self.assertEqual(sanitize_title(""), "")
108+
109+
def test_only_non_alphabetic(self):
110+
self.assertEqual(sanitize_title("!@#"), "")

0 commit comments

Comments
 (0)