Skip to content

Commit 271a5e6

Browse files
d3rkyd3rky
authored
MPT-18131: add version field to store the version when the migration was a… (#47)
…pplied <!-- This is an auto-generated comment: release notes by coderabbit.ai --> Closes [MPT-18131](https://softwareone.atlassian.net/browse/MPT-18131) - Added `SERVICE_VERSION` environment variable to persist the service version at the time each migration is applied - Extended `Migration` model with `version` (str | None) and `started_at` (dt.datetime | None) fields - Extended `MigrationListItem` model with `applied_at`, `migration_type`, `started_at`, and `version` fields - Added `get_service_version()` function to `mpt_tool/config.py` to retrieve the SERVICE_VERSION environment variable - Updated Airtable state manager to initialize, read, and persist version field in migration state records - Updated file-based state manager to populate version field when creating new migrations - Added version column to `MigrationRender` table display - Updated documentation and tests to reflect the new version field in migration state schema and output <!-- end of auto-generated comment: release notes by coderabbit.ai --> [MPT-18131]: https://softwareone.atlassian.net/browse/MPT-18131?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
2 parents 25adb09 + e310966 commit 271a5e6

File tree

9 files changed

+89
-37
lines changed

9 files changed

+89
-37
lines changed

‎README.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,12 @@ The following environment variables are typically set in `.env`. Docker Compose
113113

114114
### Application
115115

116-
| Environment Variable | Default | Example | Description |
117-
|----------------------------------------|-------------------------|-------------------------------------------|-------------------------------------------------------------------------------------------|
118-
| `MPT_API_BASE_URL` | `http://localhost:8000` | `https://portal.softwareone.com/mpt` | SoftwareONE Marketplace API URL |
119-
| `MPT_API_TOKEN` | - | eyJhbGciOiJSUzI1N... | SoftwareONE Marketplace API Token |
120-
| `MPT_TOOL_STORAGE_TYPE` | `local` | `airtable` | Storage type for MPT tools (local or airtable) |
121-
| `MPT_TOOL_STORAGE_AIRTABLE_API_KEY` | - | patXXXXXXXXXXXXXX | Airtable API key for MPT tool storage (required when storage type is airtable) |
122-
| `MPT_TOOL_STORAGE_AIRTABLE_BASE_ID` | - | appXXXXXXXXXXXXXX | Airtable base ID for MPT tool storage (required when storage type is airtable) |
123-
| `MPT_TOOL_STORAGE_AIRTABLE_TABLE_NAME` | - | MigrationTracking | Airtable table name for MPT tool storage (required when storage type is airtable) |
116+
| Environment Variable | Default | Example | Description |
117+
|----------------------------------------|-------------------------|--------------------------------------|---------------------------------------------------------------------------------------------|
118+
| `MPT_API_BASE_URL` | `http://localhost:8000` | `https://portal.softwareone.com/mpt` | SoftwareONE Marketplace API URL |
119+
| `MPT_API_TOKEN` | - | eyJhbGciOiJSUzI1N... | SoftwareONE Marketplace API Token |
120+
| `MPT_TOOL_STORAGE_TYPE` | `local` | `airtable` | Storage type for MPT tools (local or airtable) |
121+
| `MPT_TOOL_STORAGE_AIRTABLE_API_KEY` | - | patXXXXXXXXXXXXXX | Airtable API key for MPT tool storage (required when storage type is airtable) |
122+
| `MPT_TOOL_STORAGE_AIRTABLE_BASE_ID` | - | appXXXXXXXXXXXXXX | Airtable base ID for MPT tool storage (required when storage type is airtable) |
123+
| `MPT_TOOL_STORAGE_AIRTABLE_TABLE_NAME` | - | MigrationTracking | Airtable table name for MPT tool storage (required when storage type is airtable) |
124+
| `SERVICE_VERSION` | empty | `5.4.2` | Optional service version saved in migration state when a migration state is created |

‎docs/PROJECT_DESCRIPTION.md‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ The tool uses the following environment variables:
4646
- `MPT_API_TOKEN`: Your MPT API key (required when using `MPTAPIClientMixin`)
4747
- `MPT_TOOL_STORAGE_TYPE`: Storage backend for migration state (`local` or `airtable`, default: `local`). See [Storage Configuration](#storage)
4848
- `MPT_TOOL_STORAGE_AIRTABLE_API_KEY`: Your Airtable API key (required when using `AirtableAPIClientMixin` or when `MPT_TOOL_STORAGE_TYPE=airtable`)
49+
- `SERVICE_VERSION`: Optional service version persisted into each new migration state. If missing, the stored value is empty
4950

5051
## Configuration
5152

@@ -77,6 +78,7 @@ Your Airtable table must have the following columns:
7778
| started_at | dateTime | ❌ |
7879
| applied_at | dateTime | ❌ |
7980
| type | singleSelect (data, schema) | ✅ |
81+
| version | singleLineText | ❌ |
8082

8183

8284
**Airtable configuration steps:**
@@ -220,23 +222,25 @@ When running a single migration (`--data MIGRATION_ID` or `--schema MIGRATION_ID
220222
"order_id": 20260113180013,
221223
"started_at": "2026-01-13T18:05:20.000000",
222224
"applied_at": "2026-01-13T18:05:23.123456",
223-
"type": "data"
225+
"type": "data",
226+
"version": "5.3.2"
224227
},
225228
"schema_example": {
226229
"migration_id": "schema_example",
227230
"order_id": 20260214121033,
228231
"started_at": null,
229232
"applied_at": null,
230-
"type": "schema"
233+
"type": "schema",
234+
"version": ""
231235
}
232236
}
233237
```
234238
**Migration Table (Airtable):**
235239
236-
| order_id | migration_id | started_at | applied_at | type |
237-
|----------------|----------------|----------------------------|----------------------------|--------|
238-
| 20260113180013 | data_example | 2026-01-13T18:05:20.000000 | 2026-01-13T18:05:23.123456 | data |
239-
| 20260214121033 | schema_example | | | schema |
240+
| order_id | migration_id | started_at | applied_at | type | version |
241+
|----------------|----------------|----------------------------|----------------------------|--------|---------|
242+
| 20260113180013 | data_example | 2026-01-13T18:05:20.000000 | 2026-01-13T18:05:23.123456 | data | 5.3.2 |
243+
| 20260214121033 | schema_example | | | schema | |
240244
241245
242246
If a migration succeeds during execution:
@@ -279,6 +283,9 @@ To see all migrations and their status:
279283
```
280284
281285
The output shows execution order, status, and timestamps.
286+
It also shows the `version` column as the last column:
287+
- Applied/created state: value from `SERVICE_VERSION`, or empty if the variable is not set
288+
- Not applied (no state yet): `-`
282289
283290
The status column is derived from the persisted timestamps:
284291

‎mpt_tool/config.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@ def get_mpt_config(config_key: str) -> str | None:
1717
return config.get(config_key)
1818

1919

20+
def get_service_version() -> str | None:
21+
"""Get service version."""
22+
return os.getenv("SERVICE_VERSION")
23+
24+
2025
def get_storage_type() -> str:
2126
"""Get storage type."""
2227
return os.getenv("MPT_TOOL_STORAGE_TYPE", "local")

‎mpt_tool/managers/state/airtable.py‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
from pyairtable.orm import Model, fields
66
from requests import HTTPError
77

8-
from mpt_tool.config import get_airtable_config
8+
from mpt_tool.config import get_airtable_config, get_service_version
99
from mpt_tool.enums import MigrationTypeEnum
1010
from mpt_tool.managers import StateManager
1111
from mpt_tool.managers.errors import InitializationError, StateNotFoundError
@@ -20,6 +20,7 @@ class MigrationStateModel(Model):
2020
type = fields.RequiredSelectField("type")
2121
started_at = fields.DatetimeField("started_at")
2222
applied_at = fields.DatetimeField("applied_at")
23+
version = fields.TextField("version")
2324

2425
class Meta:
2526
@staticmethod
@@ -64,6 +65,7 @@ def get_by_id(cls, migration_id: str) -> Migration:
6465
type=MigrationTypeEnum(state.type),
6566
started_at=state.started_at,
6667
applied_at=state.applied_at,
68+
version=state.version,
6769
)
6870

6971
@override
@@ -105,6 +107,7 @@ def initialize(cls) -> None:
105107
"timeZone": "utc",
106108
},
107109
},
110+
{"name": "version", "type": "singleLineText"},
108111
]
109112
try:
110113
base.create_table(table_name, fields=table_fields)
@@ -122,6 +125,7 @@ def load(cls) -> dict[str, Migration]:
122125
type=MigrationTypeEnum(state.type),
123126
started_at=state.started_at,
124127
applied_at=state.applied_at,
128+
version=state.version,
125129
)
126130
migrations[migration.migration_id] = migration
127131

@@ -131,7 +135,10 @@ def load(cls) -> dict[str, Migration]:
131135
@classmethod
132136
def new(cls, migration_id: str, migration_type: MigrationTypeEnum, order_id: int) -> Migration:
133137
state = MigrationStateModel(
134-
migration_id=migration_id, order_id=order_id, type=migration_type.value
138+
migration_id=migration_id,
139+
order_id=order_id,
140+
type=migration_type.value,
141+
version=get_service_version(),
135142
)
136143
state.save()
137144
return Migration(
@@ -140,6 +147,7 @@ def new(cls, migration_id: str, migration_type: MigrationTypeEnum, order_id: int
140147
type=MigrationTypeEnum(state.type),
141148
started_at=state.started_at,
142149
applied_at=state.applied_at,
150+
version=state.version,
143151
)
144152

145153
@override
@@ -153,13 +161,15 @@ def save_state(cls, state: Migration) -> None:
153161
migration_state_model.type = state.type.value
154162
migration_state_model.started_at = state.started_at
155163
migration_state_model.applied_at = state.applied_at
164+
migration_state_model.version = state.version
156165
else:
157166
migration_state_model = MigrationStateModel(
158167
migration_id=state.migration_id,
159168
order_id=state.order_id,
160169
type=state.type.value,
161170
started_at=state.started_at,
162171
applied_at=state.applied_at,
172+
version=state.version,
163173
)
164174

165175
migration_state_model.save()

‎mpt_tool/managers/state/file.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
from pathlib import Path
33
from typing import override
44

5+
from mpt_tool.config import get_service_version
56
from mpt_tool.constants import MIGRATION_STATE_FILE
67
from mpt_tool.enums import MigrationTypeEnum
78
from mpt_tool.managers import StateManager
@@ -61,6 +62,7 @@ def new(cls, migration_id: str, migration_type: MigrationTypeEnum, order_id: int
6162
migration_id=migration_id,
6263
order_id=order_id,
6364
type=migration_type,
65+
version=get_service_version(),
6466
)
6567
cls.save_state(new_state)
6668
return new_state

‎mpt_tool/models.py‎

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,10 @@ class Migration:
1414
migration_id: str
1515
order_id: int
1616
type: MigrationTypeEnum
17-
started_at: dt.datetime | None = None
17+
1818
applied_at: dt.datetime | None = None
19+
started_at: dt.datetime | None = None
20+
version: str | None = None
1921

2022
@classmethod
2123
def from_dict(cls, migration_data: dict[str, Any]) -> Self:
@@ -24,12 +26,13 @@ def from_dict(cls, migration_data: dict[str, Any]) -> Self:
2426
migration_id=migration_data["migration_id"],
2527
order_id=migration_data["order_id"],
2628
type=MigrationTypeEnum(migration_data["type"]),
27-
started_at=dt.datetime.fromisoformat(migration_data["started_at"])
28-
if migration_data["started_at"]
29-
else None,
3029
applied_at=dt.datetime.fromisoformat(migration_data["applied_at"])
3130
if migration_data["applied_at"]
3231
else None,
32+
started_at=dt.datetime.fromisoformat(migration_data["started_at"])
33+
if migration_data["started_at"]
34+
else None,
35+
version=migration_data.get("version"),
3336
)
3437

3538
def to_dict(self) -> dict[str, Any]:
@@ -38,8 +41,9 @@ def to_dict(self) -> dict[str, Any]:
3841
"migration_id": self.migration_id,
3942
"order_id": self.order_id,
4043
"type": self.type.value,
41-
"started_at": self.started_at.isoformat() if self.started_at else None,
4244
"applied_at": self.applied_at.isoformat() if self.applied_at else None,
45+
"started_at": self.started_at.isoformat() if self.started_at else None,
46+
"version": self.version,
4347
}
4448

4549
def applied(self) -> None:
@@ -48,8 +52,9 @@ def applied(self) -> None:
4852

4953
def failed(self) -> None:
5054
"""Mark the migration as failed."""
51-
self.started_at = None
5255
self.applied_at = None
56+
self.started_at = None
57+
self.version = None
5358

5459
def manual(self) -> None:
5560
"""Mark the migration as manual."""
@@ -118,11 +123,13 @@ class MigrationListItem:
118123

119124
migration_id: str
120125
order_id: int
121-
migration_type: MigrationTypeEnum | None
122-
started_at: dt.datetime | None
123-
applied_at: dt.datetime | None
124126
status: MigrationStatusEnum
125127

128+
applied_at: dt.datetime | None = None
129+
migration_type: MigrationTypeEnum | None = None
130+
started_at: dt.datetime | None = None
131+
version: str | None = None
132+
126133
@classmethod
127134
def from_sources(
128135
cls,
@@ -133,8 +140,9 @@ def from_sources(
133140
return cls(
134141
migration_id=migration_file.migration_id,
135142
order_id=migration_file.order_id,
143+
status=MigrationStatusEnum.from_state(migration_state),
144+
applied_at=migration_state.applied_at if migration_state else None,
136145
migration_type=migration_state.type if migration_state else None,
137146
started_at=migration_state.started_at if migration_state else None,
138-
applied_at=migration_state.applied_at if migration_state else None,
139-
status=MigrationStatusEnum.from_state(migration_state),
147+
version=migration_state.version if migration_state else None,
140148
)

‎mpt_tool/renders.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
class MigrationRender:
88
"""Render migration state information as a formatted table."""
99

10-
_fields = ("order_id", "migration_id", "started_at", "applied_at", "type", "status")
10+
_fields = ("order_id", "migration_id", "started_at", "applied_at", "type", "status", "version")
1111

1212
def __init__(self, migration_items: list[MigrationListItem]):
1313
self.migration_items = migration_items
@@ -25,6 +25,7 @@ def __rich_console__(self, console: Console, options: ConsoleOptions) -> RenderR
2525
migration.applied_at.isoformat(timespec="seconds") if migration.applied_at else "-",
2626
migration.migration_type.value if migration.migration_type else "-",
2727
migration.status.title(),
28+
migration.version or "-",
2829
)
2930

3031
yield table

‎tests/cli/test_cli_airtable_storage.py‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ def applied_migration(data_migration_file, mock_airtable):
3333
"type": "data",
3434
"started_at": "2025-04-06T13:00:00+00:00",
3535
"applied_at": "2025-04-06T13:00:00+00:00",
36+
"version": None,
3637
}
3738
],
3839
)
@@ -54,6 +55,7 @@ def test_migrate_data_migration(mock_airtable, runner, log):
5455
"type": "data",
5556
"started_at": "2025-04-06T13:00:10.000Z",
5657
"applied_at": "2025-04-06T13:00:10.000Z",
58+
"version": "",
5759
}
5860
assert "Running data migrations..." in result.output
5961
assert "Running migration: fake_data_file_name" in log.text
@@ -73,7 +75,9 @@ def test_migrate_skip_migration_already_applied(runner, log):
7375

7476
@freeze_time("2025-04-06 13:00:00")
7577
@pytest.mark.usefixtures("data_migration_file_error")
76-
def test_migrate_data_run_script_fail(mock_airtable, runner, log):
78+
def test_migrate_data_run_script_fail(monkeypatch, mock_airtable, runner, log):
79+
monkeypatch.setenv("SERVICE_VERSION", "5.1.2")
80+
7781
result = runner.invoke(app, ["migrate", "--data"])
7882

7983
assert result.exit_code == 1, result.output
@@ -85,6 +89,7 @@ def test_migrate_data_run_script_fail(mock_airtable, runner, log):
8589
"type": "data",
8690
"started_at": None,
8791
"applied_at": None,
92+
"version": None,
8893
}
8994
assert "Running data migrations..." in result.output
9095
assert "Running migration: fake_error_file_name" in log.text
@@ -93,7 +98,9 @@ def test_migrate_data_run_script_fail(mock_airtable, runner, log):
9398

9499
@freeze_time("2025-04-06 10:11:24")
95100
@pytest.mark.usefixtures("data_migration_file")
96-
def test_migrate_manual(mock_airtable, runner):
101+
def test_migrate_manual(monkeypatch, mock_airtable, runner):
102+
monkeypatch.setenv("SERVICE_VERSION", "5.2.3")
103+
97104
result = runner.invoke(app, ["migrate", "--manual", "fake_data_file_name"])
98105

99106
assert result.exit_code == 0, result.output
@@ -107,6 +114,7 @@ def test_migrate_manual(mock_airtable, runner):
107114
"type": "data",
108115
"started_at": None,
109116
"applied_at": "2025-04-06T10:11:24.000Z",
117+
"version": "5.2.3",
110118
}
111119

112120

@@ -158,6 +166,7 @@ def test_migrate_init(mocker, runner):
158166
"timeZone": "utc",
159167
},
160168
},
169+
{"name": "version", "type": "singleLineText"},
161170
],
162171
)
163172

@@ -182,9 +191,9 @@ def test_migrate_list(runner, log):
182191
assert result.exit_code == 0, result.output
183192
assert "No state found for migration: fake_schema_file_name" in log.text
184193
formatted_output = "".join(result.output.split())
185-
assert "┃order_id┃migration_id┃started_at┃applied_at┃type┃status┃" in formatted_output
194+
assert "┃order_id┃migration_id┃started_at┃applied_at┃type┃status┃version┃" in formatted_output
186195
assert (
187-
"│20250406020202│fake_data_file_name│2025-04-06T13:00:00+00:00│2025-04-06T13:00:00+00:00│data│Applied│"
196+
"│20250406020202│fake_data_file_name│2025-04-06T13:00:00+00:00│2025-04-06T13:00:00+00:00│data│Applied│-│"
188197
in formatted_output
189198
)
190-
assert "│20260101010101│fake_schema_file_name│-│-│-│NotApplied│" in formatted_output
199+
assert "│20260101010101│fake_schema_file_name│-│-│-│NotApplied│-│" in formatted_output

0 commit comments

Comments
 (0)