mpt-tool CLI

mpt-tool is a command-line utility to scaffold, run, and audit migrations for MPT extensions.

Quick Start

Install the tool:
```
  pip install mpt-tool
```
Initialize the migration tool:
```
  mpt-service-cli migrate --init
```

Create your first migration:

  mpt-service-cli migrate --new-data sync_users

Edit the generated file in the migrations/ folder
Run all pending data migrations
```
  mpt-service-cli migrate --data
```

Installation

Install with pip or your favorite PyPI package manager:

  pip install mpt-tool

  uv add mpt-tool

Prerequisites

Python 3.12+ in your environment
A migrations/ folder in your project (created automatically with --init or when you create your first migration)
Environment variables. See Environment Variables for details.

Environment Variables

The tool uses the following environment variables:

MPT_API_BASE_URL: The MPT API base url (required when using MPTAPIClientMixin)
MPT_API_TOKEN: Your MPT API key (required when using MPTAPIClientMixin)
MPT_TOOL_STORAGE_TYPE: Storage backend for migration state (local or airtable, default: local). See Storage Configuration
MPT_TOOL_STORAGE_AIRTABLE_API_KEY: Your Airtable API key (required when using AirtableAPIClientMixin or when MPT_TOOL_STORAGE_TYPE=airtable)
SERVICE_VERSION: Optional service version persisted into each new migration state. If missing, the stored value is empty

Configuration

Storage

The tool supports two storage backends: local and Airtable. By default, it uses the local storage.

Local storage is the simplest option and is suitable for development and testing. However, it is not suitable for production deployments. The state is stored in a .migrations-state.json file in your project root.

Airtable storage is recommended for production deployments. It allows you to track migration progress across multiple deployments.

Local Storage

No additional configuration is required.

Airtable Storage

Airtable configuration is done via environment variables:

MPT_TOOL_STORAGE_AIRTABLE_API_KEY: Your Airtable API key
MPT_TOOL_STORAGE_AIRTABLE_BASE_ID: Your Airtable base ID
MPT_TOOL_STORAGE_AIRTABLE_TABLE_NAME: The name of the table to store migration state

Your Airtable table must have the following columns:

Column Name	Field Type	Required
order_id	number	✅
migration_id	singleLineText	✅
started_at	dateTime	❌
applied_at	dateTime	❌
type	singleSelect (data, schema)	✅
version	singleLineText	❌

Airtable configuration steps:

Create a new table in your Airtable base (or use an existing one)
Add the columns listed above with the specified field types
Set the environment variables with your base ID and table name

Initialization

Before using the migration tool for the first time, you should initialize it. This creates the necessary resources:

  mpt-service-cli migrate --init

This command creates:

The migrations/ folder in your project root (if it doesn't exist)
The state storage:
- For local storage: creates .migrations-state.json file
- For Airtable storage: creates the table in Airtable with the required schema

When to use --init:

First time setting up the tool in a project
When switching from local to Airtable storage (or vice versa)
When you need to recreate the state storage

Note: If the state storage already exists, the command will fail with an error message. This prevents accidental data loss. If you need to reinitialize, manually delete the existing state file or table first.

Usage

Creating a New Migration

Decide the migration type (data or schema).
- Data: run after a release is deployed. Can take hours or days. Executed while MPT is running (e.g., updating product parameters, synchronizing Assets with external data)
- Schema: run before a release is deployed. Must be fast (not more than 15 min). Executed without ensuring the MPT is running (e.g., adding columns in Airtable)
Run the appropriate command:

  # Data migration
  mpt-service-cli migrate --new-data "migration_name"

  # Schema migration
  mpt-service-cli migrate --new-schema "migration_name"

A new file is created in migrations/ with a timestamped prefix (e.g., 20260113180013_migration_name.py) and a prefilled Migrate class.

order_id: timestamp prefix (e.g., 20260113180013) migration_id: user-provided name (e.g., migration_name) file: generated file name (e.g., 20260113180013_migration_name.py)

Generated file structure:

from mpt_tool.migration import DataBaseMigration  # or SchemaBaseMigration


class Migration(DataBaseMigration):
    def run(self):
        # implement your logic here
        pass

Using Mixins

You can add mixins to your migration commands to access external services:

from mpt_tool.migration import DataBaseMigration
from mpt_tool.migration.mixins import MPTAPIClientMixin, AirtableAPIClientMixin


class Migration(DataBaseMigration, MPTAPIClientMixin, AirtableAPIClientMixin):
    def run(self):
        # Access MPT API
        agreement = self.mpt_client.commerce.agreements.get("AGR-1234-5678-9012")
        self.log.info(f"Agreement id: {agreement.id}")

        # Access Airtable
        table = self.airtable_client.table("app_id", "table_name")
        records = table.all()

        self.log.info(f"Processed {len(records)} records")

Checking Migrations

Before running migrations, you can validate your migration folder for issues:

  mpt-service-cli migrate --check

This command:

Verifies the migration folder structure
Detects duplicate migration_id values (which could happen if migrations were created with the same name)
Exits with code 0 if all checks pass
Exits with code 1 and shows a detailed error message if duplicates are found

Example output when duplicates are found:

Checking migrations...
Error running check command: Duplicate migration_id found in migrations: 20260113180013_duplicate_name.py, 20260114190014_duplicate_name.py

Best Practice: Run --check as part of your CI/CD pipeline to catch migration issues before deployment.

Running Migrations

Run all pending data migrations:
```
mpt-service-cli migrate --data
```
Run all pending schema migrations:
```
mpt-service-cli migrate --schema
```

Run one specific data migration:

mpt-service-cli migrate --data MIGRATION_ID

Run one specific schema migration:

mpt-service-cli migrate --schema MIGRATION_ID

Migrations are executed in order based on their order_id (timestamp). The tool automatically:

Validates the migration folder structure
Skips migrations that have already been applied (applied_at is not null)
Tracks execution status in the state storage (.migrations-state.json or Airtable table)
Logs migration progress
Handles errors gracefully and updates state accordingly

When running a single migration (--data MIGRATION_ID or --schema MIGRATION_ID), the tool:

Fails if MIGRATION_ID does not exist
Fails if the migration type does not match the selected flag
Fails if the migration was already applied

Migration State File (.migrations-state.json):

{
  "data_example": {
    "migration_id": "data_example",
    "order_id": 20260113180013,
    "started_at": "2026-01-13T18:05:20.000000",
    "applied_at": "2026-01-13T18:05:23.123456",
    "type": "data",
    "version": "5.3.2"
  },
  "schema_example": {
    "migration_id": "schema_example",
    "order_id": 20260214121033,
    "started_at": null,
    "applied_at": null,
    "type": "schema",
    "version": ""
  }
}

Migration Table (Airtable):

order_id	migration_id	started_at	applied_at	type	version
20260113180013	data_example	2026-01-13T18:05:20.000000	2026-01-13T18:05:23.123456	data	5.3.2
20260214121033	schema_example			schema

If a migration succeeds during execution:

The started_at timestamp is recorded
The applied_at timestamp is recorded

If a migration fails during execution:

The started_at timestamp is recorded
The applied_at field remains null
The error is logged
Later runs will retry the failed migration as applied_at is null, unless --manual is used to mark it as applied

Manual Mode

To mark a migration as applied without running it:

  mpt-service-cli migrate --manual MIGRATION_ID

Where MIGRATION_ID is the filename without order_id and .py (e.g., test1).

Example:

File: 20260113180013_sync_users.py
Migration ID: sync_users

If the migration doesn't exist in the migrations folder:

An error is logged and the command exits

If the migration exists:

The migration state is created if it doesn't exist yet or updated:
- The started_at field is set as null
- The applied_at timestamp is recorded

Listing Migrations

To see all migrations and their status:

  mpt-service-cli migrate --list

The output shows execution order, status, and timestamps. It also shows the version column as the last column:

Applied/created state: value from SERVICE_VERSION, or empty if the variable is not set
Not applied (no state yet): -

The status column is derived from the persisted timestamps:

Status	Condition
running	`started_at` is set and `applied_at` is empty
failed	`started_at` and `applied_at` are empty for an existing state
manual	`started_at` is empty and `applied_at` is set
applied	Both `started_at` and `applied_at` are set
not applied	No state entry exists for the migration file

Getting Help

Run mpt-service-cli --help to see all available commands and params:

  mpt-service-cli --help
  mpt-service-cli migrate --help

Best Practices

Migration Validation

Run mpt-service-cli migrate --check before committing migration files
Include --check in your CI/CD pipeline to catch issues early
Verify there are no duplicate migration_id values before deployment

Migration Naming

Use descriptive, snake_case names (e.g., add_user_table, fix_null_emails, sync_agreements_from_api)
Keep names concise but meaningful
Avoid generic names like migration1, fix_bug, or update

Version Control

Never modify a migration that has been applied in production
Create a new migration to fix issues from a previous one

Troubleshooting

Common Issues

Initialization fails - state already exists:

Error: "Cannot initialize - State file already exists" (local storage) or similar for Airtable
Cause: The state storage has already been initialized
Solution: This is intentional to prevent data loss. If you need to reinitialize:
- For local storage: delete .migrations-state.json manually
- For Airtable: delete the table manually or use a different table name
Only reinitialize if you're certain you want to start fresh

Migrations not detected:

Ensure files are in the migrations/ folder
Verify filename follows the pattern: <timestamp>_<migration_id>.py (e.g., 20260121120000_migration_name.py)

Migration fails to run:

Review the error message in the terminal output
Check your Migration.run() implementation for syntax errors
Fix the issue and re-run the migration or use --manual to mark it as applied

NOTE: There is currently no automatic rollback mechanism. If a migration partially modifies data before failing, you must manually revert those changes or create a new migration to fix the state.

Mixin errors (ValueError):

Verify all required environment variables are set
Check variable names match exactly (case-sensitive)

Duplicate migration IDs:

The tool prevents duplicate migration IDs automatically
If you see this error, check for files with the same name in the migrations/ folder
Delete or rename the duplicate file

Migration already applied:

If you need to re-run a migration, either:
- Remove its entry from the state storage (use with caution)
- Create a new migration with the updated logic
Never modify an already-applied migration in production

Pre-commit

Checking migrations with pre-commit:

Add this to your .pre-commit-config.yaml

-   repo: https://github.com/softwareone-platform/mpt-tool
    rev: ''  # Use the sha / tag you want to point at
    hooks:
      - id: check-migrations

Development

For development purposes, please, check the Readme in the GitHub repository.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mpt-tool CLI

Quick Start

Installation

Prerequisites

Environment Variables

Configuration

Storage

Local Storage

Airtable Storage

Initialization

Usage

Creating a New Migration

Using Mixins

Checking Migrations

Running Migrations

Manual Mode

Listing Migrations

Getting Help

Best Practices

Migration Validation

Migration Naming

Version Control

Troubleshooting

Common Issues

Pre-commit

Development

FilesExpand file tree

PROJECT_DESCRIPTION.md

Latest commit

History

PROJECT_DESCRIPTION.md

File metadata and controls

mpt-tool CLI

Quick Start

Installation

Prerequisites

Environment Variables

Configuration

Storage

Local Storage

Airtable Storage

Initialization

Usage

Creating a New Migration

Using Mixins

Checking Migrations

Running Migrations

Manual Mode

Listing Migrations

Getting Help

Best Practices

Migration Validation

Migration Naming

Version Control

Troubleshooting

Common Issues

Pre-commit

Development