v0.11.0

Highlights

• Incompatible change: the command spirit is now spirit migrate. Additional subcommands include lint, diff and move. These features are now considered stable and ready to use by @morgo in #596 #610 #606 #609 #598 #604 #617 #625 #635 #649 #646
• Incompatible change: the --force-kill flag is now enabled by default. To disable it, use --skip-force-kill by @morgo in #605 #615 #643

Additional Changes

• spirit --version now reports the binary version by @fhe-ap and @Kiran01bm in #634 #616
• The composite chunker now supports Key above/Key below watermark optimizations by @Kiran01bm and @morgo in #590 #626
• Additional checks are now performed for set/enum datatypes by @morgo in #619
• Passwords with special characters were not handled correctly by @deanbaker in #593
• Dumping a checkpoint could panic when called too early in the migration process. Fix by @aparajon in #641
• The progress API now reports per-table progress in multi-table migrations by @aparajon in #640 #644
• The replication client now better validates binary log positions on resume. Fix by by @aparajon in #642
• Added mysql 9.6 support by @morgo in #603
• Logging refinements in the use of force-killing statements by @morgo in #637
• Improvements to documentation by @morgo and @aparajon in #595 #621
• Various improvements to the test suite and code quality by @morgo and @Kiran01bm in #639 #591 #599 #602 #622 #632 #594 #600 #614 #631 #633 #647

New Contributors

• @aparajon made their first contribution in #621
• @fhe-ap made their first contribution in #634
• @deanbaker made their first contribution in #593

Full Changelog: v0.10.2...v0.11.0

v0.10.2

This release contains bug fixes over the previous release. v0.10.2 is now the recommended version for use in production.

Highlights

• The --force-kill option did not take into account NULL processlist values, which can occur on replicas and systems where the event scheduler is running. Fixed by @fadyHemaya in #573

• The auto_increment property of a table was not correctly maintained when applying schema changes to empty tables. Fixed by @Kiran01bm in #583

• The BlockWait() function, which was used to ensure the binary log was advancing, could sometimes introduce performance issues be excessive use of FLUSH BINARY LOGS. Fixed by @eduardo-landin-mestre in #581

• Documentation has been added to many spirit packages, supporting its use as an API as well as a command line tool by @morgo in #566 #567 #570 #569 #576 #579 #584 #588 #589

Additional Changes

• Various bug fixes for the move command. We expect it to change from experimental to generally available shortly. Fixed by @morgo in #560 #562 #580
• Various test-stability and code quality fixes by @morgo in #564 #568 #571 #578 #575 #586

New Contributors

• @fadyHemaya made their first contribution in #573
• @eduardo-landin-mestre made their first contribution in #581

Full Changelog: v0.10.1...v0.10.2

v0.10.1

This release contains a number of bug fixes over the previous pre-release. v0.10.1 is now the recommended version for use in production.

Highlights

• Fixes an issue where binary log rotation events are not handled correctly in the replication applier. This may lead to issues where migrations that resume from checkpoint either skip events or fail by @morgo in #553 #545
• When the replication syncer connection fails, it did not always reconnect in a way that is recoverable by @morgo in #558
• Add support for loading credentials from file provided in command line by @brirams in #422
• The stability of MoveTables and buffered copying is improved. It remains experimental for now by @morgo in #549 #543 #544 #550 #557 #551 #547 #540 #533 #538 #539

Additional Changes

• Improved test infrastructure and GitHub Actions workflows by @Kiran01bm in #530
• Change LinterSeverity to int, add default Severity to all Violations by @kolbe in #532
• Fix handling of negative string values in NewDatum for unsigned types by @tyd in #541
• Bump required Go version to 1.25 and golang-ci linter to v2.6.2 by @morgo in #555, #531

Upgrade Notes

• Checkpoint compatibility: Complete any in-progress schema changes before upgrading, as the checkpoint format is not guaranteed to be stable between versions
• Go version: Requires Go 1.25 or later for building from source

Full Changelog: v0.10.0...v0.10.1

v0.10.0-prerelease

This release contains a large number of changes over the previous, and introduces the concept of experimental features to Spirit.

With experimental features:

• We will try to introduce them in ways where the code is well isolated, and you will always have to opt-in to using them.
• We will always document why they are still considered experimental.

Highlights

• Support for --force-kill to kill connections preventing lock acquisition by @kolbe, @morgo in #428 #458 #438 #506 #503
• Support for atomic multi-table changes by @morgo in #469 #431 #434 #437 #505 #482 #516
• The --force-inplace option has been removed, as we instead rely on parsing and explicitly allowing known safe inplace operations by @Kiran01bm, @kolbe in #470 #429
• The --checksum option has been removed and is now always enabled by @morgo in #489
• When flushing replication changes, Spirit now waits for the copier to advance past that point. This helps reduce potential contention scenarios at the cost of checkpoints sometimes being delayed. We call this optimization keyBelowLowWatermark by @morgo in #490

Additional Changes

• Improved TLS support by @Kiran01bm #443 #510
• Improved handling for cases where a table is externally modified, or can't write checkpoint handling by @morgo in #483
• The sentinel table option is now consistently named _spirit_sentinel #447 by @morgo
• Refactor alter visibility to the checks sub-system by @Kiran01bm in #484
• Fix RDS hostname detection to prevent potential subdomain spoofing by @Kiran01bm in #509
• move checksum retries into checksum package by @morgo in #507
• make tests parallel by @morgo in #514
• panic: could not convert datum to uint64 by @tyd in #518
• Have metadatalocks support multiple locks in one connection by @morgo in #525
• feat: Add maxRecreateAttempts logic by @tyd in #527
• Add additional info to ErrNotSupportedStatement error by @kolbe in #528
• Experimental support for buffered changes based on the DBLog algorithm by @morgo in #460 #453 #452 #459 #473
• Experimental support for native linting by @kolbe #486 #492 #501 #511 #529 #500
• Experimental support for a move command by @morgo #467 #496 #515 #523

New Contributors

• @tyd made their first contribution in #518

Full Changelog: v0.9.0...v0.10.0

v0.9.2

This addresses a critical issue where after several retries (handling errors such as deadlocks or lock waits), the replication applier will drop changes that could not be applied.

In many cases, this corruption would have been caught by the checksum, but regardless, all users are encouraged to upgrade immediately.

See issue #472

v0.9.1

This addresses a critical issue where updating rows such that the value of a PRIMARY KEY column changed were not tracked correctly, potentially leading to changes being lost in the new table.

In many cases, this corruption would have been caught by the checksum, but regardless, all users are encouraged to upgrade immediately.

See issue #417

v0.9.0

This release includes bug fixes and some refactoring so that the replication client can support multiple subscriptions. This is preparation work for a planned feature (atomic changes across multiple tables).

Multiple replication subscriptions took a few bug fixes to get right, but now that it is looking stable I want to issue a release ahead of new features.

What's Changed

• Refactor repl to support multiple subscriptions by @morgo in #389
• remove unnecessary test by @morgo in #398
• Change NewTableLock to take []tables as input by @morgo in #400
• fix: Add ServerName to TLSConfig by @kolbe in #402
• Support add partition. Fix syntax errors for partition statements. by @JashLal in #403
• Add poison pill protection by @morgo in #407
• remove query from log by @morgo in #408
• Fix the type conversion issue when resuming from checkpoint for PK ty… by @prudhvi in #409
• Bump tidb parser version by @morgo in #410
• Add logging and increase timeout of BlockWait calls by @morgo in #415
• use defer for MDL by @morgo in #413
• Fix repl client getting stuck when encountering errors by @morgo in #416
• [checks] Reintroduce hasTriggers check as a Preflight check by @brirams in #420
• update interface types to any - no functional change by @Kiran01bm in #423
• bump tidb parser version by @kolbe in #424

New Contributors

• @brirams made their first contribution in #420

Full Changelog: v0.8.0...v0.9.0

v0.8.0

New Features

• The minimum golang version required is now 1.24 by @kolbe
• We've added support for --statement instead of specifying --table and --alter. For now, both are supported, but we may prefer --statement moving forward by @morgo
• The inplace alter operations for DROP/TRUNCATE partition are explicitly marked as safe by @JashLal

Enhancements and Fixes

• minor - update/fix lint command in dev guide by @Kiran01bm in #368
• dev/test improvement - log test output in json and summarise using tparse by @Kiran01bm in #369
• certbundle reorg - change to load rds cert bundle from a file by @Kiran01bm in #370
• enhance the mdl refresh runner to handle connection closures and temporary network issues resiliently by @Kiran01bm in #371
• Fix converting latin1 to utf8 charset checksum failures by @morgo in #373
• embed certificate instead of file load by @morgo in #376
• compose lock name on spirit side - clients acquiring locks can get to know their exact lock name by @Kiran01bm in #374
• Fix non printable characters in checkpoint by @morgo in #382
• trim trailing semicolon from alter statement by @kolbe in #385
• Add test leak detection by @morgo in #390
• exclude generated columns from checksum by @morgo in #396

New Contributors

• @Kiran01bm made their first contribution in #368
• @JashLal made their first contribution in #387

Full Changelog: v0.7.0-prerelease...v0.8.0

v0.7.0-prerelease

What's Changed

• support 4hr delayed replicas by @morgo in #354
• binary type conversion by @morgo in #356
• use a hash for metadata lock name by @kolbe in #357
• bump linter to 1.62.2 by @kolbe in #359
• Bump go by @kolbe in #358
• Upgrade go-mysql version to v1.9.1 by @prudhvi in #363
• sort USAGE alphabetically, add lock-wait-timeout docs by @morgo in #366

Full Changelog: v0.6.0-prerelease...v0.7.0-prerelease

v0.6.0-prerelease

This release contains a number of improvements over previous releases.

What's Changed

• update throttler latency query by @morgo in #294
• Add asserty feature by @morgo in #293
• reduce deadlocks in recopying chunks by @morgo in #295
• Add --strict option by @samongyr-sq in #304
• Add support for running tests with docker-compose by @sahilm in #300
• [#306] Upgrade to go 1.22 by @sahilm in #309
• [#310] upgrade to latest golangci-lint and enable new linters by @sahilm in #311
• prevent concurrent table modifications by @jayjanssen in #308
• Jayj/old table timestamped by @jayjanssen in #307
• cleanup server lock code by @morgo in #302
• Reject read only connections to primary nodes by @sahilm in #313
• rename test tables to avoid conflicts by @jayjanssen in #318
• add CopierEstimateHistory by @jayjanssen in #316
• fix unlock in progress API by @morgo in #322
• Add virtual column support by @morgo in #324
• dbconn: improve MDL lock output by @morgo in #326
• Use dbconn.Exec in checkpoint by @morgo in #328
• Remove copier word from log to have the throttler be used outside of spirit by @prudhvi in #334
• Fix changes not synced error by @morgo in #332
• repl: fix div by zero error by @morgo in #336
• workaround timeout issues by @morgo in #338
• repl: revert to inject-noise based master wait by @morgo in #339
• require RELOAD privilege by @morgo in #340
• Use quoted table name as lock name by @prudhvi in #342
• Add resume from checksum by @morgo in #344
• Add support for interpolate params by @morgo in #346
• repl: Improve Flush() to handle BlockWait() timeouts by @morgo in #349
• Use tablename for metadatalock name as tablename respsects both the m… by @prudhvi in #350

New Contributors

• @samongyr-sq made their first contribution in #304
• @sahilm made their first contribution in #300
• @jayjanssen made their first contribution in #308

Full Changelog: v0.1.0-prerelease...v0.6.0-prerelease