YDB database support #1215

asmyasnikov · 2025-01-06T18:49:48Z

Added YDB support to golang-migrate

This PR implements support for YDB in golang-migrate. The main changes include:

Added the YDB driver
Implemented necessary functions for migrations, including Open, Close, Lock, Unlock, Run, SetVersion, Version, and Drop.
Added tests for the YDB driver in ydb_test.go, including common SQL instructions (such as CREATE/ALTER/DROP TABLE) and YDB-specific instructions (such as CREATE/DROP TOPIC - YDB message bus)
Updated the documentation in README.md to include instructions on how to use the YDB driver.

How to Test:

Set up an YDB environment.
Run the unit and integration tests to ensure all functionalities are working correctly.
Follow the instructions in the documentation to test real migrations using the YDB driver.

YDB Support

YDB Reimplement Lock Mechanics

coveralls · 2025-01-31T09:41:31Z

coverage: 56.532% (+0.2%) from 56.314%
when pulling 001b2f0 on ydb-platform:master
into 2788339 on golang-migrate:master.

asmyasnikov · 2025-03-23T22:45:59Z

@dhui Can you see this PR? We are waiting YDB support in golang-migrate very much

Copilot

Pull Request Overview

This PR adds support for YDB (Yandex Database) to golang-migrate, implementing a complete database driver with migration capabilities, locking mechanisms, and comprehensive test coverage.

Key Changes:

New YDB database driver with full migration lifecycle support (Open, Close, Lock, Unlock, Run, SetVersion, Version, Drop)
Extensive test suite including integration tests with Docker containers
Complete documentation with connection string format and configuration options

Reviewed Changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`internal/cli/build_ydb.go`	Build tag configuration to conditionally include YDB driver in CLI builds
`go.mod` / `go.sum`	Added YDB SDK dependencies (v3.100.0) and related transitive dependencies
`database/ydb/ydb.go`	Main driver implementation with connection handling, migrations, versioning, and locking
`database/ydb/ydb_test.go`	Comprehensive test suite with Docker-based integration tests for multiple YDB versions
`database/ydb/examples/migrations/*.sql`	Example migrations demonstrating YDB-specific features (tables, columns, indexes, topics)
`database/ydb/README.md`	Detailed documentation covering connection strings, authentication methods, TLS options, and troubleshooting
`README.md`	Added YDB to the list of supported database drivers
`Makefile`	Added `ydb` to the DATABASE variable for build/test targets

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-20T13:12:39Z

database/ydb/README.md

+If golang-migrate fails to acquire the lock and no migrations are currently running.
+This may indicate that one of the migrations did not complete successfully.


The sentence is incomplete. Consider revising to: "If golang-migrate fails to acquire the lock when no migrations are currently running, this may indicate that one of the migrations did not complete successfully." This makes it a complete, grammatically correct sentence.

Suggested change

If golang-migrate fails to acquire the lock and no migrations are currently running.

This may indicate that one of the migrations did not complete successfully.

If golang-migrate fails to acquire the lock when no migrations are currently running, this may indicate that one of the migrations did not complete successfully.

Copilot · 2025-11-20T13:12:40Z

database/ydb/ydb.go

+	if err := tx.Commit(); err != nil {
+		return &database.Error{OrigErr: err, Err: "transaction commit failed"}
+	}
+	return err


The function should return nil on success, not err (which is nil at this point but makes the intent unclear). Change to return nil for consistency and clarity.

Suggested change

return err

return nil

Copilot · 2025-11-20T13:12:40Z

database/ydb/ydb.go

+var (
+	ErrNilConfig             = fmt.Errorf("no config")
+	ErrNoDatabaseName        = fmt.Errorf("no database name")
+	ErrUnsupportedTLSVersion = fmt.Errorf("unsupported tls version: use 1.0, 1.1, 1,2 or 1.3")


Typo in error message: "1,2" should be "1.2" to be consistent with the other version numbers in the list.

Suggested change

ErrUnsupportedTLSVersion = fmt.Errorf("unsupported tls version: use 1.0, 1.1, 1,2 or 1.3")

ErrUnsupportedTLSVersion = fmt.Errorf("unsupported tls version: use 1.0, 1.1, 1.2 or 1.3")

Copilot · 2025-11-20T13:12:41Z

database/ydb/ydb.go

+	case err != nil:
+		return 0, false, &database.Error{OrigErr: err, Query: []byte(getVersionQuery)}
+	default:
+		return int(v), dirty, err


The function returns err in the default case, but at this point err is always nil (since we didn't hit the error cases above). This should be return int(v), dirty, nil for clarity and correctness.

Suggested change

return int(v), dirty, err

return int(v), dirty, nil

Copilot · 2025-11-20T13:12:41Z

database/ydb/ydb.go

+			getLockQuery := fmt.Sprintf("SELECT * FROM %s WHERE lock_id = '%s'", y.config.LockTable, aid)
+			rows, err := tx.Query(getLockQuery, aid)


The SQL query at line 315 already has aid interpolated (WHERE lock_id = '%s'), but line 316 attempts to pass aid as an additional parameter to tx.Query. This will cause an error or unexpected behavior. Either:

Remove the aid parameter from the tx.Query call (line 316), or

Use parameterized queries with placeholders instead of string interpolation (recommended for security).

The same issue exists at line 332 where the query has aid interpolated but the parameters should be passed separately.

Copilot · 2025-11-20T13:12:41Z

database/ydb/ydb.go

+		config.LockTable = defaultLockTable
+	}
+
+	conn, err := instance.Conn(context.TODO())


The code uses both context.TODO() (lines 76, 126, 225, 261, 349) and context.Background() (lines 210, 300, 369, 399) inconsistently. For consistency with other database drivers in this codebase (e.g., Postgres uses context.Background() throughout), consider using context.Background() consistently across all operations.

Suggested change

conn, err := instance.Conn(context.TODO())

conn, err := instance.Conn(context.Background())

Copilot · 2025-11-20T13:12:42Z

database/ydb/ydb.go

+	deleteVersionQuery := fmt.Sprintf(`
+		DELETE FROM %s 
+	`, y.config.MigrationsTable)
+
+	insertVersionQuery := fmt.Sprintf(`
+		INSERT INTO %s (version, dirty, created) VALUES (%d, %t, CurrentUtcTimestamp())
+	`, y.config.MigrationsTable, version, dirty)
+
+	tx, err := y.conn.BeginTx(context.TODO(), &sql.TxOptions{})
+	if err != nil {
+		return &database.Error{OrigErr: err, Err: "transaction start failed"}
+	}
+
+	if _, err := tx.Exec(deleteVersionQuery); err != nil {
+		if errRollback := tx.Rollback(); errRollback != nil {
+			err = multierror.Append(err, errRollback)
+		}
+		return &database.Error{OrigErr: err, Query: []byte(deleteVersionQuery)}
+	}
+
+	// Also re-write the schema version for nil dirty versions to prevent
+	// empty schema version for failed down migration on the first migration
+	// See: https://github.com/golang-migrate/migrate/issues/330
+	if version >= 0 || (version == database.NilVersion && dirty) {
+		if _, err := tx.Exec(insertVersionQuery, version, dirty); err != nil {


There are two issues with this implementation:

The SQL query uses string interpolation for values (%d, %t) at line 222, which is a security concern and should use parameterized queries instead.

Line 241 attempts to pass version and dirty as parameters to tx.Exec, but the query already has these values interpolated, causing a mismatch.

The correct approach is to build the query without interpolating the values (use placeholders like ? or positional parameters if YDB supports them), then pass version and dirty as parameters to the Exec call. This is also the pattern used in other database drivers (e.g., Postgres uses $1, $2).

art22m and others added 28 commits October 19, 2024 20:07

add skeleton

32bc791

implement methods

d8c5027

makefile, cli build

8f56312

go mod tidy

44abc30

some fixes

4dd2ac5

refactor

ee172d8

fix drop func

754a74d

update readme

a7e222d

add auth methods

532b4ac

refactor

86ae1fa

update readme

7bfd098

add tests

bd9bdae

fix test

0d18ddb

rm logs

2e3253b

refactor test files

5b21bf5

add tls options

1aa3455

change readme

fecf8d7

support min tls version

76093a1

fix reviews #1

bfe9fba

fix reviews #2

8baa73b

migrate to database/sql

81f152f

Apply suggestions from code review

063c1d0

fixes

c2ac3bc

reverted dependencies

1e4515c

fixed env

b5aa72f

Merge pull request #3 from art22m/support_ydb

3835934

YDB Support

go mod tidy

744dd5e

rename yql to sql

5444c1e

asmyasnikov force-pushed the master branch from d32255f to 11f2a26 Compare January 6, 2025 23:35

more sql-migrations

8d20ea1

asmyasnikov force-pushed the master branch from 11f2a26 to 8d20ea1 Compare January 8, 2025 22:44

art22m and others added 11 commits January 16, 2025 22:29

add params, table creation

bcb7e84

implement locks

7edc143

multiple images testing

3143ea4

fix tests

970b831

rm query mode selection

b19af9c

fix tests

677160f

rm force lock

fa12828

Merge pull request #4 from art22m/refactor-ydb-lock

e2d7653

YDB Reimplement Lock Mechanics

Merge branch 'golang-migrate:master' into master

3b0367b

bump ydb-go-sdk

fd6f3ae

exclude ydb 24.2 from specs

63a7f44

bump ydb-go-sdk

3a93627

asmyasnikov force-pushed the master branch from d9be4aa to 3a93627 Compare February 7, 2025 17:21

asmyasnikov added 2 commits February 20, 2025 20:35

Merge branch 'golang-migrate:master' into master

8981237

update ydb-go-sdk/v3

067ca8f

asmyasnikov added 3 commits April 23, 2025 18:16

Merge branch 'golang-migrate:master' into master

8d57316

Merge branch 'golang-migrate:master' into master

001b2f0

Merge branch 'master' into master

0a409b6

Copilot AI review requested due to automatic review settings November 20, 2025 13:05

Copilot started reviewing on behalf of asmyasnikov November 20, 2025 13:06 View session

Copilot finished reviewing on behalf of asmyasnikov November 20, 2025 13:11

Copilot AI reviewed Nov 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

YDB database support #1215

YDB database support #1215

Uh oh!

asmyasnikov commented Jan 6, 2025 •

edited

Loading

Uh oh!

coveralls commented Jan 31, 2025 •

edited

Loading

Uh oh!

asmyasnikov commented Mar 23, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Copilot AI Nov 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		If golang-migrate fails to acquire the lock and no migrations are currently running.
		This may indicate that one of the migrations did not complete successfully.

	ErrUnsupportedTLSVersion = fmt.Errorf("unsupported tls version: use 1.0, 1.1, 1,2 or 1.3")
	ErrUnsupportedTLSVersion = fmt.Errorf("unsupported tls version: use 1.0, 1.1, 1.2 or 1.3")

		getLockQuery := fmt.Sprintf("SELECT * FROM %s WHERE lock_id = '%s'", y.config.LockTable, aid)
		rows, err := tx.Query(getLockQuery, aid)

	conn, err := instance.Conn(context.TODO())
	conn, err := instance.Conn(context.Background())

YDB database support #1215

Are you sure you want to change the base?

YDB database support #1215

Uh oh!

Conversation

asmyasnikov commented Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asmyasnikov commented Mar 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

asmyasnikov commented Jan 6, 2025 •

edited

Loading

coveralls commented Jan 31, 2025 •

edited

Loading

asmyasnikov commented Mar 23, 2025 •

edited

Loading