-
Notifications
You must be signed in to change notification settings - Fork 1.5k
YDB database support #1215
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
asmyasnikov
wants to merge
46
commits into
golang-migrate:master
Choose a base branch
from
ydb-platform:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
YDB database support #1215
Changes from all commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
32bc791
add skeleton
art22m d8c5027
implement methods
art22m 8f56312
makefile, cli build
art22m 44abc30
go mod tidy
art22m 4dd2ac5
some fixes
art22m ee172d8
refactor
art22m 754a74d
fix drop func
art22m a7e222d
update readme
art22m 532b4ac
add auth methods
art22m 86ae1fa
refactor
art22m 7bfd098
update readme
art22m bd9bdae
add tests
art22m 0d18ddb
fix test
art22m 2e3253b
rm logs
art22m 5b21bf5
refactor test files
art22m 1aa3455
add tls options
art22m fecf8d7
change readme
art22m 76093a1
support min tls version
art22m bfe9fba
fix reviews #1
art22m 8baa73b
fix reviews #2
art22m 81f152f
migrate to database/sql
art22m 063c1d0
Apply suggestions from code review
asmyasnikov c2ac3bc
fixes
asmyasnikov 1e4515c
reverted dependencies
asmyasnikov b5aa72f
fixed env
asmyasnikov 3835934
Merge pull request #3 from art22m/support_ydb
asmyasnikov 744dd5e
go mod tidy
asmyasnikov 5444c1e
rename yql to sql
asmyasnikov 8d20ea1
more sql-migrations
asmyasnikov bcb7e84
add params, table creation
art22m 7edc143
implement locks
art22m 3143ea4
multiple images testing
art22m 970b831
fix tests
art22m b19af9c
rm query mode selection
art22m 677160f
fix tests
art22m fa12828
rm force lock
art22m e2d7653
Merge pull request #4 from art22m/refactor-ydb-lock
asmyasnikov 3b0367b
Merge branch 'golang-migrate:master' into master
asmyasnikov fd6f3ae
bump ydb-go-sdk
asmyasnikov 63a7f44
exclude ydb 24.2 from specs
asmyasnikov 3a93627
bump ydb-go-sdk
asmyasnikov 8981237
Merge branch 'golang-migrate:master' into master
asmyasnikov 067ca8f
update ydb-go-sdk/v3
asmyasnikov 8d57316
Merge branch 'golang-migrate:master' into master
asmyasnikov 001b2f0
Merge branch 'golang-migrate:master' into master
asmyasnikov 0a409b6
Merge branch 'master' into master
asmyasnikov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # [YDB](https://ydb.tech/docs/) | ||
|
|
||
| `ydb://[user:password@]host:port/database?QUERY_PARAMS` | ||
|
|
||
| | URL Query | Description | | ||
| |:----------:|:---------------------------------------:| | ||
| | `user` | The user to sign in as. | | ||
| | `password` | The user's password. | | ||
| | `host` | The host to connect to. | | ||
| | `port` | The port to bind to. | | ||
| | `database` | The name of the database to connect to. | | ||
|
|
||
| | URL Query Params | Description | | ||
| |:----------------------------:|:--------------------------------------------------------------------------------------------:| | ||
| | `x-auth-token` | Authentication token. | | ||
| | `x-migrations-table` | Name of the migrations table (default `schema_migrations`). | | ||
| | `x-lock-table` | Name of the table which maintains the migration lock (default `schema_lock`). | | ||
| | `x-use-grpcs` | Enables gRPCS protocol for YDB connections (default grpc). | | ||
| | `x-tls-ca` | The location of the CA (certificate authority) file. | | ||
| | `x-tls-insecure-skip-verify` | Controls whether a client verifies the server's certificate chain and host name. | | ||
| | `x-tls-min-version` | Controls the minimum TLS version that is acceptable, use 1.0, 1.1, 1.2 or 1.3 (default 1.2). | | ||
|
|
||
| ### Secure connection | ||
|
|
||
| Query param `x-use-grpcs` enables secure TLS connection that requires certificates. | ||
| You can declare root certificate using ENV | ||
| variable: `export YDB_SSL_ROOT_CERTIFICATES_FILE=/path/to/ydb/certs/CA.pem` or | ||
| by using `x-tls-ca` query param: `?x-tls-ca=/path/to/ydb/certs/CA.pem`. | ||
|
|
||
| ### Authentication | ||
|
|
||
| By default, golang-migrate connects to YDB | ||
| using [anonymous credentials](https://ydb.tech/docs/en/recipes/ydb-sdk/auth-anonymous). \ | ||
| Through the url query, you can change the default behavior: | ||
|
|
||
| - To connect to YDB using [static credentials](https://ydb.tech/docs/en/recipes/ydb-sdk/auth-static) you need to specify | ||
| username and password: | ||
| `ydb://user:password@host:port/database` | ||
| - To connect to YDB using [token](https://ydb.tech/docs/en/recipes/ydb-sdk/auth-access-token) you need to specify token | ||
| as query parameter: | ||
| `ydb://host:port/database?x-auth-token=<YDB_TOKEN>` | ||
|
|
||
| ### Locks | ||
|
|
||
| If golang-migrate fails to acquire the lock and no migrations are currently running. | ||
| This may indicate that one of the migrations did not complete successfully. | ||
| In this case, you need to analyze the previous migrations, rollback if necessary, and manually remove the lock from the | ||
| `x-lock-table`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| DROP TABLE `test/users`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| CREATE TABLE `test/users` ( | ||
| id Uint64, | ||
| name String, | ||
| email String, | ||
| PRIMARY KEY (id) | ||
| ); |
3 changes: 3 additions & 0 deletions
3
database/ydb/examples/migrations/002_add_city_to_users.down.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| DROP TABLE `test/cities`; | ||
|
|
||
| ALTER TABLE `test/users` DROP COLUMN city; |
7 changes: 7 additions & 0 deletions
7
database/ydb/examples/migrations/002_add_city_to_users.up.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| CREATE TABLE `test/cities` ( | ||
| id Uint64, | ||
| name String, | ||
| PRIMARY KEY (id) | ||
| ); | ||
|
|
||
| ALTER TABLE `test/users` ADD COLUMN city Uint64; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| DROP TOPIC `test/topic`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| CREATE TOPIC `test/topic`; |
1 change: 1 addition & 0 deletions
1
database/ydb/examples/migrations/004_add_index_on_user_emails.down.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ALTER TABLE `test/users` DROP INDEX `users_email_index`; |
1 change: 1 addition & 0 deletions
1
database/ydb/examples/migrations/004_add_index_on_user_emails.up.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ALTER TABLE `test/users` ADD INDEX `users_email_index` GLOBAL UNIQUE ON (`email`); |
1 change: 1 addition & 0 deletions
1
database/ydb/examples/migrations/005_create_books_table.down.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| DROP TABLE `test/books`; |
6 changes: 6 additions & 0 deletions
6
database/ydb/examples/migrations/005_create_books_table.up.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| CREATE TABLE `test/books` ( | ||
| user_id Uint64, | ||
| name Text, | ||
| author Text, | ||
| PRIMARY KEY (user_id) | ||
| ); |
1 change: 1 addition & 0 deletions
1
database/ydb/examples/migrations/006_create_movies_table.down.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| DROP TABLE `test/movies`; |
6 changes: 6 additions & 0 deletions
6
database/ydb/examples/migrations/006_create_movies_table.up.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| CREATE TABLE `test/movies` ( | ||
| user_id Uint64, | ||
| name Text, | ||
| director Text, | ||
| PRIMARY KEY (user_id) | ||
| ); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sentence is incomplete. Consider revising to: "If golang-migrate fails to acquire the lock when no migrations are currently running, this may indicate that one of the migrations did not complete successfully." This makes it a complete, grammatically correct sentence.