-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
seatunnel version:2.3.11
It's can be done by the following config file if there's no requirement for specify table columns.
`
source {
MySQL-CDC {
base-url="jdbc:mysql://10.0.0.12:3306/bocai"
username = "root"
password = "xxx"
database-names=[
"bocai"
]
table-names=[
"bocai.account",
]
startup.mode = "initial"
snapshot.split.size = 8096
snapshot.fetch.size = 1024
}
}
sink {
Jdbc {
url = "jdbc:mysql://10.0.0.181:3306/bocai?useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true"
driver = "com.mysql.cj.jdbc.Driver"
user = "root"
password = "xxx"
database = "bocai"
table = "${table_name}"
schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST"
generate_sink_sql = true
data_save_mode = "APPEND_DATA"
}
}
**BUT If I want to sync several tables(each table needs to sync specific columns), how to specify several tables' columns.** right now I just add source for one table, and transform for the table.
source {
MySQL-CDC {
base-url="jdbc:mysql://10.0.0.12:3306/bocai"
username = "root"
password = "xxx"
database-names=[
"bocai"
]
table-names=[
"bocai.access_log",
]
startup.mode = "initial"
snapshot.split.size = 8096
snapshot.fetch.size = 1024
plugin_output = "my_access_log";
}
}
transform {
Sql {
query = "select id, createTime, requestMethod,requestTime from access_log"
result_table_name = "access_log"
plugin_input = "my_access_log"
}
}
`
If there're 10 tables to sync with specifying table columns. Shoud I need to add ten more sources and transforms for each table?
And for MySQL-CDC, each source acts as one mysql slaver or use the same mysql slave?
Thanks in advance!!!