Flink CDC 2.4.0 无法用多个并行任务进行全量导入 #2275
Replies: 1 comment
-
已经发现问题原因了,因为把mode 调成了earliest,所以没有经过snapshot过程,自始至终都是单一在跑,谢谢。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello:
运行了官方文档的代码
`
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import com.ververica.cdc.debezium.JsonDebeziumDeserializationSchema;
import com.ververica.cdc.connectors.mysql.source.MySqlSource;
public class MySqlSourceExample {
public static void main(String[] args) throws Exception {
MySqlSource mySqlSource = MySqlSource.builder()
.hostname("yourHostname")
.port(yourPort)
.databaseList("yourDatabaseName")
.tableList("yourDatabaseName.yourTableName")
.username("yourUsername")
.password("yourPassword")
.deserializer(new JsonDebeziumDeserializationSchema())
.build();
}
}
`
但是在log中发现,只有一条
Connected to <ip>:3306 at /4 (sid:6348, cid:196)
然后我把set parallelism 换成不同数字,1,2,4等。发现,吞吐量是一样的。期待中的,高并发飞速snapshot并没有发生。我怀疑只有一个任务在做snapshot,而不是多个。
请问是哪里调整的不对,没有开启parallel sync snapshot?
谢谢。
Beta Was this translation helpful? Give feedback.
All reactions