Kyuubi use SPARK_SQL as an engine, and it is very slow to get metadata information using a jdbc connection

[as10128]

Kyuubi use SPARK_SQL as an engine, and it is very slow to get metadata information (Getschemas Gettables Getcolumns) using a jdbc connection, and Kyuubi logs show 1 minute. But using beeline is fast, how should I check this issue ?

[pan3793]

The performance of Getschemas Gettables Getcolumns has room to improve, e.g. parallel to get those information.

And there is a workaround kyuubi.operation.getTables.ignoreTableProperties=true to skip fetching some information to speed up the method.

[as10128]

@pan3793 Thanks, but I still have a question, Getschemas Gettables Getcolumns takes about the same time, is it every time you get metadata to re-fetch all the information and then filter?

[pan3793]

Almost, since there is no pushdown support.

[pan3793]

FYI: kyuubi.operation.getTables.ignoreTableProperties=true was enhanced to support session catalog cases in #6018