How to kill a spark query operation?

[MilkWangStudio]

I shoud devleop a sql editor. it connect spark wtih kyuubi USER LEVEL, Use hive-jdbc to send query request and wait response.
How can i get the query job process and kill it when user click KILL button on page.

I tried use hive-jdbc's statement.close function, but kyuubi front server response "Invalid OperationHandle". It's like not implement this feature?

kyuubi version is 1.6.1. Want some helps, Thanks!

[pan3793]

Do you mean hive-jdbc? It's not part of Kyuubi.

[pan3793]

I didn't take a deep look at the Hive code, Kyuubi has a modified Kyuubi Hive JDBC Driver, my following answer is based on this one.

JDBC is absolutely a sync API, and is not thread-safe, any call across threads of stmt is not officially supported. But the underlying thrift API is async, directly calling the async thrift API is straightforward.

KyuubiStatement#executeAsync could be used as well.

[MilkWangStudio]

Ok, Thanks~ I will report it to Hive Community.

For this Discussion, I want some idea for two things.
1.When running a spark query, How to get this progress of timeline? I should display on the page.
2.How to kill the query job, when user click Kill btn on my page.

I search some keyworks on kyubbi doc, just find a RestfullAPI、JDBC maybe can do this.
Thanks！

[pan3793]

see #2529 for how to get query progress

[pan3793]

Specifically to your original question

kyuubi front server response "Invalid OperationHandle".

The description is too rough.

• Have you searched the log about the op uuid across the services log to see what happened on such op?
• any stack trace and context?

[MilkWangStudio]

I just debuging the exception stack. And find it's a mult-thread bug. Like you said:

JDBC is absolutely a sync API, and is not thread-safe

When the query thread is running , the other thread call the statement.close function, it will make kyuubi close the query job.
But the client's query thread always fetch result from kyuubi server. then make this exception.

It's hive-jdbc's question.

And thanks your reply for the query progress ^_^.