Support Kyuubi way of defining endpoints

[jankogasic]

https://github.com/DeepDiagnostix-AI/mcp-apache-spark-history-server/blob/8a05982f3e6eaa364687e72abb4f15b757136102/src/spark_history_mcp/api/spark_client.py#L275

I have both Spark Thrift and Kyuubi applications in my Spark History Server. The way that Kyuubi creates endpoints is that it embeds attempt ID into the URL. So instead of
http://my.history.server.com:18080/history/application_1751489495712_45692/jobs/
it goes as
http://my.history.server.com:18080/history/application_1751489495712_45692/1/jobs/

and because of this I am getting error that first URL is non existent and MCP server can't find it. It would be nice to find a way of supporting Kyuubi applications.

Thanks!

[nabuskey]

Interesting. We could add support for templating URL paths to allow for customization for each server.

[zemin-piao]

+1 to this issue

I feel the solution should be:

• Call the applications endpoint, check items at attempts whether there is attemptId specified.
• If not, call {app_id}/{endpoint}
• If so, call {app_id}/{attemptId}/{endpoint}

[zemin-piao]

https://spark.apache.org/docs/latest/monitoring.html#rest-api

In the API, an application is referenced by its application ID, [app-id]. When running on YARN, each application may have multiple attempts, but there are attempt IDs only for applications in cluster mode, not applications in client mode. Applications in YARN cluster mode can be identified by their [attempt-id]. In the API listed below, when running in YARN cluster mode, [app-id] will actually be [base-app-id]/[attempt-id], where [base-app-id] is the YARN application ID.

[zemin-piao]

After reviewing the code, this commit actually already fixed this issue (in case of 404 not found, it adds attempt_id as 1 by default)

Verified latest version (v0.1.3) has the fix. @jankogasic could you verify further?

[wForget]

Maybe we only need to add a prompt like this: "When application attempt is not empty, use {application_id}/{attempt_id} as the app_id parameter for spark mcp tools.". Alternatively, modifying the description of app_id in the tool arguments might also work.

I tried this prompt, and it seems to work, although we got another related error.

[zemin-piao]

Maybe we only need to add a prompt like this: "When application attempt is not empty, use {application_id}/{attempt_id} as the app_id parameter for spark mcp tools.". Alternatively, modifying the description of app_id in the tool arguments might also work.

I tried this prompt, and it seems to work, although we got another related error.

@wForget which version MCP spark history server package are you using?

[wForget]

@wForget which version MCP spark history server package are you using?

I'm using the main branch, and #81 works well most of the time (attemptId isn't always 1).