Expose Logical and Physical plan details in the REST API

[milenkovicm]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Expand information REST API provides, making it the first point of contact to understand job behaviour. Logical and Physical plan are crucial for job understanding.

Describe the solution you'd like

REST API to provide more job related information, like logical and physical plan, it can also provide all the relevant plan execution statistic.

Describe alternatives you've considered

Additional context

[KR-bluejay]

Hi, I’d be happy to contribute to this.
Would it be okay if I take this issue on?

Also, could you share a bit more detail about what you’d like the REST API to expose —
for example which endpoints, what kind of information, and the expected response format?
I think these details will be important to align on before starting.

If there isn’t a clear spec yet, I can start working on it and clarify details as I go.

[milenkovicm]

There is rest api already in place, it provides some information but it does not expose details about logical, physical plans , and stages. have a look at current api implementation

[KR-bluejay]

Okay, I’ll look into it.

[milenkovicm]

i believe rest api is a feature and its disabled by default,