[KED-1148] Kedro-Server

[yetudada]

Description

Kedro-Server is a RESTful API for Kedro pipelines allowing you to:

• Trigger a pipeline run using a platform or code agnostic post request
• Get requests on execution status or progress
• Get requests on pipelines available to run
• Trigger tests or linting from other applications or browsers

Context

Our users have struggled with two things:

• Determining the status of Kedro runs beyond looking at logs,
• And, triggering pipeline runs from a front-end when a different language is required.

These changes will have implications for kedro-viz, our data pipeline visualisation plugin.

[BartekSzpak]

Hi Guys,

This looks very exciting and main page already mentions "Coming soon". Any thoughts on when we can have a chance to give it a try?

[lorenabalan]

Hi @BartekSzpak , apologies for the delayed response!! We're in the process of exposing this to our internal users to get targeted feedback to iterate on, so you could say we're in "beta" phase. Timelines are hard to judge at this point, but my guess is it'll be another couple of months at least? @yetudada can keep me honest here.

[yetudada]

The alpha release of Kedro-Server goes live this week @BartekSzpak! We'll get focussed user testing to get this one out sooner and will update you here when we're ready for the beta open-source release.

[LukasHaas]

@yetudada Great work! This is a feature I have really been looking forward to use; it will make building applications for clients so much easier!

[yetudada]

We're going to be rebuilding Kedro-Server as part of our roadmap. I'll close this ticket in the meanwhile and update the changes. In the meanwhile we recommend that once you have the model, you are free to choose any model serving mechanism to expose it via an API. Popular options include MLflow Serving and rolling your thin API wrapper with fastapi.

[cshaley]

@yetudada Is there any followup on this? It sounds like maybe you shipped a proof of concept privately, but then never got it into a position to be open sourced?

A couple of similar projects are kedro_fast_api and kedro_serving.

[astrojuanlu]

Hello @cshaley, everyone,

A long time has passed since there was activity in this issue and I'm trying to figure out what's the status of our internal Kedro-Server. Regardless, it's unlikely that this becomes a priority in the short term, and if you need a solution now, I'd recommend you to look into @Galileo-Galilei's kedro-serving (which seems more up to date than kedro_fast_api).

I'm mindful that this is one of the most upvoted issues of this repository, so I'll keep an eye on it and if you're missing anything from kedro-serving or the alternatives, feel free to ping me.