Add support for rye as an environment manager

[marrov]

Hi all!

This discussion is to argue in favour of adding rye as a supported package manager in kedro. Considering that ruff is currently supported, rye is astral's take an a python package manager on steroids (my own words). It uses ruff and optionally uv under the hood (faster drop in replacement for pip) and acts like a better poetry (a nice comparison - though slightly outdated - of the different tools in the space and what they can do). We have been quite happy with it at my organization with it despite it being a relatively recent project. Actually, we have such a high conviction with rye that currently we have a custom and clunky approach to initialize a rye project, then a kedro project and merge them before starting to develop our kedro pipelines.

It is true that given the toml based approach it is not impossible to use the two projects in tandem in a customized way but I believe it would be great if there could be direct support for rye within this project. I have used it quite extensively and would be happy to contribute with adding this tool with some guidance. I know kedro's templating is based on ccds so I have also opened a similar discussion there in case that may be a blocker.

[noklam]

Thanks for starting the discussion. We recognised there are some friction with certain workflow, particular because there is a conflict between kedro new and these package manager.

Can you explains what's the challenge when you try to use rye and describe the current process that you use to create a Kedro project?

It's unlikely kedro is going to switch to any package manager in a near future, the approach to solve this is instead having a plugin. There is an example here https://github.com/astrojuanlu/kedro-init for poertry.

[astrojuanlu]

The rye maintainers have expressed that they intend to maintain rye but they eventually want people to migrate to uv

https://x.com/charliermarsh/status/1825965866607063049

We're continuing to maintain Rye but focusing on uv for all new development. Our goal is to enable Rye users to migrate to uv.

So I think we should evolve the question to "Add support to uv".

In any case, what does this mean? You can take any Kedro starter and use uv add (or rye add) on it. So my hunch is that this is already supported.

Therefore my echo @noklam's question @marrov: What does "add support for Rye (or uv) in Kedro" mean to you?

[astrojuanlu]

There's now consensus to proceed with #4116 let's continue the discussion there