Feat: Simple @function entrypoint

[mrkaye97]

Adding a new @function entrypoint to hatchet to let you register a python function directly as a workflow. The main important thing here is we define a Function class that we can pass to register_function so that all the types check out correctly, and the function contains a step, workflow config, and everything else we need for registration. And then when we go to register the function, we just instantiate a new workflow and manually create its step(s).

At least in theory, this should get us really close to Celery. One thing that's a little unfortunate is that you run the risk of circular import issues if you import one of these functions into a different module, but I'd argue in that case it's probably better to use the workflow declaration pattern. I'll keep thinking about that though! There might be some middle ground here.

The simple example lays out usage, but here's it copied over:

from hatchet_sdk import Context
from hatchet_sdk.v2 import Hatchet

hatchet = Hatchet(debug=True)


@hatchet.function()
def step1(context: Context) -> dict[str, str]:
    message = "Hello from Hatchet!"

    context.log(message)

    return {"message": message}


if __name__ == "__main__":
    worker = hatchet.worker("test-worker", max_runs=1)
    worker.register_function(step1)
    worker.start()

[mrkaye97]

one question here is if we want to prefix these or something, since some pertain to the workflow, some pertain to the step, and some to both (IMO this is confusing)

[grutt]

i think one of the goals we should consider is to make everything effectively run as a step so all features are always in scope. in other words, a dag is a step that calls steps (somehow...). ultimately, i think the hatchet service should still be responsible for dags but i think we can orchestrate through the main scheduler. i know @abelanger5 has thoughts here.

we might want to feature space some of these things to simplify?

i.e.:

on: TriggerConfig
retries: int | RetryConfig

[mrkaye97]

I have mixed feelings here I think. I sorta like the class-based thing for defining DAGs like we have it now, because it's pretty clear what's related to what, and I'm nervous that without that we'd just end up with a mishmash of random functions hanging around that all depend on each other, which could be hard to untangle. That said, I do really like being able to just do @function or whatever and have it all kinda magically work.

I'm sure there's a way to make this work nicely though (I don't think there's any reason that e.g. you couldn't specify parents on a function or something, similar to how we have it now). Happy to discuss some more here!

re: the second thing - feature spacing sounds great to me, I think this makes a lot of sense 👍