Should I create new workflow instances every time or share the same instance? Which approach is correct?

[minmie]

share:

from flask import request
from llama_index.core.workflow import Workflow


class MyWorkflow(Workflow):
    pass

agent = MyWorkflow()


@app.route("/query", methods=["GET"])
def query_index():
    query = request.args.get("text", None)
    response = agent.run(query=query)
    return str(response), 200

create each time:

from flask import request
from llama_index.core.workflow import Workflow


class MyWorkflow(Workflow):
    pass


@app.route("/query", methods=["GET"])
def query_index():
    query = request.args.get("text", None)
    agent = MyWorkflow()
    response = agent.run(query=query)
    return str(response), 200

[dosubot[bot]]

The decision to create new instances of MyWorkflow for each request or to share a single instance depends on several factors, including thread safety and resource management.

1. Thread Safety: If MyWorkflow or the Workflow class maintains internal state that could affect the outcome of subsequent requests, sharing a single instance across multiple requests could lead to race conditions or inconsistent results. In such cases, creating a new instance for each request is safer.

2. Resource Management: Creating a new instance for each request can be more resource-intensive, especially if the initialization of MyWorkflow is costly. However, if the class is lightweight and does not hold significant resources, this might not be a concern.

3. Concurrency: If your application handles concurrent requests, and MyWorkflow is not designed to be thread-safe, sharing a single instance could lead to issues. In this scenario, creating a new instance for each request is advisable.

Unfortunately, specific details about the thread safety and state management of the Workflow class in the llama_index.core.workflow module are not available in the provided information. If the class documentation or source code does not explicitly mention thread safety, it's generally safer to assume that it is not thread-safe and create a new instance for each request.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[logan-markewich]

@minmie Workflow instances can be shared. When you call workflow.run(), that run is completely isolated. In fact, if you do something like

handler = workflow.run(..)
async for ev in handler.stream_events():
  print(ev)
result = await handler

It exposes the handler, which holds all the logic and context for that specific run.

The only reason I wouldn't re-use the class instance is if your custom workflow is using self to store runtime data instead of using the context (I would not advise using self for anything but static data)

[minmie]

The only reason I wouldn't re-use the class instance is if your custom workflow is using self to store runtime data instead of using the context (I would not advise using self for anything but static data)

Thanks for your reply！