How to build a voice/sms virtual agent using langgraph

[knightsrule]

Hi,

I am building a voice and sms virtual agent powered by langgraph.

I have a fastapi server with routes for incoming sms and voice handling. These routes, then call the langgraph app.

Current, minimal create_agent and build_graph looks like this:

async def build_graph():

    builder = StateGraph(VirtualAgentState)

    idv_agent = AgentFactory.create_agent("idv")
    appts_agent = AgentFactory.create_agent("appts")

    supervisor = create_supervisor(
        
agents
=[idv_agent, appts_agent],
        
model
=LLMFactory.get_llm("small_llm"),
        
prompt
=(
            "You manage a user authentication assistant and an appointment assistant. Assign work to them."
        )
    )

    builder.add_node("supervisor", supervisor)

    builder.add_edge(START, "supervisor")
    
#builder.add_node("human", human_node)

    checkpointer = MemorySaver()
    graph = 
await
 builder.compile(
checkpointer
=checkpointer)

    
return
 graph

@staticmethod
async def lookup_agent_config(agent_id: str):
    
if agent_id == "idv":
        
return
 {
            "model": LLMFactory.get_llm("small_llm"),
            "tools": [lookup_customer, send_otp, verify_otp],
            "prompt": "You are a user authentication assistant. You will prompt the user for their phone number and pin. Then, you will validate this information using lookup_customer tool. If you find a vaild customer, send a one time passcodde using send_otp tool and then validate this otp using verify_otp tool. If the otp is valid, return the customer id to the user.",
            "agent_id": 
agent_id
        }

There are few things that I havne't been able to sort out.

How should each agent indicate that they need a user input. Looking at the documentation, i should be using the human in the loop mechanism, but it is not clear where in the graph that will show and how will the tools indicate the need for an input.

When the user input comes via sms/voice channel, will graph ainvoke/astream be sufficient to resume the conversation within each agent?

most of the examples that i've seen are notebook or console based and don't show FastAPI. Is there an better example that shows the same concept with FastAPI.

Thanks!