[Question] Multi-tenant configuration

[ChrisW-priv]

Hi again,
fast-agent seems to be designed heavily around the idea that the configuration of an agent is for a single user (correct me if I'm wrong).

But I would like to design a single endpoint on my server (/chat) that would take the auth header from the caller (team member or client) and create a custom Agent only for them (MCP configs, prompt and even models different, based on some internal rules).
When I started I setup some simple stdio server just to test the connections etc.
But, now I want to create a config based on some rules.

Would you say this is feasible? I was diving quite deep today on the code and seems to me like it should be doable, just with some Settings defined manually and possibly some cache not to kill the performance.

Do you have any experience with this? Would you say this approach is fine or is this discouraged? In case of the latter, what would be your suggested solution?

Regards,
KW

[evalstate]

Related discord discussion: https://discord.com/channels/1358470293990936787/1358470293990936790/1420065029797056565

[ChrisW-priv]

ok, I think I've got an Idea on how to do this. It would require a bit of a rewrite but the result is quite elegant from both internal app arch and end user perspectives.

This configuration here was stuck in my head and I could not figure out how to do it elegantly with the entire App etc.

But my proposal is: If you want to create an agent that will call MCP servers (or any other setting) with some diff config than the default, you should define some functions that will modify the core, and pass them into the "run" context manager.

    @asynccontextmanager
    async def run(self) -> AsyncIterator["AgentApp"]:
        """
        Context manager for running the application.
        Initializes all registered agents.
        """

This way, you will still get the nice-ness of the decorator implementation:

"""
"knowledge" server is just some generic name of the server you can define in the fastagent.config.yaml
For the sake of example, let's say that the knowledge access is filter by some user access policy.
To authenticate, user needs to pass "KNOWLEDGE_USER_TOKEN" env variable. 
We want to build an app that dynamically (in the same runtime) will start diff mcp servers with different 
values of this env variable.
"""

import asyncio
from fast_agent import FastAgent

def change_env_of_server(server_name: str, env_name: str, new_env: str):
    def inner(server_registry: ServerRegistry):
        new_registry = copy.deepcopy(server_registry)
        new_server_settings = copy.deepcopy(new_registry.registry[server_name])
        new_server_settings.env[token_name] = new_token
        new_registry.registry[server_name] = new_server_settings
        return new_registry
    return inner

change_token_to_diff = change_env_of_server('knowledge', 'KNOWLEDGE_USER_TOKEN', 'DIFFERENT_TOKEN_THAN_ORIGINAL')

# Create the application
fast = FastAgent("RAG Agent")
  
@fast.agent(
    "knowledge_retriever",
    instruction="Given a user query, retrieve relevant information to the conversation, and answer the query",
    servers=["knowledge"],
)
async def main() -> None:
    async with fast.run(
            core_modifiers = [change_token_to_diff]  # Here pass some list of functions that modify the Core object of the "fast" application
        ) as agent:
        return await agent.knowledge_retriever.generate("Tell me about the history of The Spanish Inquisition")


if __name__ == "__main__":
    asyncio.run(main())

@evalstate What do you think? This would require changing one of the core components to the framework, but can be easily done in backwards compatible way and all that.

[evalstate]

Not sure...

The decorators are a convenience for creating AgentConfig() -- which can then be used to instantiate as many Agents instances as you want. So for straightforward workflows that's the easiest route - but for Multi-Tenant systems, we can (and probably should) allow AgentConfig/MCPServerSettings to be loaded from a file. See #371 and #372 for some ideas there (and even #199).

I'm not sure that a mutator function being passed around is necessarily the right path, but certainly open to exploring the idea.

[ChrisW-priv]

This is not a small change, but I'd back this idea with a fact that we still get the TEMPLATE from the decorators and then can modify the params AT RUNTIME. This is super convenient if you want to build a system that will respond to many different users that may not have the same permissions for example. I'm thinking: User A and B that have to use Tokens "ABC" and "DEF" for the MCP server to return correct data (according to their authorization level). Without this change, you'd have to define separate files for ALL users which could be (countibly) infinate. Right now, the only option I can think of is to deploy separate server, each running the dedicated instance of the agent.

Here, all you have to do is overwrite some specific part of the config based on some policy. Quite straightforward really. Default config, and optional overwrites.

[evalstate]

Sure -- bear with me for a second... pseudocode incoming:

If we have an MCPServerSettings:

user_token = lookup( token_for_user (user_id) )

my_server = MCPServerSettings(
   ...
   headers: { "foo": user_token } 
   ... 
)

agent_config = AgentConfig(name="service",servers=[my_server])  <---- this overload for servers is missing
my_agent=McpAgent(agent_config)

my_agent.attach_llm()


etc. etc.

Isn't that kind of what you'd want? Or you could of course copy and mutate my_server of course - but this seems potentially less surprising.

[ChrisW-priv]

interesting... and I guess it solves the problem of "per user" mcp tokens. But then, we are forced to use lower level Agent creation not a FastAgent App. I will sleep on how to integrate it more easily...

[evalstate]

But then, we are forced to use lower level Agent creation

No reason not to be able to get hold of AgentConfigs easily (can't imagine this would be too hard):

@fast.agent(name="foo",instruction="bar",...)



async def main():
    # use the --model command line switch or agent arguments to change model
    async with fast.run() as agent_app:
        config = agent_app.configs["foo"]
        # mutate config copy
        actual_agent = agent_app.create(config)

I'm sure there's a bit more to it than that - and probably better abstractions to be had, but the pieces should all roughly be there.