Scalability using NATS queue

[GHkrishna]

What

Currently we had explored NATS queue and I think we can leverage it for horizontal scaling of agent. Here's how the overall architecture would look like:

This architecture demonstrates a horizontally scalable system where multiple credo-agent instances consume messages from a central NATS server in a controlled and distributed manner using queue groups.

What advantages do we get:

• Horizontal scaling of controller
• Reuse existing NATS infra (if suitable)
• Avoid API calls to the controller
• An added NATS authentication (security)layer on top of the existing API key

Blockers (might have)

• Request received from agent to agent might need to be additionally load-balanced between the inbound endpoint for agents
• Database sync between the agents need to be considered

References

Read more about NATS queue here: https://docs.nats.io/nats-concepts/core-nats/queue