Multi Database subscriptions

[ItsRobbAllen]

Here is our situation. We are following a microservice pattern where each service has its own database. We also are not using a multi-tenant strategy, so each client gets a set of databases for themselves.

For example, we have two clients, Acme and Bravo. With one db per service we'd have— Acme.Units, Acme.Concepts, Acme.Master, Acme.Financials as well as Bravo.Units, Bravo.Concepts, Bravo.Master, Bravo.Financials

If we call the Financials microservice, each endpoint examines the JWT sent across in the authorization header to determine which tenant the call belongs to and loads up the correct database. This works so well & was so easy to implement I keep thinking I must have made a critical security mistake somewhere, but it's Raven... it just works ;)

My issue now is that I have sales data in Excel format that I need to upload and then process asynchronously. Subscriptions are the way to go for this, but... what is the best approach for subscribing to many databases?

Do I create a single service that I deploy per tenant? Do I deploy one service that covers all tenants & just create many subscriptions from many stores? A single subscription with multi-tenant data in a single database isn't allowed as there's a hard rule about not mingling client data.

Just looking for ideas or gotchas that I might need to look out for.

Thanks,
R

[georgiosd]

@ItsRobbAllen just curious, what is the benefit of splitting up the databases so much?

Beyond being curious, I'd imagine that the answer on how to best organize the subscriptions lies in this answer, too.

[ItsRobbAllen]

Microservice theology aside ("If you have microservices, but one shared database... you do not have microservices") the issue exists because I am not doing a multi-tenant system. Each tenant would get its own database still, thus I need to have a single service that can span multiple databases.

[mcrio]

I guess that heavily depends on the scenario and the size of the tenant. I am more for the approach of a shared database with the ability to migrate a tenant to a new, empty database.

[georgiosd]

@ItsRobbAllen I appreciate the "theology" comment very much. Very often I think devs get lost in "scripture" trying to materialize an imagined benefit.

Having said that I think the easiest way would be to have one service that creates one subscription per tenant. I don't see a benefit to having multiple services each creating one subscription to one tenant, unless you want to be able to have different versions of the service for different tenants - to manage upgrades differently. From a processing perspective, you're still going to have N subscriptions to N databases - so it's potato potato.

[ItsRobbAllen]

Been doing this job professionally for over 27 years and I fully appreciate how we can make our lives difficult in order to stay "pure". I mean, eventually, everyone gives up on CSS and just uses

, right? :)

This is basically the approach I'm working with. Single service, multiple subscriptions, one per Tenant. Everything else felt like I was just adding complexity.

[ayende]

A subscription is per database. In your context, I would recommend that you'll start with a single service that would create a subscription and listen to all the databases. The isolation here is that you basically have a container for subscription workers, so even though they are running in the same space, they are still separated.

[ItsRobbAllen]

That's where I was leaning, just wanted to make sure I'm not missing something obvious.

Thanks!