Multi-Database Support Limitations

[miguelmarcondesf]

Hello there!
I'm doing some studies and experiments with multi-database setups, and I stumbled upon some limitations and conflicts when trying to use multiple tenanted connection classes, which I think are expected for now.

Use Case

Healthcare management system requiring:

• ApplicationRecord: Users, Staff, Appointments (main business logic)
• PatientsRecord: Sensitive patient data (separate database for HIPAA compliance)
• Both databases need tenant isolation per medical practice

My database config:

// config/database.yml
development:
  global: # storing Practice to handle new Tenant creations and Session
    <<: *default
    database: storage/development_global.sqlite3
  primary: 
    <<: *default
    tenanted: true 
    database: storage/development/%{tenant}.sqlite3
  patients: 
    <<: *default
    tenanted: true
    database: storage/development/patients_%{tenant}.sqlite3
    migrations_paths: db/patients_migrate
  patients_replica:
    <<: *default
    tenanted: true
    replica: true
    database: storage/development/patients_%{tenant}.sqlite3

What I Tried

Manual Tenant Synchronization

Being called on active_record_tenanted.tenant_resolver

  class MultiTenantResolver
    def self.call(request)
      tenant = request.subdomain
      if tenant
        PatientsRecord.current_tenant = tenant
        return tenant
      end
    end
  end

Expected result: cannot swap sharp while shard swapping is prohibited

Custom Middleware

  class MultiTenantMiddleware
    def call(env)
      tenant = extract_tenant(env)
      ApplicationRecord.current_tenant = tenant
      PatientsRecord.current_tenant = tenant
    end
  end

Expect Result: Same shard swapping prohibition

Connection Override

  class PatientsRecord < ActiveRecord::Base
    def self.connection
      if ApplicationRecord.current_tenant && !current_tenant
        with_tenant(ApplicationRecord.current_tenant) { super }
      end
    end
  end

expected Result: Circular dependencies and shard swapping errors

Rails connects_to + activerecord-tenanted

  class PatientsRecord < ApplicationRecord
    tenanted :patients
    connects_to database: { writing: :patients }
  end

Expected Result: Conflicts between gems tenant switching and native multi-database

Other problems

Migration Management

• Only runs migrations for configured connection class
• Secondary databases (like db/patients_migrate/) ignored

Final Workaround

For now, to continue digging the gem, I just migrated things from the second tenanted database to the primary one

Suggested Improvements

Multi-Connection Support

Helping with handling the context switching and correctly finding migrations

  Rails.application.configure do
    config.active_record_tenanted.connection_classes = {
      "ApplicationRecord" => :primary,
      "PatientsRecord" => :patients
    }
    config.active_record_tenanted.tenant_resolver = ->(request) { request.subdomain }
  end

Thank you in advance and let me know if it makes sense

[murny]

Not to distract from your issue, but I am quite interested in this as I think using Solid gems like SolidCache also suffers from a similar issue.

As when using SolidCache you may be using multiple database under the with_tenant block. So when it connects to the cache DB inside the with_tenant it ends up with a similar error:

cannot swap `shard` while shard swapping is prohibited.

[flavorjones]

Yes, I mention this in GUIDE.md, it's a limitation in Rails' horizontal sharding that I'd like to fix:

• upstream
  • feature: make shard swap prohibition database-specific
    • which would work around Solid Cache config wonkiness caused by Always manage connections when database is specified rails/solid_cache#219

[flavorjones]

In the meantime you can work around it by updating your solid cache config to look something like this:

# config/cache.yml
default_connection: &default_connection
  connects_to:
    database:
      writing: :cache

default: &default
  <<: *default_connection
  <<: *default_options

development: *default
test: *default_options
beta: *default
staging: *default
production: *default

[murny]

Awesome, will give this a try! Thanks @flavorjones

[flavorjones]

@miguelmarcondesf Thanks for asking this question!

multiple tenanted connection classes

This is not something I've attempted to do myself. The database connections should mostly work, but the Rails integrations controlled by setting config.active_record_tenanted.connection_class are definitely not going to work.

I'm deeply curious about this use case -- generally speaking, what are the shear zones?

For example, I can imagine that Action Cable integration would be affined with the ApplicationRecord in your app -- the user who is logged in.

But I have no idea what Active Storage should do here, since obviously there are going to be some blobs associated with the user (e.g., avatars) and some blobs associated with the patient (e.g., x-rays). Have you thought through how you want that to behave?

My recommendation to you is to set config.active_record_tenanted.connection_class to nil and then build what you need based on patterns in this codebase.

[miguelmarcondesf]

It's a project I started to study and experiment with what's coming in 8.1 and the tenanted so I could write a blog post about it.

I started just using multiple databases to isolate patient data (thinking about some more rigorous criteria for this data perhaps) and also to be able to have a read-only replica.
So when I added tenanted, I kept this idea of ​​trying to maintain multiple tenanted databases.

Thanks for the recommendation and attetion, for the next steps I want to take I will try to follow what you said.