feat: d1 multi-tenancy

[zpg6]

D1 Multi-Tenancy

What is this? - Well, isolating data for your customers into separate databases by-user or by-organization is a common requirement. This offers that capability but in a way that feels native to better-auth. Utilizes hooks for when users and organizations are created and deleted to likewise create and delete the tenant databases. We added some support to the CLI to make this fit, and we extended both Drizzle and BetterAuth to make this happen.

TODOs

• Create new internal BetterAuthPlugin cloudflareD1MultiTenancy.
• Track migration status of tenants
• Offer by-user and by-organization multi-tenancy options
• Use Cloudflare's D1 Rest API to create and delete the databases on user/org hooks.
• Extend CLI to split the schema into "main database" schema (core Better-Auth tables) and tenants schema when multi-tenancy settings are detected
• Populate a newly created tenant database with the latest schema for tenants
• Offer hooks for before/after database creation and before/after database deletion
• Extend Drizzle with a new Driver d1-http that wraps Cloudflare's D1 Rest API's endpoint to execute raw SQL. This is needed since current driver only supports your tenant DBs can't be "bound" to your Worker (ie in your wrangler.toml).
  • feat: REST-based D1 - feat: REST-based D1 drizzle-team/drizzle-orm#4881
• Benchmark that runs within a deployed Cloudflare Worker to compare the latency of different queries and such on Bound D1 vs D1 HTTP drizzle drivers.
  • See source code zpg6/d1-binding-vs-http-benchmark
• Extend BetterAuth with a new "AdapterRouter" concept that allows powerful configuration over how adapter calls are routed to different databases.
  • feat: AdapterRouter (experimental): feat: AdapterRouter (experimental) better-auth/better-auth#4236
• Basic integration of both d1-http driver and adapterRouter into cloudflareD1MultiTenancy plugin.
• Create new examples/opennextjs-org-multi-tenancy example that demonstrates a multi-tenant birthday app
• Extend CLI to migrate an updated tenant schema to all existing tenants,
• Documentation in docs/d1-multi-tenancy
• Add Cursor rules file to examples/opennextjs-org-multi-tenancy

Limitations

• Core BetterAuth models (users, accounts, sessions, organizations, members, invitations, tenants) are stored in main database. Other models added by better-auth plugins or linked in your tenant schema by you manually will go to tenant database. What goes where could be more configurable.
• For now all tenants must have same schema
• Detection of multi-tenancy settings when generating split main/tenant schemas with CLI is fragile

How it Works - Routing to the correct database

[zpg6]

Able to create and delete empty databases:

[zpg6]

Added d1-http driver to drizzle:

feat: REST-based D1 - drizzle-team/drizzle-orm#4881

Now we should be able to:

function createTenantDb(tenantDatabaseId: string) {
	return drizzle({
		accountId: 'your-cloudflare-account-id',
		databaseId: tenantDatabaseId,
		token: 'your-cloudflare-api-token',
	});
}

[zpg6]

Can split schema now with the CLI migrate command - detects if multi-tenancy setting is enabled and if so splits based on configuration.

┌  Better Auth Cloudflare v0.1.16 · migrate
│
◇  Auth schema updated.
│
◇  Schema split into auth.schema.ts (core) and tenant.schema.ts (tenant-specific).
│
◇  Database migrations generated.
│
└  Migration completed successfully! Database migration was skipped as requested.

[zpg6]

Added PR to better-auth for Adapter Router - this would handle determining which database to hit (main or tenant) based on the model being queried.

feat: AdapterRouter (experimental): better-auth/better-auth#4236

[zpg6]

Able to create databases on org creation and it migrates the latest version of the schema. Includes migration version tracking in the main tenants table.

[zpg6]

First tenant record 🎉

[zpg6]

Schema splitting

Added full drizzle support for the tenant schema, so now you have drizzle/ and drizzle-tenant/ migration dirs. And you have a separate drizzle.config.ts and drizzle-tenant.config.ts.

examples/opennextjs-org-d1-multi-tenancy/
├── 📄 drizzle.config.ts              # Main/Auth database configuration
├── 📁 drizzle/                       # Main/Auth migrations
│   ├── 0000_clumsy_ultimates.sql
│   ├── 0001_eminent_meggan.sql
│   └── meta/
│       ├── _journal.json
│       ├── 0000_snapshot.json
│       └── 0001_snapshot.json
├── 📄 drizzle-tenant.config.ts       # Tenant database configuration  
├── 📁 drizzle-tenant/                # Tenant-specific migrations
│   ├── 0000_steady_falcon.sql
│   ├── 0001_wide_agent_zero.sql
│   ├── 0002_kind_carnage.sql
│   └── meta/
│       ├── _journal.json
│       ├── 0000_snapshot.json
│       ├── 0001_snapshot.json
│       └── 0002_snapshot.json
└── src/
    └── db/
        ├── auth.schema.ts            # Main/Auth schema definitions
        ├── tenant.schema.ts          # Tenant schema definitions
        └── tenant.raw.ts             # Raw tenant database utilities

Tenant Migrating

Applying new migrations to existing tenants now looks like:

┌  Better Auth Cloudflare vunknown · migrate:tenants
│
◇  Found 1 tenant database(s)

Migration Plan:
Will check and apply any pending migrations to 1 tenant database(s):
  pkugmrQlW0fxHggm11Lf6FtZB60HK16i (org_tenant_pkugmrQlW0fxHggm11Lf6FtZB60HK16i)
│
◇  Check and apply migrations to 1 tenant database(s)?
│  Yes

Applying migrations:
  → pkugmrQlW0fxHggm11Lf6FtZB60HK16i - Checking for migrations
  ✓ pkugmrQlW0fxHggm11Lf6FtZB60HK16i - Migrations applied successfully
│
└  ✅ 1 of 1 tenant databases migrated successfully

[zpg6]

CLI

npx @zpg6-test-pkgs/better-auth-cloudflare-cli@0.1.16-tenants.2 --help
npx @zpg6-test-pkgs/better-auth-cloudflare-cli@0.1.16-tenants.2 migrate
npx @zpg6-test-pkgs/better-auth-cloudflare-cli@0.1.16-tenants.2 migrate:tenants

better-auth-cloudflare

npm install @zpg6-test-pkgs/better-auth-cloudflare@0.2.4-tenants.1

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/zpg6/better-auth-cloudflare/@zpg6-test-pkgs/better-auth-cloudflare@17

commit: 55aebc8

[zpg6]

This is still unstable - the D1 orchestration works well but the BetterAuth Adapter split across two databases isn't nice to work with.

[ataylorme]

This is still unstable - the D1 orchestration works well but the BetterAuth Adapter split across two databases isn't nice to work with.

@zpg6 I would like this feature and am willing to contribute. Can you expand on what you have in mind to make the BetterAuth Adapter split experience better?

[zpg6]

This is still unstable - the D1 orchestration works well but the BetterAuth Adapter split across two databases isn't nice to work with.

@zpg6 I would like this feature and am willing to contribute. Can you expand on what you have in mind to make the BetterAuth Adapter split experience better?

Hi @ataylorme !

TLDR: try to run it and feel the pain points. It could work but it needs more adapter logic and will have some limitations for sure. It's just fragile.

• Since I began this feature, I've noticed that better-auth has added more relationships between the tables. More adapter logic is needed to bridge better-auth's expected relationships in your single database to the different databases

• no clear way of maintaining the separated drizzle schemas in peoples long term projects as better-auth team makes updates to their tooling

• it should handle plugins that modify the schema (this is why we use the better-auth cli for that part)

[ataylorme]

That is quite a lot of pain points. I was hoping to have everything on CloudFlare but Neon or similar with Hyperdrive seems like the path of least resistance compared to overcoming the above.

I came across a blog post on D1 sharding architecture that describes a Universal ID-Based Sharding approach that might reduce some of the fragility here.

The key idea: embed shard/routing metadata directly in record IDs such as <timestamp><shardHash><typeHash><random>. This would eliminate the need to hit the tenant mapping table on every read as the IDs are self-describing, having all the info necessary to determine which database to query.

The d1-http driver and lifecycle hooks for DB provisioning could still handle the dynamic database creation, but the routing would become more resilient since it's not dependent on a central lookup for every operation.

[zpg6]

That is quite a lot of pain points. I was hoping to have everything on CloudFlare but Neon or similar with Hyperdrive seems like the path of least resistance compared to overcoming the above.

I came across a blog post on D1 sharding architecture that describes a Universal ID-Based Sharding approach that might reduce some of the fragility here.

The key idea: embed shard/routing metadata directly in record IDs such as <timestamp><shardHash><typeHash><random>. This would eliminate the need to hit the tenant mapping table on every read as the IDs are self-describing, having all the info necessary to determine which database to query.

The d1-http driver and lifecycle hooks for DB provisioning could still handle the dynamic database creation, but the routing would become more resilient since it's not dependent on a central lookup for every operation.

This is awesome! 👏

I think you may have found the answer to the biggest pain point

[ataylorme]

@zpg6 I had Copilot take a first pass at a hybrid approach. Mind taking a look and seeing if it is a direction you are interested in pursuing?

[zpg6]

@zpg6 I had Copilot take a first pass at a hybrid approach. Mind taking a look and seeing if it is a direction you are interested in pursuing?

I like the plan at a high level, we should pursue it.

Before we go too far, I want to check back in on how this handles the separate drizzle migrations. Still have to tackle the "relationships mapping" adapter a bit I think. Add a few plugins to trigger the change in schema.

[ataylorme]

Oh yeah, I’m not suggesting to accept the copilot code but it is a good reference to get some ideas