Commit f2e05d2
authored
Replace Azure SQL Server with PostgreSQL Flexible Server (breaking change) (#860)
### Summary & Motivation
Migrate the entire database layer from Azure SQL Server to PostgreSQL
Flexible Server. PostgreSQL provides a lighter local development
experience (smaller Docker image, faster startup), lower Azure hosting
costs, and better alignment with the open-source ecosystem.
This is a clean break with no migration path from SQL Server. All
existing migrations are deleted and replaced with fresh
PostgreSQL-native migrations using snake_case conventions throughout.
**Application changes:**
- Replace the SQL Server EF Core provider with Npgsql, applying
snake_case naming via `EFCore.NamingConventions` with explicit
`MigrationsHistoryTable("__ef_migrations_history")` since the naming
convention does not apply to the history table
- Delete all SQL Server migrations and create fresh PostgreSQL
migrations using native data types (`text`, `timestamptz`, `boolean`,
`integer`, `jsonb`) with snake_case naming, including
`__data_migrations_history` table and
`fk_subscriptions_tenants_tenant_id` foreign key
- Store manually serialized JSON columns (`ExternalIdentities`,
`PaymentTransactions`, `BillingInfo`, `Payload`) as `jsonb` with
`.HasColumnType("jsonb")` alongside `HasConversion`
- Rewrite `DataMigrationRunner` with PostgreSQL advisory locks using a
deterministic SHA256-based lock key, an independent `NpgsqlDataSource`
connection, a required per-migration `Timeout` (max 20 minutes),
`ManagesOwnTransactions` opt-out for batch migrations outside execution
strategy retry, and error handling on lock release
- Replace SQL Server locking patterns: `FOR UPDATE` in
`SubscriptionRepository`, LINQ queries in
`UserRepository.GetUserSummaryAsync` (with SQLite raw SQL fallback for
`DateTimeOffset` comparison), and a single snake_case SQL statement in
`SessionRepository.TryRefreshAsync`
- Add `.OrderBy(u => u.Id)` to `GetUsersByEmailUnfilteredAsync` since
PostgreSQL does not guarantee row ordering without `ORDER BY`, and use
`SingleOrDefaultAsync` for primary key lookups
- Use `DefaultAzureCredential` with `ManagedIdentityClientId` for the
database token provider via `UsePeriodicPasswordProvider`
- Apply `UseSnakeCaseNamingConvention()` to SQLite test configuration
and update all raw SQL across 55 test files to snake_case
- Add `Plan` property to `Tenant` to denormalize the subscription plan,
restrict `GetPaymentHistory` to owners only
- Fix flaky Stripe webhook tests with `[Collection("StripeTests")]`
**Infrastructure changes:**
- Replace SQL Server Bicep modules with PostgreSQL Flexible Server using
Entra ID-only authentication, Private Endpoint with Private DNS Zone on
a dedicated `private-endpoints` subnet, and `Ssl Mode=VerifyFull` for
certificate verification
- Split VNet into two subnets: `container-apps` (delegated to
`Microsoft.App/environments`) and `private-endpoints`, since delegated
subnets cannot host private endpoints
- Upgrade Container Apps from consumption-only to workload profiles with
a Consumption profile, providing identical billing, higher per-app
limits (4 vCPUs/8 GiB), and future dedicated compute support
- Configure `pg_stat_statements` for query monitoring,
`log_statement=mod` for audit logging with diagnostic logs routed to a
storage account with 90-day lifecycle retention, and `wal_level=logical`
for logical replication support
- Chain server configuration changes after Private DNS Zone Group
provisioning to prevent first-time deployment failures
- Replace Bicep `dependsOn` with module output references for proper
implicit dependencies and `--what-if` visibility
- Move Entra ID admin provisioning to a post-deployment CLI step since
PostgreSQL cannot provision admins with `passwordAuth: 'Disabled'` in a
single Bicep deployment
**CI/CD changes:**
- Use `psql` with `sslmode=verify-full sslrootcert=system` and `-v
ON_ERROR_STOP=1` (without `--single-transaction` to support `CREATE
INDEX CONCURRENTLY`) for migration apply, and `Ssl Mode=VerifyFull` for
`dotnet ef` connections
- Open firewall once for all database permission grants instead of
per-database
- Add `set -e` to shell scripts and IP address validation in
`firewall.sh`
- Authenticate using the Entra admin group principal name as the
PostgreSQL username with the service principal's access token
- Rename security group from "SQL Admins" to "PostgreSQL Admins" and all
related GitHub variables
**Documentation changes:**
- Update README files, legal documents (DPA, cross-references), and
developer CLI to reference PostgreSQL
- Update AI rules (`api-tests.md`, `domain-modeling.md`,
`database-migrations.md`, `backend.md`, `repositories.md`) to reflect
PostgreSQL conventions including snake_case examples,
`HasColumnType("jsonb")` guidance, `CREATE INDEX CONCURRENTLY` support,
and PostgreSQL ordering behavior
### Downstream projects
**This is a breaking change.** The database provider changed from SQL
Server to PostgreSQL. Delete existing SQL Server migrations and create a
new initial migration using PostgreSQL conventions. There is no
migration path to move data in a production system.
See the updated [database-migrations rule
file](/.claude/rules/backend/database-migrations.md) and the new
[account initial
migration](/application/account/Core/Database/Migrations/20260303023200_Initial.cs)
for reference.
Rename the Entra ID security groups in Azure from "SQL Admins" to
"PostgreSQL Admins" (e.g., `SQL Admins - Staging -
GitHubOrganisation/GitHubRepository` becomes `PostgreSQL Admins -
Staging - GitHubOrganisation/GitHubRepository`). Then rename the
following GitHub Actions variables:
- `STAGING_SQL_ADMIN_OBJECT_ID` to `STAGING_POSTGRES_ADMIN_OBJECT_ID`
- `PRODUCTION_SQL_ADMIN_OBJECT_ID` to
`PRODUCTION_POSTGRES_ADMIN_OBJECT_ID`
### Optional: Enable Microsoft Defender for PostgreSQL
For vulnerability assessments and security alerts, enable **Microsoft
Defender for Open-Source Relational Databases** in your Azure
subscription. This provides brute force detection, anomalous access
alerts, and vulnerability scanning for PostgreSQL Flexible Server
(~$15/month per server). Enable it in the Azure Portal under Microsoft
Defender for Cloud > Environment settings > Defender plans, or via Azure
CLI:
```bash
az security pricing create --name OpenSourceRelationalDatabases --tier Standard
```
### Checklist
- [x] I have added tests, or done manual regression tests
- [x] I have updated the documentation, if necessaryFile tree
135 files changed
+2284
-8703
lines changed- .agent/workflows
- modes
- process
- .claude/rules/backend
- .cursor/rules/workflows
- modes
- process
- .github/workflows
- .windsurf/workflows
- modes
- process
- application
- AppHost
- account
- Core
- Database
- DataMigrations
- Migrations
- Features
- Authentication/Domain
- Billing/Queries
- EmailAuthentication/Domain
- ExternalAuthentication/Domain
- Subscriptions
- Domain
- Tenants/Domain
- Users/Domain
- Tests
- Authentication
- Billing
- EmailAuthentication
- ExternalAuthentication
- Signups
- Subscriptions
- Tenants
- Users
- WebApp/routes/legal
- back-office
- Core/Database/Migrations
- Tests
- main
- Core/Database/Migrations
- Tests
- cloud-infrastructure
- cluster
- environment
- modules
- developer-cli/Commands
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
135 files changed
+2284
-8703
lines changedThis file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
0 commit comments