Skip to content

Commit f2e05d2

Browse files
authored
Replace Azure SQL Server with PostgreSQL Flexible Server (breaking change) (#860)
### Summary & Motivation Migrate the entire database layer from Azure SQL Server to PostgreSQL Flexible Server. PostgreSQL provides a lighter local development experience (smaller Docker image, faster startup), lower Azure hosting costs, and better alignment with the open-source ecosystem. This is a clean break with no migration path from SQL Server. All existing migrations are deleted and replaced with fresh PostgreSQL-native migrations using snake_case conventions throughout. **Application changes:** - Replace the SQL Server EF Core provider with Npgsql, applying snake_case naming via `EFCore.NamingConventions` with explicit `MigrationsHistoryTable("__ef_migrations_history")` since the naming convention does not apply to the history table - Delete all SQL Server migrations and create fresh PostgreSQL migrations using native data types (`text`, `timestamptz`, `boolean`, `integer`, `jsonb`) with snake_case naming, including `__data_migrations_history` table and `fk_subscriptions_tenants_tenant_id` foreign key - Store manually serialized JSON columns (`ExternalIdentities`, `PaymentTransactions`, `BillingInfo`, `Payload`) as `jsonb` with `.HasColumnType("jsonb")` alongside `HasConversion` - Rewrite `DataMigrationRunner` with PostgreSQL advisory locks using a deterministic SHA256-based lock key, an independent `NpgsqlDataSource` connection, a required per-migration `Timeout` (max 20 minutes), `ManagesOwnTransactions` opt-out for batch migrations outside execution strategy retry, and error handling on lock release - Replace SQL Server locking patterns: `FOR UPDATE` in `SubscriptionRepository`, LINQ queries in `UserRepository.GetUserSummaryAsync` (with SQLite raw SQL fallback for `DateTimeOffset` comparison), and a single snake_case SQL statement in `SessionRepository.TryRefreshAsync` - Add `.OrderBy(u => u.Id)` to `GetUsersByEmailUnfilteredAsync` since PostgreSQL does not guarantee row ordering without `ORDER BY`, and use `SingleOrDefaultAsync` for primary key lookups - Use `DefaultAzureCredential` with `ManagedIdentityClientId` for the database token provider via `UsePeriodicPasswordProvider` - Apply `UseSnakeCaseNamingConvention()` to SQLite test configuration and update all raw SQL across 55 test files to snake_case - Add `Plan` property to `Tenant` to denormalize the subscription plan, restrict `GetPaymentHistory` to owners only - Fix flaky Stripe webhook tests with `[Collection("StripeTests")]` **Infrastructure changes:** - Replace SQL Server Bicep modules with PostgreSQL Flexible Server using Entra ID-only authentication, Private Endpoint with Private DNS Zone on a dedicated `private-endpoints` subnet, and `Ssl Mode=VerifyFull` for certificate verification - Split VNet into two subnets: `container-apps` (delegated to `Microsoft.App/environments`) and `private-endpoints`, since delegated subnets cannot host private endpoints - Upgrade Container Apps from consumption-only to workload profiles with a Consumption profile, providing identical billing, higher per-app limits (4 vCPUs/8 GiB), and future dedicated compute support - Configure `pg_stat_statements` for query monitoring, `log_statement=mod` for audit logging with diagnostic logs routed to a storage account with 90-day lifecycle retention, and `wal_level=logical` for logical replication support - Chain server configuration changes after Private DNS Zone Group provisioning to prevent first-time deployment failures - Replace Bicep `dependsOn` with module output references for proper implicit dependencies and `--what-if` visibility - Move Entra ID admin provisioning to a post-deployment CLI step since PostgreSQL cannot provision admins with `passwordAuth: 'Disabled'` in a single Bicep deployment **CI/CD changes:** - Use `psql` with `sslmode=verify-full sslrootcert=system` and `-v ON_ERROR_STOP=1` (without `--single-transaction` to support `CREATE INDEX CONCURRENTLY`) for migration apply, and `Ssl Mode=VerifyFull` for `dotnet ef` connections - Open firewall once for all database permission grants instead of per-database - Add `set -e` to shell scripts and IP address validation in `firewall.sh` - Authenticate using the Entra admin group principal name as the PostgreSQL username with the service principal's access token - Rename security group from "SQL Admins" to "PostgreSQL Admins" and all related GitHub variables **Documentation changes:** - Update README files, legal documents (DPA, cross-references), and developer CLI to reference PostgreSQL - Update AI rules (`api-tests.md`, `domain-modeling.md`, `database-migrations.md`, `backend.md`, `repositories.md`) to reflect PostgreSQL conventions including snake_case examples, `HasColumnType("jsonb")` guidance, `CREATE INDEX CONCURRENTLY` support, and PostgreSQL ordering behavior ### Downstream projects **This is a breaking change.** The database provider changed from SQL Server to PostgreSQL. Delete existing SQL Server migrations and create a new initial migration using PostgreSQL conventions. There is no migration path to move data in a production system. See the updated [database-migrations rule file](/.claude/rules/backend/database-migrations.md) and the new [account initial migration](/application/account/Core/Database/Migrations/20260303023200_Initial.cs) for reference. Rename the Entra ID security groups in Azure from "SQL Admins" to "PostgreSQL Admins" (e.g., `SQL Admins - Staging - GitHubOrganisation/GitHubRepository` becomes `PostgreSQL Admins - Staging - GitHubOrganisation/GitHubRepository`). Then rename the following GitHub Actions variables: - `STAGING_SQL_ADMIN_OBJECT_ID` to `STAGING_POSTGRES_ADMIN_OBJECT_ID` - `PRODUCTION_SQL_ADMIN_OBJECT_ID` to `PRODUCTION_POSTGRES_ADMIN_OBJECT_ID` ### Optional: Enable Microsoft Defender for PostgreSQL For vulnerability assessments and security alerts, enable **Microsoft Defender for Open-Source Relational Databases** in your Azure subscription. This provides brute force detection, anomalous access alerts, and vulnerability scanning for PostgreSQL Flexible Server (~$15/month per server). Enable it in the Azure Portal under Microsoft Defender for Cloud > Environment settings > Defender plans, or via Azure CLI: ```bash az security pricing create --name OpenSourceRelationalDatabases --tier Standard ``` ### Checklist - [x] I have added tests, or done manual regression tests - [x] I have updated the documentation, if necessary
2 parents c3aa3c8 + 525016d commit f2e05d2

File tree

135 files changed

+2284
-8703
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

135 files changed

+2284
-8703
lines changed

.agent/workflows/modes/agentic-workflow.md

Lines changed: 0 additions & 478 deletions
This file was deleted.

.agent/workflows/modes/coordinator.md

Lines changed: 0 additions & 21 deletions
This file was deleted.

.agent/workflows/modes/tech-lead.md

Lines changed: 0 additions & 38 deletions
This file was deleted.

.agent/workflows/process/implement-end-to-end-tests.md

Lines changed: 0 additions & 228 deletions
This file was deleted.

0 commit comments

Comments
 (0)