Skip to content

feat: multi-schema support, auto-container, CI workflows, and docs#44

Open
supunappri99 wants to merge 53 commits into
mainfrom
development
Open

feat: multi-schema support, auto-container, CI workflows, and docs#44
supunappri99 wants to merge 53 commits into
mainfrom
development

Conversation

@supunappri99
Copy link
Copy Markdown
Collaborator

feat: multi-schema support, auto-container, CI workflows, and docs

Branch: developmentmain

Summary

  • Adds full multi-schema PostgreSQL support: schemas: string[] config, per-schema db/schema/<name>/ directories, per-schema pgschema plan runs with intermediate apply for cross-schema resolution, and combined migration output
  • Introduces auto-container mode (Docker-managed local Postgres), config split into postkit.config.json (committed) + postkit.secrets.json (gitignored), postkit db schema add command, and separate db/infra/ directory for DB-level infrastructure
  • Adds GitHub Actions PR checks workflow, updates release workflow with gated unit + E2E test jobs, overhauls documentation site with new pages, blog posts, and hero images, and fixes TypeScript strict-mode errors across CLI test utilities

Changes

  • cli/src/modules/db/types/schemas: string[] replaces schema: string; session planFiles/schemaFingerprints become Record<string, string | null> maps
  • cli/src/modules/db/utils/db-config.tsgetPlanFilePath(schemaName), getGeneratedSchemaPath(schemaName), getInfraPath() per-schema path helpers
  • cli/src/modules/db/commands/plan.ts — loops over config.schemas, runs pgschema per schema, intermediate apply between schemas for cross-schema reference resolution
  • cli/src/modules/db/commands/apply.ts — iterates planFiles record, combines wrapped SQL from all schemas into one migration file
  • cli/src/modules/db/commands/schema.ts — new postkit db schema add <name> command via schema-scaffold.ts service
  • cli/src/modules/db/services/container.ts — new: auto-starts postgres:{version}-alpine Docker container when localDbUrl is empty; used by start, deploy, import
  • cli/src/modules/db/services/schema-generator.ts — per-schema reads from db/schema/<name>/, writes schema_<name>.sql, fingerprints per schema
  • cli/src/modules/db/services/infra-generator.ts — reads from config.infraPath (db/infra/) instead of db/schema/infra/
  • cli/src/commands/init.ts — scaffolds schemas: ["public"], infraPath: "db/infra", gitignore entries updated to plan_*.sql / schema_*.sql
  • cli/src/common/config.tsPostkitPublicConfig / PostkitSecrets split; deepMerge for secrets overlay; auto-migration from legacy remoteDbUrl
  • .github/workflows/pr-checks.yml — new: cli-buildcli-unit-testscli-e2e-tests (Docker) on PRs to main/development
  • .github/workflows/release.ymlunit-tests and e2e-tests jobs; release job gated on both
  • agent/skills/ — all four PostKit skills updated for multi-schema layout and correct config structure
  • .claude/skills/create-pr/SKILL.md — fixed to generate description only (no push/create); reads template first
  • docs/ — new blog posts, hero images via @docusaurus/plugin-ideal-image, updated all module docs, new cross-schema-migrations.md, schema.md command page, agent skills overview
  • cli/test/ — TypeScript strict-mode fixes; new container.test.ts, schema-scaffold.test.ts; E2E case-6-multi-schema-full-flow.test.ts and auto-container.test.ts

Type of Change

  • feat: New feature

Test Plan

  • Unit tests pass (npm run test)
  • E2E tests pass (npm run test:e2e) (if applicable)
  • Build succeeds (npm run build)
  • Manually tested: multi-schema plan/apply/commit flow with public + app schemas on local Postgres; auto-container start/abort; postkit db schema add; docs site builds and hero images render correctly in light/dark mode

Breaking Changes

  • Breaking changes — existing projects must migrate before upgrading:

    1. Config key renamedpostkit.config.json

    - "schema": "public"
    + "schemas": ["public"]
    + "infraPath": "db/infra"

    2. Directory structure change — infra SQL moves out of the schema folder

    Before:  db/schema/infra/   (roles, extensions, CREATE SCHEMA)
    After:   db/infra/          (same files, new root-level location)
    

    3. Schema SQL must be in a named subdirectory

    Before:  db/schema/tables/   db/schema/functions/   ...
    After:   db/schema/public/tables/   db/schema/public/functions/   ...
    

    4. Session state shape changed — existing .postkit/db/session.json files with the old planFile/schemaFingerprint scalar fields will be treated as having no pending plan. Run postkit db abort before upgrading to clear any active session.

    5. Ephemeral file names changed.postkit/db/plan.sqlplan_<schema>.sql, schema.sqlschema_<schema>.sql. Update .gitignore entries (or re-run postkit init).

default and others added 30 commits April 25, 2026 19:02
…actor configuration into committed and ignored secrets files
…m postkit.config.json to postkit.secrets.json in E2E tests
…lizing utility functions and standardizing deployment steps.
…iguration storage, and internal service functions
…crets

refactor: split config/secrets, auto-container mode, dedup utilities
…ingle to array-based schema configuration and per-schema file handling
…architectures and update import documentation
…-support

feat: multi-schema support, docs overhaul, and CI pipelines
@supunappri99 supunappri99 requested a review from Yasirunet May 13, 2026 09:41
Copy link
Copy Markdown

@nipunappri nipunappri Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential Issues

1. Intermediate apply error handling in cli/src/modules/db/commands/plan.ts:154

The intermediate apply for cross-schema resolution catches errors but only warns:

} catch (err) {
  spinner.warn(`Intermediate apply for "${schemaName}" failed (non-fatal): ${err instanceof Error ? err.message : String(err)}`);
}

If a schema creates a table that another schema references, and the intermediate apply fails silently, the subsequent pgschema plan for the dependent schema will likely fail anyway with a more obscure error. Consider making this fatal for certain error types, or at least logging more details about what errors are safe to ignore.


2. Schema fingerprint validation could block rapid iteration

The apply command validates that schema files haven't changed since plan was generated:

if (current !== stored) {
  throw new PostkitError(
    `Schema files for "${schemaName}" have changed since the plan was generated.`,
    'Run "postkit db plan" again to regenerate the plan.',
  );
}

This is good for safety but could be frustrating during active development when making rapid changes. Consider adding a --skip-fingerprint-check flag for development workflows, or make the error message more actionable (show what changed).


3. Container cleanup could be incomplete if process is killed

In startSessionContainer, if the process is killed before stopSessionContainer is called, the container remains running:

export async function stopSessionContainer(containerID: string): Promise<void> {
  await runCommand(`docker stop ${containerID}`);
  await runCommand(`docker rm ${containerID}`);
}

This is called from abort and deploy, but if the CLI process crashes (SIGKILL), containers leak. Consider a cleanup function that runs on process exit signals.


4. Free port scan has no upper bound retry

In findFreePort, if all ports 15432-15532 are taken:

for (let port = start; port <= end; port++) {
  if (await isPortFree(port)) return port;
}
throw new Error(`No free port found between ${start} and ${end}.`);

The error is thrown without any suggestion to the user. Could the error message include a hint to check for leftover PostKit containers? e.g., docker ps | grep postkit-session


Questions

1. Why is intermediate apply for cross-schema resolution non-fatal?

In plan.ts, when applying a schema's plan to enable cross-schema references for subsequent schemas:

spinner.start(`Applying "${schemaName}" plan to local DB for cross-schema resolution...`);
try {
  await applyPlanToLocalDb(session.localDbUrl, planFilePath);
  spinner.succeed(...);
} catch (err) {
  spinner.warn(`Intermediate apply for "${schemaName}" failed (non-fatal): ...`);
}

Is this non-fatal behavior intentional? What types of errors are expected to be safe to ignore here?


2. Should empty schema directories be allowed?

In plan.ts, schemas with no directory are skipped with a warning:

if (!existsSync(schemaDir)) {
  logger.warn(`Schema "${schemaName}" has no directory at ${schemaDir} — skipping. Run "postkit db schema add ${schemaName}" to scaffold it.`);
  planFiles[schemaName] = null;
  schemaFingerprints[schemaName] = null;
  continue;
}

Is this the intended UX? Should users be required to run postkit db schema add before a schema can be added to config, or is this flexible skip-by-warn acceptable?


3. Any plans for automated config migration?

The breaking changes (5 of them) require manual migration. Is there any plan to add a postkit db migrate-config command to automate this, or is the manual approach preferred going forward?


4. Is SHOW server_version_num reliable across all cloud providers?

For container version detection:

const pgVersion = await getRemotePgMajorVersion(remoteUrl);

This uses SHOW server_version_num. Are there any known issues with this on managed Postgres services (RDS, Cloud SQL, Neon, Supabase, etc.) that might report different version formats?

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 15, 2026

Deploying postkit with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2324bd7
Status: ✅  Deploy successful!
Preview URL: https://1ce9fd7f.postkit.pages.dev

View logs

@supunappri99
Copy link
Copy Markdown
Collaborator Author

⏺ Issue 1 — No Fix Needed

Cross-schema migration is not a supported feature of the plan command.
The intermediate apply step exists only to enable cross-schema reference resolution, which is outside the intended scope.

As a result, a failure during this step has no impact on supported workflows.
The reviewer's concern does not apply in this context.

⏺ Issue 2 — No Fix Needed

The fingerprint check is a valid and necessary safety mechanism.

The existing error message already provides clear guidance:

Run postkit db plan again

Re-running the plan command takes only a few seconds and is the correct and safest action.
There is no practical scenario where bypassing this check is safer than regenerating the plan.

Introducing a --skip-fingerprint-check flag would:

  • Expand the API surface unnecessarily
  • Increase testing and maintenance overhead
  • Allow users to apply a plan against mismatched schema files

This friction is intentional and helps prevent unsafe operations.


Answers for Questions

1. Why is intermediate apply for cross-schema resolution non-fatal?

The non-fatal behavior is intentional. The intermediate apply strips structural SQL only (CREATE/ALTER/DROP — no policies, no grants) and runs with ON_ERROR_STOP=1.

Error types that are safe to ignore:

  • "already exists" — the object was applied to the local DB in a previous session run, so it is already present for cross-schema resolution. This is the most common case.
  • Policy/grant leakage — the line-level filter in plan.ts:169-183 is a simple startsWith check, so multi-line statements can occasionally leak through and fail harmlessly without affecting structural resolution.

If the apply fails for a genuinely structural reason (e.g. a missing type), the dependent schema's pgschema run will fail anyway with a direct, clear error. The non-fatal warn avoids false-fatal errors from the "already exists" class while still surfacing that something went wrong.

Note: Cross-schema migration is not a currently supported feature. The intermediate apply block is aspirational code and its failure mode has no impact on supported workflows.


2. Should empty schema directories be allowed?

The skip-by-warn behavior is intentional and the correct UX for the current design.

postkit.config.json is committed and shared with the team, but individual developers may be at different stages of setup. A hard error on a missing directory would break postkit db plan for any developer who hasn't yet scaffolded a schema that a teammate added to config. The warn + continue approach lets everyone keep working while clearly surfacing that action is needed.

Running postkit db schema add is not a prerequisite gate — it is a convenience command. The flexible skip-by-warn is the intended behavior.


3. Any plans for automated config migration?

Automated config migration already exists. common/config.ts:182-211 runs on every loadPostkitConfig() call and silently migrates:

  • remoteDbUrlremotes.default
  • environments.*remotes.*

The migrated result is written back to postkit.secrets.json in place, so the transition is transparent to the user.

A separate postkit db migrate-config command would only be needed if breaking changes are made to postkit.config.json (the committed file). Currently all breaking format changes are in the secrets file, which the runtime auto-migration handles. No explicit command is planned unless committed config structure changes.


4. Is SHOW server_version_num reliable across all cloud providers?

Yes. server_version_num is a core PostgreSQL GUC defined in the PostgreSQL source. It cannot be overridden by providers and is not provider-specific.

Every managed Postgres service (RDS, Aurora, Cloud SQL, Neon, Supabase) runs real PostgreSQL and exposes this value correctly.

Format: Always a zero-padded 6-digit integer — XXYYZZ where XX = major, YY = minor, ZZ = patch.

Example value Meaning
160003 PostgreSQL 16.3
150008 PostgreSQL 15.8
140012 PostgreSQL 14.12

The implementation at database.ts:148 extracts the major version with Math.floor(num / 10000), which is correct for this format. No known provider deviates from it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants