Skip to content

fix(db): use NullPool for SQLite to eliminate silent connection pooling (#347) - #391

Open
santhiprakash wants to merge 3 commits into
OpenHands:mainfrom
santhiprakash:fix/sqlite-nullpool
Open

fix(db): use NullPool for SQLite to eliminate silent connection pooling (#347)#391
santhiprakash wants to merge 3 commits into
OpenHands:mainfrom
santhiprakash:fix/sqlite-nullpool

Conversation

@santhiprakash

Copy link
Copy Markdown

Problem

_create_sqlite_engine() claimed "No pooling for SQLite - it handles this internally" but passed pool_pre_ping=True without specifying a poolclass. SQLAlchemy therefore applied its default AsyncAdaptedQueuePool (5 connections, 10 overflow, 30 s timeout). Under concurrent fan-out (e.g. the Automation view fetching runs for 21 automations), this pool silently exhausted and produced HTTP 500s on 6 of 21 concurrent requests (#347).

The db_pool_size, db_max_overflow, and db_pool_timeout config values were also silently ignored by the SQLite path.

Root cause

create_async_engine() without an explicit poolclass uses the default AsyncAdaptedQueuePool regardless of the comment claiming otherwise. The misleading pool_pre_ping=True reinforced the wrong mental model.

Fix

The N+1 fan-out problem (separate batch-read endpoint) is a separate fix tracked in the same issue; this PR addresses only the pool configuration bug.

Verification

uv run pytest tests/ -q --ignore=tests/integration
→ 1457 passed, 0 failed

uv run pre-commit run --files openhands/automation/db.py tests/test_db.py
→ ruff format, ruff lint, pycodestyle, pyright: all Passed

Notes / Risks

HUMAN: This PR was authored with AI assistance.

…ng (OpenHands#347)

- Problem: _create_sqlite_engine() claimed "No pooling for SQLite" but
  passed pool_pre_ping=True without specifying a poolclass, so SQLAlchemy
  applied its default AsyncAdaptedQueuePool (5 connections, 10 overflow,
  30s timeout). Under concurrent fan-out this caused HTTP 500s when
  the pool exhausted.

- Fix: pass poolclass=NullPool to disable SQLAlchemy-level pooling
  entirely, matching the documented intent. Remove pool_pre_ping (no-op
  with NullPool). Update the docstring to explain why.

- Verification: uv run pytest tests/ -q (1457 passed, 0 failed).
  Pre-commit clean (ruff format, ruff lint, pycodestyle, pyright).
@github-actions github-actions Bot added the type: fix A bug fix label Aug 26, 2026
@santhiprakash

Copy link
Copy Markdown
Author

Cross-linking #382 since it touches the same _create_sqlite_engine() path with the opposite direction on the same bug: it honors db_pool_size / db_max_overflow / db_pool_timeout for file-backed SQLite, while this PR disables SQLAlchemy-level pooling entirely.

Trade-off summary for whoever reviews first (defaults verified against the pinned SQLAlchemy 2.0.49):

  • The failure mode in Run summaries are only readable one automation at a time, and the SQLite engine's undeclared pool turns the resulting fan-out into 30s stalls + HTTP 500 #347 was pool exhaustion under fan-out (6/21 concurrent requests → 500 after ~30 s). Honoring the configured limits on SQLite keeps that failure mode, just with configurable numbers; NullPool removes it and matches the function's documented intent ("no pooling for SQLite").
  • In fix(db): honor SQLite pool settings #382's favor: it preserves SQLAlchemy's StaticPool default for :memory: URLs, where a shared connection is required (with NullPool, each new connection is a fresh empty database). This PR's unconditional NullPool changes that edge too — not reachable in production today (db_url defaults to empty/Postgres, and local mode uses a file path), but it's a fair difference. Happy to add a :memory: guard here if that direction is preferred.
  • Whichever merges first, the other needs a rework of the same hunk — glad to rebase this onto whichever direction maintainers pick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant