Summary
_create_sqlite_engine() (openhands/automation/db.py:151) never forwards the
db_pool_* settings to create_async_engine:
engine = create_async_engine(
db_url,
# SQLite-specific settings
connect_args={"check_same_thread": False},
# No pooling for SQLite - it handles this internally
pool_pre_ping=True,
)
The Postgres branch (db.py:136-146) and the GCP Cloud SQL branch
(db.py:198-206) both pass pool_size, max_overflow and pool_timeout. The
SQLite branch passes none of them, so db_pool_size (10), db_max_overflow (5)
and db_pool_timeout (30) from ServiceSettings (config.py:480-499) are
silently ignored on SQLite deployments — and so is the AUTOMATION_DB_POOL_SIZE
environment variable that the AUTOMATION_ env prefix implies.
The comment on line 167 — "No pooling for SQLite - it handles this internally" —
is not correct for file-backed aiosqlite.
Verification
Against the versions actually deployed (openhands-automation 1.8.0,
SQLAlchemy 2.0.52):
>>> from sqlalchemy.ext.asyncio import create_async_engine
>>> e = create_async_engine("sqlite+aiosqlite:////tmp/probe.db",
... connect_args={"check_same_thread": False},
... pool_pre_ping=True)
>>> type(e.pool).__name__, e.pool.size(), e.pool._max_overflow, e.pool._timeout
('AsyncAdaptedQueuePool', 5, 10, 30.0)
SQLAlchemy hands file-backed sqlite+aiosqlite a real AsyncAdaptedQueuePool
with its own defaults (5 + 10, 30s), not a NullPool. Only :memory: gets
pool-less treatment.
Impact
This is an active defect, not a latent one. On the OSS automation VM
(dev-oss-automation-vm, SQLite backend) the resulting exhaustion fires
regularly and surfaces as 500s from the automation API:
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached,
connection timed out, timeout 30.00
722 occurrences in 7 days, in bursts that line up exactly with the minutes when
several 5-minute-cron automations fire simultaneously (Aug 17 ~346, Aug 19 ~203,
Aug 21 ~95). The 5/10 in the message are SQLAlchemy's defaults, confirming
the configured 10/5 never took effect.
The operator-facing part is the sharpest edge: db.py:143 justifies the
fail-fast pool_timeout as
Fail fast if pool is exhausted rather than waiting indefinitely. This surfaces
pool exhaustion issues as errors instead of timeouts, making it easier to
diagnose and fix (e.g., by increasing pool_size).
but on the one path that actually hits this, increasing pool_size does nothing.
Proposed fix
Either honour the settings:
engine = create_async_engine(
db_url,
connect_args={"check_same_thread": False},
pool_size=settings.db_pool_size,
max_overflow=settings.db_max_overflow,
pool_timeout=settings.db_pool_timeout,
pool_pre_ping=True,
)
(_create_sqlite_engine currently takes only db_url, so it would need the
settings object.)
Or make the existing comment true:
from sqlalchemy.pool import NullPool
engine = create_async_engine(
db_url,
connect_args={"check_same_thread": False},
poolclass=NullPool,
pool_pre_ping=True,
)
Either way, the current state — settings that exist, are documented by their env
prefix, and are silently dropped — is the worst of the three.
Relation to other issues
Distinct from #290, which covers Postgres pool size multiplying across
replicas × workers and is explicitly filed as a latent scaling hazard
("Nothing has broken because of this yet"). This one is SQLite-only, the
settings are ignored outright rather than mis-scaled, and it is causing
failures today. Likely belongs under the #291 hardening tracker.
Summary
_create_sqlite_engine()(openhands/automation/db.py:151) never forwards thedb_pool_*settings tocreate_async_engine:The Postgres branch (
db.py:136-146) and the GCP Cloud SQL branch(
db.py:198-206) both passpool_size,max_overflowandpool_timeout. TheSQLite branch passes none of them, so
db_pool_size(10),db_max_overflow(5)and
db_pool_timeout(30) fromServiceSettings(config.py:480-499) aresilently ignored on SQLite deployments — and so is the
AUTOMATION_DB_POOL_SIZEenvironment variable that the
AUTOMATION_env prefix implies.The comment on line 167 — "No pooling for SQLite - it handles this internally" —
is not correct for file-backed
aiosqlite.Verification
Against the versions actually deployed (
openhands-automation1.8.0,SQLAlchemy 2.0.52):
SQLAlchemy hands file-backed
sqlite+aiosqlitea realAsyncAdaptedQueuePoolwith its own defaults (5 + 10, 30s), not a
NullPool. Only:memory:getspool-less treatment.
Impact
This is an active defect, not a latent one. On the OSS automation VM
(
dev-oss-automation-vm, SQLite backend) the resulting exhaustion firesregularly and surfaces as 500s from the automation API:
722 occurrences in 7 days, in bursts that line up exactly with the minutes when
several 5-minute-cron automations fire simultaneously (Aug 17 ~346, Aug 19 ~203,
Aug 21 ~95). The
5/10in the message are SQLAlchemy's defaults, confirmingthe configured 10/5 never took effect.
The operator-facing part is the sharpest edge:
db.py:143justifies thefail-fast
pool_timeoutasbut on the one path that actually hits this, increasing
pool_sizedoes nothing.Proposed fix
Either honour the settings:
(
_create_sqlite_enginecurrently takes onlydb_url, so it would need thesettingsobject.)Or make the existing comment true:
Either way, the current state — settings that exist, are documented by their env
prefix, and are silently dropped — is the worst of the three.
Relation to other issues
Distinct from #290, which covers Postgres pool size multiplying across
replicas × workers and is explicitly filed as a latent scaling hazard
("Nothing has broken because of this yet"). This one is SQLite-only, the
settings are ignored outright rather than mis-scaled, and it is causing
failures today. Likely belongs under the #291 hardening tracker.