[timescaledb] Initial contribution

[ulbi]

[timescaledb] Initial contribution

Description

This PR adds a new persistence service for TimescaleDB, a time-series extension for PostgreSQL.

Classification: Novel Addition

While the existing JDBC persistence service can connect to TimescaleDB via a PostgreSQL driver, it treats it as a plain relational database and does not use any of its time-series capabilities. This new service is purpose-built for TimescaleDB and takes advantage of its native features to provide better performance, lower storage usage, and more control over how historical data is managed.

What's new for users

• Hypertable-backed storage — item states are written into a single TimescaleDB hypertable, which handles automatic time-based partitioning and enables fast range queries.
• In-place downsampling — users can configure per-item downsampling via item metadata (timescaledb namespace). Raw data is aggregated (AVG, MAX, MIN, SUM) and replaced in-place at a configurable interval (e.g. 15m, 1h, 1d). openHAB reads raw and downsampled data from the same table without any query changes needed.
• Retention policies — both a global retention threshold (via service config) and per-item retention (via metadata) are supported. An item can have retention-only configured (no aggregation, just data expiry) by using a single space as the metadata value.
• Automatic compression — when configured, TimescaleDB compresses older chunks automatically, reducing disk usage significantly for long-running installations.
• Units stored per measurement — QuantityType items have their unit stored alongside the numeric value, making direct SQL and Grafana queries self-describing.
• Grafana-friendly — the schema is a plain PostgreSQL table that Grafana's built-in PostgreSQL data source can query directly without any additional setup.
• Console command — a Karaf console command lets users trigger the downsampling job manually for testing or recovery.

How it compares to JDBC persistence

Feature	JDBC Persistence	TimescaleDB Persistence
Hypertables	No	Yes
In-place downsampling	No	Yes
Per-item aggregation config	No	Yes (item metadata)
Automatic compression	No	Yes
Retention policies	No	Yes (global + per-item)
Unit stored per measurement	No	Yes
Schema	One table per item	Single hypertable
Multiple DB backends	Yes	No (TimescaleDB only)

Full persistence service support

The add-on implements both QueryablePersistenceService and ModifiablePersistenceService.

Implemented query operations include: historicState, averageSince, sumSince, minSince, maxSince, countSince, getAllStatesBetween, removeAllStatesBetween.

In addition, write and delete operations used by openHAB persistence are supported (store, remove).

Schema setup

The service creates all required tables (items hypertable + item_meta lookup table) automatically on startup. No manual DDL is needed beyond creating the database user and enabling the TimescaleDB extension.

---

Testing

Unit and integration tests

183 tests across 9 test classes, all passing as of 2026-03-13.

Test class	Tests	Result
BundleManifestTest	2	PASS
TimescaleDBMapperTest	47	PASS
TimescaleDBMetadataServiceTest	26	PASS
TimescaleDBQueryTest	14	PASS
TimescaleDBDownsampleJobTest	8	PASS
TimescaleDBDownsampleSemanticsTest	29	PASS
TimescaleDBSchemaTest	7	PASS
TimescaleDBPersistenceServiceTest	20	PASS
TimescaleDBContainerTest	30	PASS
Total	183	0 failures

TimescaleDBContainerTest spins up a real TimescaleDB instance via Testcontainers and covers the full persistence lifecycle including schema init, write, query, downsampling, and retention.

Performance tests

TimescaleDBPerformanceIT covers 12 load scenarios: single write latency, sustained 600 writes/s, cold-start burst, concurrent read/write, downsampling job runtime, connection pool saturation, and an opt-in 18-month bulk scenario. These tests are tagged @Tag("performance") and require an external TimescaleDB instance. They are not part of the standard CI run. See PERFORMANCE_TESTS.md for details and how to run them.

How to test manually

1. Start a TimescaleDB instance:

   docker run -d --name timescaledb -p 5432:5432 \
     -e POSTGRES_PASSWORD=openhab \
     timescale/timescaledb:latest-pg17

2. Install the bundle from the build artifact linked above.
3. Configure the service in services/org.openhab.persistence.timescaledb.cfg (or via the UI under Settings → Add-ons → TimescaleDB).
4. Add a timescaledb.persist file to your persistence folder.
5. Optionally add timescaledb metadata to items to enable per-item downsampling or retention.

---

Documentation

The README covers: prerequisites, schema layout, state type mapping, all configuration properties, the .persist format, per-item metadata syntax with examples for .items files and the UI, how in-place downsampling works step by step, the full query mapping table, compression and retention setup, a Grafana example query, and a comparison table with JDBC persistence.

---

Notes

• No breaking changes to existing services.
• Coding guidelines followed.
• Static code analysis checked for the bundle.
• Work signed-off.

Suggested label: enhancement

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/new-timescaledb-persistence-service-for-openhab/168837/1

[Copilot]

Pull request overview

This PR introduces a new openHAB persistence add-on that targets TimescaleDB specifically, adding hypertable-backed storage plus built-in downsampling, retention, and optional compression support.

Changes:

• Adds the new org.openhab.persistence.timescaledb bundle/module and wires it into the reactor + BOM.
• Implements schema initialization/migration, JDBC query/insert/delete operations, a daily downsampling/retention job, and a Karaf console command.
• Adds extensive unit + integration tests (mocked JDBC + Testcontainers) and bundle documentation.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
bundles/pom.xml	Adds the TimescaleDB persistence module to the bundles reactor.
bom/openhab-addons/pom.xml	Adds the new bundle to the add-ons BOM.
CODEOWNERS	Assigns code ownership for the new bundle.
bundles/org.openhab.persistence.timescaledb/pom.xml	Declares bundle dependencies and test setup.
bundles/org.openhab.persistence.timescaledb/src/main/feature/feature.xml	Declares the Karaf feature and required runtime bundles (PostgreSQL driver, HikariCP, the add-on bundle).
bundles/org.openhab.persistence.timescaledb/src/main/resources/OH-INF/config/timescaledb.xml	Adds UI/service configuration description for the persistence service.
bundles/org.openhab.persistence.timescaledb/src/main/resources/OH-INF/addon/addon.xml	Registers the add-on in the openHAB UI.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBPersistenceService.java	Implements the OSGi persistence service lifecycle, pooling, scheduling, store/query/remove.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBSchema.java	Creates/migrates the DB schema (tables, hypertable, constraints, policies).
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBQuery.java	Provides SQL for insert/query/delete and item_id lookup helpers.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMapper.java	Maps openHAB state types to/from persisted DB row format.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBHistoricItem.java	Implements HistoricItem for query results.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMetadataService.java	Parses per-item metadata for downsampling/retention configuration.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBDownsampleJob.java	Implements the scheduled downsampling + retention job logic.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBConsoleCommandExtension.java	Adds Karaf console command to run downsampling on-demand.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/DownsampleConfig.java	Holds validated per-item downsampling/retention configuration + interval allowlist.
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/AggregationFunction.java	Defines supported aggregation functions and their SQL names.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/BundleManifestTest.java	Validates OSGi manifest imports and addon.xml presence.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBSchemaTest.java	Unit tests for schema DDL generation/migration using mocks.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBQueryTest.java	Unit tests for query/insert/remove SQL construction and result mapping.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBPersistenceServiceTest.java	Unit tests for service store/query/remove and lifecycle behavior with mocked datasource.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMetadataServiceTest.java	Unit tests for metadata parsing and interval allowlist enforcement.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMapperTest.java	Unit tests for state type mapping to/from DB representation.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBDownsampleJobTest.java	Unit tests for downsampling job SQL/transaction behavior with mocks.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBDownsampleSemanticsTest.java	Unit tests for ON/OFF and OPEN/CLOSED aggregation semantics.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBContainerTest.java	Integration tests using Testcontainers for end-to-end persistence flows.
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBExternalIT.java	Optional external integration test against a user-provided TimescaleDB instance.
bundles/org.openhab.persistence.timescaledb/README.md	End-user documentation for setup, config, metadata syntax, and Grafana querying.
bundles/org.openhab.persistence.timescaledb/PERFORMANCE_TESTS.md	Performance/scaling test plan and how to run performance tests.
bundles/org.openhab.persistence.timescaledb/AGENTS.md	Developer guide documenting architecture, schema, metadata format, and testing strategy.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ulbi]

@copilot thanks for the review - check again...

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 6 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ulbi]

@copilot @wborn Thanks for the review. Hopefully we catched most if it:

Fixed

• query() and remove() now resolve item_id from the DB on cache miss and populate the cache, so both work correctly after a service restart.
• deactivate() now nulls downsampleJobInstance.
• chunkInterval is no longer string-formatted into DDL — SQL_CREATE_HYPERTABLE uses a PreparedStatement with ?::INTERVAL.
• Negative retainRawDays and retentionDays values are rejected with a warning in TimescaleDBMetadataService.
• DownsampleConfig.retentionOnly() now throws IllegalArgumentException for values <= 0.
• Added a warning during activation when compressionAfterDays > 0, advising that per-item retainRawDays must be less than compressionAfterDays to avoid writing into already-compressed chunks.
• README corrected: last(unit, time) → MAX(unit).
• Javadoc for SQL_INSERT_AGGREGATED_TEMPLATE corrected to reflect one placeholder, not two.
• Removed the unnecessary -javaagent argLine from pom.xml — all mocked types are interfaces, Mockito 5 needs no agent for that. Note: mockito-inline as a separate dependency no longer exists in Mockito 5, it was merged into mockito-core.

Not changed

• The two @Activate annotations (constructor + method) are intentional: OSGi DS 1.4 supports @Activate on a constructor for reference injection alongside an @Activate method for the configuration callback.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ulbi]

@wborn
The remaining open comments:

• Dual @Activate: Intentional. OSGi DS 1.4 explicitly supports @Activate
  on a constructor for reference injection alongside an @Activate method for the
  configuration callback. This pattern is used throughout the DS 1.4 spec.

• Scheduling drift: The job is a daily maintenance task; sub-minute alignment
  to midnight is not a requirement. scheduleWithFixedDelay with a 24h period is
  the standard approach for this in openHAB persistence services.

• Testcontainers artifact IDs: The IDs testcontainers-postgresql and
  testcontainers-junit-jupiter are correct — Testcontainers 2.x changed its
  artifact naming convention from the 1.x line.

[wborn]

Thanks for addressing all the comments @ulbi. 👍 Sometimes Copilot hallucinates a bit. 😵‍💫 Often it does add useful review comments. 😉

[ulbi]

@wborn I know - and it has uncovered some very relevant stuff claude did oversee here. I think the critical things are adressed know. Hope the bundle will make it in the addons. If there is something else required, let me know ;-)

[lsiepel]

Thanks, i did a partial and quick review, mainly targeting the metadata and documentation files.

[lsiepel]

Remove this file

[ulbi]

@lsiepel Please be more specific why this should be removed? This makes live much easier when working with agents on this bundle.

[lsiepel]

We have a strict policy about bundle structure and files added to source control. This file is not part of it

[ulbi]

@lsiepel Thanks for your very strict answer, you may think about further evolving the policies, but as you insist on this, I will move the file in my own storage and keep it there.

[jlaur]

@lsiepel - I think we can allow AI agent instructions at binding level to give agents more context and thereby improve the quality of their output. We also have an agent file here by now: https://github.com/openhab/openhab-addons/blob/main/AGENTS.md

[jlaur]

Have to publish review to get my comment out, since I left one yesterday as well. 🙂

[jlaur]

@lsiepel - I think we can allow AI agent instructions at binding level to give agents more context and thereby improve the quality of their output. We also have an agent file here by now: https://github.com/openhab/openhab-addons/blob/main/AGENTS.md

[ulbi]

@lsiepel @jlaur Thanks a lot for your reviews, I resolved mostly all of your comments - but the AGENTS.md I left in there. Please decide, whether you want to further support agentic coding in openhab or not (at least @wborn leaves the first review to copilot ;-) )