Skip to content

feat(pj_datastore): in-place dataset replace primitives (replaceDatasetFrom) for reload#112

Closed
facontidavide wants to merge 2 commits into
mainfrom
feat/dataset-replace-on-reload
Closed

feat(pj_datastore): in-place dataset replace primitives (replaceDatasetFrom) for reload#112
facontidavide wants to merge 2 commits into
mainfrom
feat/dataset-replace-on-reload

Conversation

@facontidavide
Copy link
Copy Markdown
Contributor

Summary

Adds DataEngine::replaceDatasetFrom and ObjectStore::replaceDatasetFrom — primitives that swap a dataset's data in place while keeping its DatasetId / TopicId / ObjectTopicId stable. This lets a host (PlotJuggler 4) implement data reload as a true replace: refresh a dataset's data without orphaning anything keyed by those ids (plot curve keys, 2D object-dock bindings).

What & how

  • DataEngine::replaceDatasetFrom(staged, staged_id, primary_id) — match staged↔primary topics by name. Matched topics: clear the primary's chunks and move the staged topic's chunks in, re-stamping each TopicChunk.topic_id to the primary id (+ move the inline column layout). Staged-only topics: created under the primary dataset. Primary-only topics: retired — chunks cleared, id hidden from listTopics, but the TopicStorage kept so an in-flight reader pointer hits an empty deque rather than freed memory (the engine is append-only; this is the honest "logical removal").
  • ObjectStore::replaceDatasetFrom(...) — same on the object side, preserving the primary ObjectTopicId; returns the staged→primary id map so the host re-registers object parsers under stable ids.
  • Both reuse the existing cross-store move seams: a new private DataEngine::adoptChunksFrom now single-sources the chunk-deque move shared with flushTo; ObjectStore::eraseTopicLocked is shared by removeTopic and the replace path.
  • replace_result.hpp: DatasetReplaceResult / ObjectDatasetReplaceResult.

Tests

7 new unit tests (engine_integration_test, object_store_test): chunk move + topic_id rewrite, added/retired topics, id stability (reader sees new data under the same id); object entry move + ObjectTopicId preservation, removed/added topics. All pass; the existing flushTo tests confirm the adoptChunksFrom unification is safe.

Versioning / ABI

Additive API only (DataEngine/ObjectStore are pimpl; no vtable/struct/C-ABI changes) → a MINOR bump 0.5.1 → 0.6.0. pj_base/abi/baseline.abi is unchanged — the additions live in pj_datastore, outside the pj_base ABI canary.

Consumed by PlotJuggler/PJ4#124 (reload = in-place replace).

🤖 Generated with Claude Code

….6.0)

Add DataEngine::replaceDatasetFrom and ObjectStore::replaceDatasetFrom: swap a
dataset's data in place while keeping its DatasetId/TopicId/ObjectTopicId stable,
so a host reload can refresh data without orphaning curve keys or 2D-dock
bindings. Topics are matched by name; matched topics have their chunks/entries
replaced (each moved chunk's topic_id re-stamped to the primary id), staged-only
topics are created/registered under the primary dataset, and primary-only topics
are retired (scalars: chunks cleared + hidden from listTopics, storage kept so a
cached reader pointer hits an empty deque; objects: removed). Returns the
staged->primary id map so the host re-registers object parsers under stable ids.

- engine: replaceDatasetFrom + retired_topic_ids (listTopics filter); factor the
  chunk-deque move shared with flushTo into adoptChunksFrom (re-stamps topic_id).
- object_store: replaceDatasetFrom; factor find-and-erase into eraseTopicLocked.
- replace_result.hpp: DatasetReplaceResult / ObjectDatasetReplaceResult.
- tests: 7 new (engine + object store): chunk move/id rewrite, added/retired,
  id stability; entry move/id preserve, removed/added.

Additive API only (pimpl, no vtable/struct changes) -> MINOR bump 0.5.1 -> 0.6.0;
abi/baseline.abi unchanged (the additions live in pj_datastore, outside the
pj_base ABI canary).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@facontidavide
Copy link
Copy Markdown
Contributor Author

@copilot fix pre-commit

Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 2, 2026

@copilot fix pre-commit

Fixed in 0664a92 by applying the required clang-format pre-commit formatting updates. Pre-commit now passes.

@facontidavide
Copy link
Copy Markdown
Contributor Author

Superseded by the pj_datastore extraction (#113/#114). The datastore primitives in this PR were re-landed in the app repo as PlotJuggler/PJ4#124 (identical content — all 7 source/test files are byte-for-byte the same); closing as obsolete. The only parts not carried over were the SDK-local 0.5.1 → 0.6.0 version bump in CMakeLists.txt/conanfile.py, which were superseded by the actual 0.6.0 release in #113/#114 and the plotjuggler_core → plotjuggler_sdk rename.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants