Skip to content

feat(registry): add raw registry versions dump to JSON#1763

Merged
sasa-tomic merged 7 commits intomainfrom
feat/registry-dump-all-versions
Oct 20, 2025
Merged

feat(registry): add raw registry versions dump to JSON#1763
sasa-tomic merged 7 commits intomainfrom
feat/registry-dump-all-versions

Conversation

@sasa-tomic
Copy link
Member

@sasa-tomic sasa-tomic commented Oct 13, 2025

Motivation

  • Enable exporting registry history as JSON for precise diffs.

Solution

  • Adds --dump-versions to dre registry.
  • Introduces direct reading of local registry pb files, decode records, and emitting a flat JSON array of {version, key, value}. The existing registry API was not sufficient for this requirement since it (ic_registry_local_registry::LocalRegistry, FakeRegistryClient, etc.) provides snapshots at a version (latest or specified) and key-family APIs, but not "list mutations per version" like the .pb changelog files store does. So it was necessary to read the raw .pb files directly.
  • Supports Python-style negative indices for ranges and base64-encodes unknown bytes; decodes common protobuf records when possible. This allows queries like: give me the last 5 records --dump-versions -5 or give me the first 5 records --dump-versions 0 5

Details

  • Looks up local_registry dirs under dre-store/local_registry for current network; honors DRE_LOCAL_REGISTRY_DIR_OVERRIDE for tests.
  • New unit tests under rs/cli/src/unit_tests/registry_versions.rs validating per-version dumps and range.

Meta

Example usage to get the last 5 registry versions (changes):

❯ dre registry --dump-versions -5
 2025-10-20T11:27:03.436Z INFO  dre > Running version 0.6.7-a012c0f9
 2025-10-20T11:27:03.460Z INFO  dre::store > Using local registry path for network mainnet: /home/sat/.cache/dre-store/local_registry/mainnet
[
  {
    "version": 54100,
    "key": "blessed_replica_versions",
    "value": {
      "blessed_version_ids": [
        "45657852c1eca6728ff313808db29b47c862ad13",
        "285897dae3a2cb60c50d9d8dd7327f18c3c372b9",
        "286295f0397521cd59ca7792df446c57379204fa"
      ]
    }
  },
  {
    "version": 54100,
    "key": "replica_version_206b61a8616bc93d36d6a014e5cc8edf1ba256ae",
    "value": null
  },
[...]

### Motivation
- Add ability to dump raw registry versions/records to JSON for precise diffs and debugging.

### Solution
- Extend dre registry with --dump-version and --dump-version-range options to emit a flat JSON array of records (version, key, value).
- Implement dump_versions_json to read local registry data, decode known keys via protobuf schemas, and base64-encode unknown bytes; supports Python-style version indexing including negatives.
- Add unit tests that exercise single-version and range-dump behavior using a test local registry dir override.

### Details
- Output shape is a flat array of { version, key, value } per record.
- Version selection supports single version, explicit range, and default 0 to -1 (including negative indices).
- Tests cover single-version dumps and range dumps using DRE_LOCAL_REGISTRY_DIR_OVERRIDE.

### Meta
updated tests accordingly
@sasa-tomic sasa-tomic self-assigned this Oct 13, 2025
@sasa-tomic sasa-tomic marked this pull request as ready for review October 13, 2025 12:56
@sasa-tomic sasa-tomic requested a review from a team as a code owner October 13, 2025 12:56
@cursor
Copy link

cursor bot commented Oct 13, 2025

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on November 1.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@sasa-tomic sasa-tomic enabled auto-merge (squash) October 20, 2025 11:26
@sasa-tomic sasa-tomic merged commit d75a08d into main Oct 20, 2025
10 checks passed
@sasa-tomic sasa-tomic deleted the feat/registry-dump-all-versions branch October 20, 2025 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants