Skip to content

[NIDX-02B] Drive the native encoder through the segment plugin contract #14074

Description

@hanahmily

Parent: #14007
Blocked by: #14073 (NIDX-02A). An open pull request does not unblock this; 02A must merge first.

Provisional shape. This leaf is the weakest of the NIDX-02 ladder against the workstream's "no layer-shaped tickets" rule: its consumer is the index library's own lifecycle rather than a live Property caller. It is filed separately so the encoder (#14073) and the production cutover (NIDX-02C) each stay reviewable, and so the cutover's dependency-policy question does not block the encoder. If maintainers judge it not vertical, fold it into NIDX-02C and close this as superseded — that is a cheaper outcome than an oversized cutover leaf, and the decision should be made before this becomes the oldest unblocked leaf.

Boundary

Wrap #14073's encoder in the SegmentPlugin contract so the existing lifecycle manager can drive it: New(results []segment.Document, normCalc func(string, int) float32) (segment.Segment, uint64, error), Load(*segment.Data) (segment.Segment, error), and Merge([]segment.Segment, []*roaring.Bitmap, int) segment.Merger.

The returned value must satisfy the full segment.Segment (14 methods) and segment.Merger (5 methods) contracts, not merely serialize bytes. Every type is reachable through the existing neutral segment alias.

Not registered in this leaf. No WithSegmentPlugin call, no change to inverted.NewStore configuration, no production behavior change. The plugin is constructed and driven only from tests.

Delegation question this leaf must answer, not assume

The cheaper implementation wraps encoded bytes with segment.NewDataBytes and delegates Load to the pinned loader, which would give the full interface for free. That delegation reaches the retired dependency's loader and cannot be written without a new token in tracked source. Do not promise delegation as available. Either a pre-approved neutral bridge exists by the time this leaf starts, or the adapter implements the contracts natively. Resolve this before the first commit and record the answer in the PR description.

Required behavior

  • The lifecycle manager persists a segment produced by New through the normal Segment.WriteTo path, and Load reopens the persisted bytes.
  • Merge produces a segment whose visible documents are the union of its inputs minus the supplied deletion bitmaps.
  • Output remains readable by nativeice.Open and by the pinned compatibility reader, exactly as in [NIDX-02A] Encode Property documents as native ICE v3 segment bytes #14073.

RED / E2E / suites

To be sharpened against 02A's landed API when this becomes the oldest unblocked leaf. The RED anchors on the lifecycle manager driving the plugin — failing on main because no plugin implementation exists — and the e2e on a persist/reopen/merge cycle whose output the merged reader still accepts.

Lexical gate

Zero new case-insensitive retired-dependency tokens or matching tracked paths. The segment alias is neutral and sufficient for the contracts; the aliased index package is not referenced from this leaf.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions