-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
data parsingTasks related to the ingestion and mapping of raw game JSONTasks related to the ingestion and mapping of raw game JSONenhancementNew feature or requestNew feature or request
Description
Summary
Refactor CBNData to precompute and cache canonical per-type lists during construction so byType() returns one entry per canonical key without per-call deduping, while preserving the raw merged layer stack required for mod override/self-copy inheritance and provenance.
Context
- Core data layer:
src/data.ts - Current mod merge is append-only concatenation in
mergeDataWithActiveMods(...) CBNDataconstructor builds both_byType(raw ordered rows) and_byTypeById/_abstractsByType(canonical maps)byType()now deduplicates at read time to avoid duplicate UI entries for modded objects- User-facing symptom that motivated this: duplicate entries on weapon category pages when mods override martial arts/items (example:
SUBMACHINE_GUNSwithMagicalNights)
Impact
- Current behavior is correct, but canonicalization now runs on every
byType()call byType()is used widely across the app, so repeated per-call deduping adds avoidable work and duplicates canonicalization logic with constructor indexing concepts- The API contract is still somewhat implicit: the constructor stores raw layered rows, while
byType()now synthesizes canonical lists on demand - A constructor-time canonical cache would make the data model clearer and reduce risk of future regressions when new
byType()consumers are added
Steps to reproduce
- Load data with a mod override that redefines an object using the same canonical key (e.g. same
idforGENERICitem ormartial_art) - Iterate objects using
data.byType(...) - Observe duplicate entries representing base + mod override when
byType()returns raw-layer semantics - Compare with
data.byId(...), which already returns only the canonical latest object
Tech details
- Raw mod merge intentionally preserves layer order (
coreDatafollowed by active mod data arrays) - Constructor uses this order to build override chains (
_overrides) for self-copy patterns (id === copy-from) - Deduping too early in raw merge can break override lineage because earlier overridden nodes may be needed for inheritance/provenance traversal
- Safer refactor direction:
- Keep raw merged rows and override-chain bookkeeping unchanged
- Build a constructor-time canonical per-type cache (e.g.
_byTypeCanonical) - Use the same provenance key strategy as
_byTypeById/_provenanceKeyForObject(...) - Make
byType()return the cached canonical list (flattened + monster visibility filtering) - Optionally add
byTypeRaw()for debugging/provenance use cases if needed
Examples (concrete examples with existing data)
- UI example:
/stable/weapon_category/SUBMACHINE_GUNS?mods=MagicalNightspreviously showed duplicateKrav Magaentries becausebyType("martial_art")exposed both base and mod override rows - Minimal data example:
- Base item:
{ type: "GENERIC", id: "test_item", weight: "1 kg" } - Mod override:
{ type: "GENERIC", id: "test_item", "copy-from": "test_item", relative: { weight: 100 } } - Expected canonical
byType("item")result: onetest_itementry (flattened weight1100)
- Base item:
Acceptance criteria
byType()returns canonical entries (one per canonical key) for keyed object families without duplicate mod override rowsbyType()preserves stable order semantics (original first-seen position retained, later override content wins)- Self-copy override inheritance and provenance logic continue to work (no regressions in
_overrides-based parent resolution) byId()/byIdMaybe()behavior remains unchanged- Existing UI pages that iterate
byType()do not need per-page dedupe logic to avoid duplicate modded entries - Add/keep regression tests covering duplicate-id mod overrides and canonical
byType()output - If a raw-list API is needed for debugging/provenance, it is explicit and documented
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
data parsingTasks related to the ingestion and mapping of raw game JSONTasks related to the ingestion and mapping of raw game JSONenhancementNew feature or requestNew feature or request