feat(experimental): partial view updates — view_update event type#4230
Draft
feat(experimental): partial view updates — view_update event type#4230
Conversation
…ure flags, type plumbing - view_update-schema.json: new partial view update schema with optional view fields - Registered in rum-events-browser-schema.json + rum-events-schema.json - Regenerated rumEvent.types.ts → RumViewUpdateEvent generated type - ExperimentalFeature.VIEW_UPDATE + VIEW_UPDATE_CHAOS added - RumEventType.VIEW_UPDATE = 'view_update' added to const object - RawRumViewUpdateEvent interface with all view fields optional - RawRumEvent and AssembledRumEvent unions updated - assembly.ts: VIEW_UPDATE modifiable paths + beforeSend denylist - domainContext.types.ts: VIEW_UPDATE → RumViewEventDomainContext - trackEventCounts.ts: skip view_update events - test/fixtures.ts: VIEW_UPDATE case in createRawRumEvent() Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
…ted by VIEW_UPDATE feature flag Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
…or assembly/tracking/contexts Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
…sport unit tests Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
…ine coverage guard - Fix is_active always sent in VUs (always true — removed from processViewDiff) - Fix performance object duplicating individual web vital fields (removed from processViewDiff) - Fix display.scroll sent on every VU due to Duration/ServerDuration unit mismatch in comparison - Fix feature_flags leaked on every VU (backfill bug — now correctly backfilled after first VU) - Genericize stripUnchangedFields: routing keys (type/application/date/view/session/_dd) are always kept; everything else is stripped if equal to snapshot — new top-level fields handled automatically without code changes - Genericize snapshot backfill loop to cover future fields automatically - Add TypeScript compile-time exhaustiveness guard on RawRumViewUpdateEvent['view'] fields: adding a new schema field without updating processViewDiff produces a type error Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
|
All contributors have signed the CLA ✍️ ✅ |
|
✅ Tests 🎉 All green!❄️ No new flaky tests detected 🎯 Code Coverage (details) 🔗 Commit SHA: 836d653 | Docs | Datadog PR Page | Was this helpful? Give us feedback! |
Bundles Sizes Evolution
🚀 CPU Performance
🧠 Memory Performance
|
- Fix curly rule: add braces to single-statement if bodies - Fix unused destructured vars: replace destructuring-to-omit with delete for _dd.*, session.type, view.name/url/referrer, display.viewport - Fix no-unsafe-call: use String() instead of ((x as any).y).slice() - Fix no-unnecessary-type-assertion: remove redundant snapshot! non-null assertions (TypeScript narrows correctly via shouldSendFull guard) - Fix unused vars: eslint-disable for _vuViewFieldsCoverage coverage guard, rename configuration → _configuration in processViewDiff (param unused) - Fix spec no-unnecessary-type-assertion: remove redundant `as string` casts on interceptor.requests[0].body (already typed as string) - Fix spec no-unsafe-return: cast JSON.parse result to Record<string, any> - Fix spec camelcase: rename ci_test variable to ciTest - Fix spec typecheck: access is_active via (event.view as any).is_active - Fix eventRow.tsx typecheck: add view_update to RUM_EVENT_TYPE_COLOR map - Fix schema: commit view_update-schema.json and update rum-events-schema.json and rum-events-browser-schema.json to include view_update type; submodule pointer updated to new commit on view-update-schema branch Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
Author
|
I have read the CLA Document and I hereby sign the CLA |
Signed-off-by: Andrei Cojocaru <andrei.cojocaru@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
RUM VIEW events are expensive. Every update to a view — a new action, a resource, a CLS shift, a scroll — emits a full VIEW event with ~1–2 KB of repeated static data (usr, context, connectivity, feature_flags, service, version, configuration, etc.). Most of this is identical across all updates for the same view lifetime. At scale, this generates significant unnecessary ingestion volume.
Solution
Introduces a new
view_updateevent type — a partial view update that carries only the fields that changed since the last VIEW event for that view. All static/session-level fields are omitted when unchanged, reducing average per-update size by ~75%.The feature is gated behind an experimental feature flag:
VIEW_UPDATE.Architecture Overview
New event type:
view_updateview_updateis structurally distinct fromview:type,application,date,view.id,view.time_spent,session.id,_dd.document_versionview.is_active(VUs are only emitted for active views; view end always sends a full VIEW)Four-layer implementation
1. Schema layer (
rum-events-formatsubmodule)view_update-schema.jsondefining the partial event shaperum-events-schema.jsonandrum-events-browser-schema.jsonto includeview_updatein theoneOfrumEvent.types.tsupdated withRumViewUpdateEventtype2. Diff engine (
viewCollection.ts)snapshotStore: Map<viewId, ViewSnapshot>— tracks the last emitted full VIEW event per viewVIEW_UPDATED(doc_version > 1, feature enabled): computes a diff viaprocessViewDiff()and emitsview_update; refreshes the snapshotprocessViewDiff()explicitly diffs all view fields: counters (action/error/long_task/resource/frustration), web vitals (loading_time, CLS, INP, FCP, LCP, FID, dom timings, first_byte), custom_timings, replay_stats, scroll/displayperformanceobject (redundant — each sub-metric is already sent as an individual flat field)3. Post-assembly strip (
startRumBatch.ts)assembledViewSnapshots(separate from the diff-engine snapshot, operates on the fully assembled + enriched event)view_updateis batched,stripUnchangedFields()removes session-static top-level fields equal to the snapshot:usr,context,connectivity,feature_flags,service,version,source,synthetics,ci_test,ddtags, and any future new fields automatically (generic key loop, not an explicit allowlist)_dd.{format_version, sdk_name, configuration, browser_sdk_version},session.type,view.{name, url, referrer},display.viewportusrset after init) are written back into the snapshot so subsequent VUs can strip them when unchangedbatch.add()(notbatch.upsert()), so each one is preserved independently — there is no deduplication4. Context plumbing (
domainContext.types.ts,assembly.ts,featureFlagContext,sessionContext)view_updateevents correctly excluded from feature flag tracking, session replay stats, and other context hooks that should only apply to full VIEW eventsCompile-time safety net (
viewCollection.ts)A TypeScript exhaustiveness guard asserts at compile time that every field in
RawRumViewUpdateEvent['view']is either explicitly diffed, always present, or explicitly omitted with justification. Adding a new field to the schema without updatingprocessViewDiffwill produce a type error.Measured Bandwidth Savings
Benchmarked with a 3-minute headless Playwright session (42 view updates):
is_activesentperformancesentdisplay.scroll(unchanged)feature_flagsFiles Changed
rum-events-format/(submodule)view_update-schema.json; added to bothrum-events-schema.jsonandrum-events-browser-schema.jsonpackages/rum-core/src/rumEvent.types.tsRumViewUpdateEventtype, added toRumEventunionpackages/rum-core/src/rawRumEvent.types.tsRawRumViewUpdateEventinterface +VIEW_UPDATEinRumEventTypeenumpackages/core/src/tools/experimentalFeatures.tsVIEW_UPDATEexperimental flagpackages/rum-core/src/domain/view/viewCollection.tsprocessViewDiff, periodic refresh, exhaustiveness guardpackages/rum-core/src/transport/startRumBatch.tsview_updaterouted tobatch.add()packages/rum-core/src/domainContext.types.tsview_updatefrom context hook typespackages/rum-core/src/domain/assembly.tsview_updateeventspackages/rum-core/src/domain/contexts/featureFlagContext.tsview_updateeventspackages/rum-core/src/domain/contexts/sessionContext.tsview_updatefrom session context trackingdeveloper-extension/src/panel/components/tabs/eventsTab/eventRow.tsxview_updateto event color mapKey Design Decisions
Why two snapshot stores?
The diff engine snapshot (in
viewCollection.ts) operates on the raw RUM event before assembly enrichment. The post-assembly snapshot (instartRumBatch.ts) operates on the fully assembled event with all context fields merged in. Both are needed: the diff engine computes view-specific deltas, while the post-assembly strip removes session-level fields that are only known after assembly.Why
batch.add()notbatch.upsert()for view_update?upsertdeduplicates by view ID, keeping only the latest. VUs must all be preserved — each one represents an independent delta. Usingadd()ensures they're all written to the encoder in order.Why a generic strip loop instead of an explicit allowlist?
An explicit allowlist (the original approach) would silently pass through any new top-level field added to the schema, sending it redundantly on every VU. The generic approach strips anything equal to the snapshot, so new fields are handled automatically. Only routing keys (
type,application,date,view,session,_dd) are preserved unconditionally.Why periodic full-VIEW refresh?
Prevents the downstream decoder from accumulating unbounded deltas if events are lost or received out of order. The 50-update / 5-minute thresholds are conservative for the initial rollout.
Testing
viewCollection.spec.tscovering the diff engine (counters, vitals, scroll, CLS, custom timings, replay stats, periodic refresh, snapshot lifecycle), plus 35 new tests instartRumBatch.spec.tscovering the post-assembly strip (all field types, backfill, snapshot eviction on view end)collectAndValidateRawRumEventsOut of Scope / Follow-up
performancesub-object from full VIEW events (separate cleanup)check-staging-mergeCI failure: expected — submodule pointer conflicts withstaging-09; resolves once therum-events-formatPR lands onmain