Skip to content

Commit e35de89

Browse files
abueideclaude
abueide
and
claude
authored
feat(e2e-cli): add flush-retry loop with drop detection (#1175)
* feat(e2e-cli): add flush-retry loop to simulate real flush policy The CLI previously called flush() once and exited, so events that received retryable errors (5xx, 429) stayed in the queue with no retry. This implements the retry loop that flush policies drive in a real app: flush → check pending → wait for backoff → repeat. - Flush-retry loop respects maxRetries from test config - Forward-compatible with tapi RetryManager (reads backoff state when available, falls back to fixed delay on master) - Tracks permanently dropped events via logger interception - Reports success=false when events remain or are dropped - Computes sentBatches from delivered event count - Enables retry test suite in e2e-config.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: remove redundant inline comments from e2e CLI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore pre-existing comments removed in previous commit Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: use real flush policies instead of manual retry loop Instead of manually calling flush() in a loop and reading private RetryManager state, let the SDK's built-in flush policies drive retries. TimerFlushPolicy fires every flushInterval (100ms default for e2e), and the RetryManager gates actual uploads during backoff. The CLI just triggers the initial flush, then polls pendingEvents() until the queue drains or 30s timeout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(e2e-cli): wire maxRetries, error reporting, and BROWSER_BATCHING - Pass maxRetries from test config into httpConfig overrides so the SDK enforces retry limits during e2e tests - Set output.error when permanentDropCount > 0 so failure reporting tests get a truthy error field - Add BROWSER_BATCHING=true to e2e-config.json to skip tests that assume ephemeral per-request batching (RN uses persistent queue re-chunking) - Add jsx: react to tsconfig.json for .tsx transitive imports Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: remove verbose inline comments from e2e CLI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(e2e-cli): simplify cli.ts with extracted helper functions Extract buildConfig, dispatchEvent, waitForQueueDrain, and interceptDropCount from main(). Drop redundant per-event validation warnings and error handling. 367 -> 256 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(e2e-cli): improve robustness of flush-retry helpers - Return boolean from waitForQueueDrain to indicate drain vs timeout - Add JSDoc for interceptDropCount noting coupling to SegmentDestination log format - Add console.warn in dispatchEvent when events are skipped for missing fields - Add comment on sentBatches approximation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(core): expose droppedEvents() on SegmentClient Replace fragile log-interception pattern in e2e-cli with a proper counter on SegmentDestination. The CLI now calls client.droppedEvents() instead of monkey-patching logger.error and regex-matching drop messages. - Add droppedEventCount property to SegmentDestination - Add droppedEvents() accessor on SegmentClient (mirrors pendingEvents) - Remove interceptDropCount helper from e2e-cli Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting in e2e CLI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(e2e-cli): remove retry test suite until retry stack is integrated The retry e2e tests require RetryManager and SegmentDestination retry wiring from downstream branches. Re-enable once that stack lands. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: replace droppedEventCount API with errorHandler callback Remove custom droppedEventCount/droppedEvents() API in favor of the existing errorHandler callback. Add ErrorType.EventsDropped so consumers can filter for drop events. Wire CLI to use errorHandler for counting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add metadata field to SegmentError for structured payloads Add optional metadata field to SegmentError so consumers can access structured data without parsing. Update CLI to read droppedCount from metadata instead of incrementing by 1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 3f0234e commit e35de89

File tree

5 files changed

+145
-192
lines changed

5 files changed

+145
-192
lines changed

‎e2e-cli/e2e-config.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,7 @@
33
"test_suites": "basic,settings",
44
"auto_settings": true,
55
"patch": null,
6-
"env": {}
6+
"env": {
7+
"BROWSER_BATCHING": "true"
8+
}
79
}

‎e2e-cli/src/cli.ts‎

Lines changed: 131 additions & 190 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
/**
22
* E2E CLI for React Native analytics SDK testing
33
*
4-
* Runs the real SDK pipeline (SegmentClient → Timeline → SegmentDestination →
5-
* QueueFlushingPlugin → uploadEvents) with stubs for React Native runtime
4+
* Runs the real SDK pipeline (SegmentClient -> Timeline -> SegmentDestination ->
5+
* QueueFlushingPlugin -> uploadEvents) with stubs for React Native runtime
66
* dependencies so everything executes on Node.js.
77
*
88
* Usage:
@@ -13,6 +13,7 @@ import { SegmentClient } from '../../packages/core/src/analytics';
1313
import { SovranStorage } from '../../packages/core/src/storage/sovranStorage';
1414
import { Logger } from '../../packages/core/src/logger';
1515
import type { Config, JsonMap } from '../../packages/core/src/types';
16+
import { ErrorType } from '../../packages/core/src/errors';
1617
import type { Persistor } from '@segment/sovran-react-native';
1718

1819
// ============================================================================
@@ -75,13 +76,106 @@ const MemoryPersistor: Persistor = {
7576
};
7677

7778
// ============================================================================
78-
// Main CLI Logic
79+
// Helper Functions
80+
// ============================================================================
81+
82+
function buildConfig(input: CLIInput): Config {
83+
return {
84+
writeKey: input.writeKey,
85+
trackAppLifecycleEvents: false,
86+
trackDeepLinks: false,
87+
autoAddSegmentDestination: true,
88+
storePersistor: MemoryPersistor,
89+
storePersistorSaveDelay: 0,
90+
...(input.apiHost && { proxy: input.apiHost, useSegmentEndpoints: true }),
91+
...(input.cdnHost && {
92+
cdnProxy: input.cdnHost,
93+
useSegmentEndpoints: true,
94+
}),
95+
defaultSettings: {
96+
integrations: {
97+
'Segment.io': {
98+
apiKey: input.writeKey,
99+
apiHost: 'api.segment.io/v1',
100+
},
101+
},
102+
},
103+
...(input.config?.flushAt !== undefined && {
104+
flushAt: input.config.flushAt,
105+
}),
106+
flushInterval: input.config?.flushInterval ?? 0.1,
107+
...(input.config?.maxRetries !== undefined && {
108+
httpConfig: {
109+
rateLimitConfig: { maxRetryCount: input.config.maxRetries },
110+
backoffConfig: { maxRetryCount: input.config.maxRetries },
111+
},
112+
}),
113+
};
114+
}
115+
116+
async function dispatchEvent(
117+
client: SegmentClient,
118+
evt: AnalyticsEvent
119+
): Promise<void> {
120+
switch (evt.type) {
121+
case 'track':
122+
if (!evt.event) {
123+
console.warn(`[e2e] skipping track: missing event name`);
124+
return;
125+
}
126+
await client.track(evt.event, evt.properties as JsonMap | undefined);
127+
break;
128+
case 'identify':
129+
await client.identify(evt.userId, evt.traits as JsonMap | undefined);
130+
break;
131+
case 'screen':
132+
case 'page':
133+
if (!evt.name) {
134+
console.warn(`[e2e] skipping ${evt.type}: missing name`);
135+
return;
136+
}
137+
await client.screen(evt.name, evt.properties as JsonMap | undefined);
138+
break;
139+
case 'group':
140+
if (!evt.groupId) {
141+
console.warn(`[e2e] skipping group: missing groupId`);
142+
return;
143+
}
144+
await client.group(evt.groupId, evt.traits as JsonMap | undefined);
145+
break;
146+
case 'alias':
147+
if (!evt.userId) {
148+
console.warn(`[e2e] skipping alias: missing userId`);
149+
return;
150+
}
151+
await client.alias(evt.userId);
152+
break;
153+
default:
154+
console.warn(`[e2e] skipping event: unknown type "${evt.type}"`);
155+
}
156+
}
157+
158+
/** Polls pendingEvents() until the queue is empty or timeout. Returns true if drained. */
159+
async function waitForQueueDrain(
160+
client: SegmentClient,
161+
timeoutMs = 30_000
162+
): Promise<boolean> {
163+
const pollMs = 50;
164+
const start = Date.now();
165+
while (Date.now() - start < timeoutMs) {
166+
if ((await client.pendingEvents()) === 0) return true;
167+
await new Promise((r) => setTimeout(r, pollMs));
168+
}
169+
return false;
170+
}
171+
172+
// ============================================================================
173+
// Main
79174
// ============================================================================
80175

81176
async function main() {
82177
const args = process.argv.slice(2);
83178
let inputStr: string | undefined;
84-
85179
for (let i = 0; i < args.length; i++) {
86180
if (args[i] === '--input' && i + 1 < args.length) {
87181
inputStr = args[i + 1];
@@ -104,217 +198,64 @@ async function main() {
104198

105199
try {
106200
const input: CLIInput = JSON.parse(inputStr);
107-
108-
// Build SDK config
201+
let permanentDropCount = 0;
109202
const config: Config = {
110-
writeKey: input.writeKey,
111-
trackAppLifecycleEvents: false,
112-
trackDeepLinks: false,
113-
autoAddSegmentDestination: true,
114-
storePersistor: MemoryPersistor,
115-
storePersistorSaveDelay: 0,
116-
// When apiHost is provided (mock tests), use proxy to direct events there
117-
...(input.apiHost && {
118-
proxy: input.apiHost,
119-
useSegmentEndpoints: true,
120-
}),
121-
// When cdnHost is provided (mock tests), use cdnProxy to direct CDN requests there
122-
...(input.cdnHost && {
123-
cdnProxy: input.cdnHost,
124-
useSegmentEndpoints: true,
125-
}),
126-
// Provide default settings so SDK doesn't require CDN response
127-
defaultSettings: {
128-
integrations: {
129-
'Segment.io': {
130-
apiKey: input.writeKey,
131-
apiHost: 'api.segment.io/v1',
132-
},
133-
},
203+
...buildConfig(input),
204+
errorHandler: (error) => {
205+
if (error.type === ErrorType.EventsDropped) {
206+
const count =
207+
(error.metadata?.droppedCount as number | undefined) ?? 1;
208+
permanentDropCount += count;
209+
}
134210
},
135-
...(input.config?.flushAt !== undefined && {
136-
flushAt: input.config.flushAt,
137-
}),
138-
...(input.config?.flushInterval !== undefined && {
139-
flushInterval: input.config.flushInterval,
140-
}),
141211
};
142212

143-
// Create storage with in-memory persistor
144213
const store = new SovranStorage({
145214
storeId: input.writeKey,
146215
storePersistor: MemoryPersistor,
147216
storePersistorSaveDelay: 0,
148217
});
149-
150-
// Suppress SDK internal logs to keep E2E test output clean.
151-
// CLI-level warnings/errors still surface via console.warn/console.error.
152218
const logger = new Logger(true);
153219
const client = new SegmentClient({ config, logger, store });
154-
155-
// Initialize — adds plugins, resolves settings, processes pending events
156220
await client.init();
157221

158-
// Process event sequences
159222
for (const sequence of input.sequences) {
160223
if (sequence.delayMs > 0) {
161224
await new Promise((resolve) => setTimeout(resolve, sequence.delayMs));
162225
}
163-
164226
for (const evt of sequence.events) {
165-
// Validate event has a type
166-
if (!evt.type) {
167-
console.warn('[WARN] Skipping event: missing event type', evt);
168-
continue;
169-
}
170-
171-
try {
172-
switch (evt.type) {
173-
case 'track': {
174-
// Required: event name
175-
if (!evt.event || typeof evt.event !== 'string') {
176-
console.warn(
177-
`[WARN] Skipping track event: missing or invalid event name`,
178-
evt
179-
);
180-
continue;
181-
}
182-
183-
// Optional: properties (validate if present)
184-
const properties = evt.properties as JsonMap | undefined;
185-
if (
186-
evt.properties !== undefined &&
187-
(evt.properties === null ||
188-
Array.isArray(evt.properties) ||
189-
typeof evt.properties !== 'object')
190-
) {
191-
console.warn(
192-
`[WARN] Track event "${evt.event}" has invalid properties, proceeding without them`
193-
);
194-
}
195-
196-
await client.track(evt.event, properties);
197-
break;
198-
}
199-
200-
case 'identify': {
201-
// Optional userId (Segment allows anonymous identify)
202-
// Optional traits (validate if present)
203-
const traits = evt.traits as JsonMap | undefined;
204-
if (
205-
evt.traits !== undefined &&
206-
(evt.traits === null ||
207-
Array.isArray(evt.traits) ||
208-
typeof evt.traits !== 'object')
209-
) {
210-
console.warn(
211-
`[WARN] Identify event has invalid traits, proceeding without them`
212-
);
213-
}
214-
215-
await client.identify(evt.userId, traits);
216-
break;
217-
}
218-
219-
case 'screen':
220-
case 'page': {
221-
// RN SDK has no page(); map to screen for cross-SDK test compat
222-
// Required: screen/page name
223-
if (!evt.name || typeof evt.name !== 'string') {
224-
console.warn(
225-
`[WARN] Skipping ${evt.type} event: missing or invalid name`,
226-
evt
227-
);
228-
continue;
229-
}
230-
231-
// Optional: properties (validate if present)
232-
const properties = evt.properties as JsonMap | undefined;
233-
if (
234-
evt.properties !== undefined &&
235-
(evt.properties === null ||
236-
Array.isArray(evt.properties) ||
237-
typeof evt.properties !== 'object')
238-
) {
239-
console.warn(
240-
`[WARN] Screen "${evt.name}" has invalid properties, proceeding without them`
241-
);
242-
}
243-
244-
await client.screen(evt.name, properties);
245-
break;
246-
}
247-
248-
case 'group': {
249-
// Required: groupId
250-
if (!evt.groupId || typeof evt.groupId !== 'string') {
251-
console.warn(
252-
`[WARN] Skipping group event: missing or invalid groupId`,
253-
evt
254-
);
255-
continue;
256-
}
257-
258-
// Optional: traits (validate if present)
259-
const traits = evt.traits as JsonMap | undefined;
260-
if (
261-
evt.traits !== undefined &&
262-
(evt.traits === null ||
263-
Array.isArray(evt.traits) ||
264-
typeof evt.traits !== 'object')
265-
) {
266-
console.warn(
267-
`[WARN] Group event for "${evt.groupId}" has invalid traits, proceeding without them`
268-
);
269-
}
270-
271-
await client.group(evt.groupId, traits);
272-
break;
273-
}
274-
275-
case 'alias': {
276-
// Required: userId
277-
if (!evt.userId || typeof evt.userId !== 'string') {
278-
console.warn(
279-
`[WARN] Skipping alias event: missing or invalid userId`,
280-
evt
281-
);
282-
continue;
283-
}
284-
285-
await client.alias(evt.userId);
286-
break;
287-
}
288-
289-
default:
290-
console.warn(
291-
`[WARN] Skipping event: unknown event type "${evt.type}"`,
292-
evt
293-
);
294-
continue;
295-
}
296-
} catch (error) {
297-
// Log but don't fail the entire sequence if one event fails
298-
console.error(
299-
`[ERROR] Failed to process ${evt.type} event:`,
300-
error,
301-
evt
302-
);
303-
continue;
304-
}
227+
await dispatchEvent(client, evt);
305228
}
306229
}
307230

308-
// Flush all queued events through the real pipeline
309231
await client.flush();
232+
const drained = await waitForQueueDrain(client);
310233

311-
// Brief delay to let async upload operations settle
312-
await new Promise((resolve) => setTimeout(resolve, 500));
234+
const finalPending = drained ? 0 : await client.pendingEvents();
235+
const totalEvents = input.sequences.reduce(
236+
(sum, seq) => sum + seq.events.length,
237+
0
238+
);
239+
const delivered = Math.max(
240+
0,
241+
totalEvents - finalPending - permanentDropCount
242+
);
243+
// Approximate: SDK doesn't expose actual batch count, so we derive it
244+
// from delivered event count and configured batch size.
245+
const sentBatches =
246+
delivered > 0
247+
? Math.ceil(delivered / Math.max(1, input.config?.flushAt ?? 1))
248+
: 0;
249+
const success = finalPending === 0 && permanentDropCount === 0;
313250

314251
client.cleanup();
315-
316-
// sentBatches: SDK doesn't expose batch count tracking
317-
output = { success: true, sentBatches: 0 };
252+
output = {
253+
success,
254+
sentBatches,
255+
...(permanentDropCount > 0 && {
256+
error: `${permanentDropCount} events permanently dropped`,
257+
}),
258+
};
318259
} catch (e) {
319260
const error = e instanceof Error ? e.message : String(e);
320261
output = { success: false, error, sentBatches: 0 };

‎e2e-cli/tsconfig.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
"strict": true,
77
"esModuleInterop": true,
88
"skipLibCheck": true,
9+
"jsx": "react",
910
"forceConsistentCasingInFileNames": true,
1011
// Note: TypeScript is used only for type-checking; build output is generated by esbuild (see build.js).
1112
"noEmit": true,

0 commit comments

Comments
 (0)