prepare 4.3.2 release (#79)

LaunchDarklyReleaseBot · eli-darkly · zmdavis · web-flow · commit 959c8d174939 · 2022-10-20T11:51:09.000-07:00
* initial move of code from js-client-sdk-private * changelog note * rm obsolete comment * add npm audit helper * update babel, jest, rollup * fix rollup config * fix ES build, dependency cleanup * add Releaser metadata * Update babel config to work in `test` without `useBuiltIns` * copyedits * fix misnamed directory * use spread operator instead of Object.assign * add issue templates * add babel-eslint * add event capacity config property * re-add deprecation warning on samplingInterval * better config validation * remove rollup-plugins-node-resolve * use newer Rollup node-resolve plugin * rm rollup-plugin-includepaths (unused) * npm audit fix (handlebars dependency from jest) * comment * copyedit * use new test helpers + misc test cleanup * clean up stream testing logic * fix hash parameter * linter * clearer way to model the config option defaults/types * test improvements * change internal param name * comment * fix default logger logic * simpler way to enforce minimum values * implement diagnostic events in common JS package (#11) * add support for function type in config options * add support for function type in config options (#13) * add wrapper metadata options and fix custom header logic * lint * lint * remove image-loading logic from common code, replace it with an abstraction * add validation for options.streaming * typo * rm unused params * typo in comment * misc fixes to merged code from external PR * add event payload ID header * npm audit fix * change exact dependencies to best-compatible * standardize linting * disallow "window" and "document" * improve diag event tests + debug logging * misc cleanup * fix updating secure mode hash with identify() * don't omit streamInits.failed when it's false * clean up init state logic, prevent unhandled rejections * lint * less strict matching of json content-type header * remove unsafe usage of hasOwnProperty * console logger must tolerate console object not always existing * fix double initialization of diagnostics manager * fix TypeScript declaration for track() and add more TS compilation tests (#27) * remove startsWith usage (#28) * prevent poll caused by a stream ping from overwriting later poll for another user (#29) * upgrade jest dependency and transitive yargs-parser dependency (#30) * Add null to LDEvaluationDetail.reason type (#31) * Revert "Add null to LDEvaluationDetail.reason type (#31)" This reverts commit bcb1573. * Revert "Add null to LDEvaluationDetail.reason type (#31)" This reverts commit bcb1573. * nullable evaluation reason (#32) * adding alias event functionality (#33) * set stream read timeout * Add prepare script (#34) * add a missing typescript verification (#36) * Removed the guides link * Correct doc link (#36) * Fix typo in LDClient.on jsdoc (#37) * add inlineUsersInEvents option in TypeScript (#37) * Filter private attributes on debug event users. Send variation for debug events. * update uuid package version (#39) * use Releaser v2 config + newer CI image * First half, add the type, create the new options, add the new util method, and add tests * Second half, call the tranform util method before calling any HTTP requests * Update the transform to work on a copy of headers instead of mutating it * add comments about removing custom event warning logic in the future * revert updating of UUID dependency (#43) * Revert "update uuid package version (#39)" This reverts commit 3b2ff6c. * update package-lock.json * better error handling for local storage operations (#44) * better error handling for local storage operations * lint * fix obsolete comments * add basic logger similar to server-side Node SDK (#45) * fix exports and add validation of custom logger (#46) * remove typedoc.js file that interferes with Releaser's docs build * update typescript version * add maintenance branch * remove deprecated things (#48) * remove deprecated options and function * rm references to obsolete function * restore deprecation logic, just leave the data empty * remove samplingInterval from TS test code * fix TS test code again * fix EvaluationDetail.reason to be nullable so we can get rid of NonNullableLDEvaluationReason type (#49) * remove deprecated options and function * rm references to obsolete function * restore deprecation logic, just leave the data empty * remove samplingInterval from TS test code * fix TS test code again * fix EvaluationDetail.reason to be nullable so we can get rid of NonNullableLDEvaluationReason type * fix TS test code * re-bump uuid package (#50) * Revert "Revert "update uuid package version (#39)"" This reverts commit 89359b1bf4ddbe6b2fedb95f1dc11240483c60f7. * remove lockfile (sc-107301) * use regular User-Agent header name unless overridden by js-client-sdk (#52) * switch to publishing js-sdk-common as a regular Node module (#51) * fix CI * remove `version` constant which can't be exported from js-sdk-common (#53) * catch errors in JSON parsing of stream data (#54) * catch errors in JSON parsing of stream data * lint * backport sc-142333 fix * prepare 3.5.1 release (#63) * initial move of code from js-client-sdk-private * changelog note * rm obsolete comment * add npm audit helper * update babel, jest, rollup * fix rollup config * fix ES build, dependency cleanup * add Releaser metadata * Update babel config to work in `test` without `useBuiltIns` * copyedits * fix misnamed directory * use spread operator instead of Object.assign * add issue templates * add babel-eslint * add event capacity config property * re-add deprecation warning on samplingInterval * better config validation * remove rollup-plugins-node-resolve * use newer Rollup node-resolve plugin * rm rollup-plugin-includepaths (unused) * npm audit fix (handlebars dependency from jest) * comment * copyedit * use new test helpers + misc test cleanup * clean up stream testing logic * fix hash parameter * linter * clearer way to model the config option defaults/types * test improvements * change internal param name * comment * fix default logger logic * simpler way to enforce minimum values * implement diagnostic events in common JS package (#11) * add support for function type in config options * add support for function type in config options (#13) * add wrapper metadata options and fix custom header logic * lint * lint * remove image-loading logic from common code, replace it with an abstraction * add validation for options.streaming * typo * rm unused params * typo in comment * misc fixes to merged code from external PR * add event payload ID header * npm audit fix * change exact dependencies to best-compatible * standardize linting * disallow "window" and "document" * improve diag event tests + debug logging * misc cleanup * fix updating secure mode hash with identify() * don't omit streamInits.failed when it's false * clean up init state logic, prevent unhandled rejections * lint * less strict matching of json content-type header * remove unsafe usage of hasOwnProperty * console logger must tolerate console object not always existing * fix double initialization of diagnostics manager * fix TypeScript declaration for track() and add more TS compilation tests (#27) * remove startsWith usage (#28) * prevent poll caused by a stream ping from overwriting later poll for another user (#29) * upgrade jest dependency and transitive yargs-parser dependency (#30) * Add null to LDEvaluationDetail.reason type (#31) * Revert "Add null to LDEvaluationDetail.reason type (#31)" This reverts commit bcb1573. * Revert "Add null to LDEvaluationDetail.reason type (#31)" This reverts commit bcb1573. * nullable evaluation reason (#32) * adding alias event functionality (#33) * set stream read timeout * Add prepare script (#34) * add a missing typescript verification (#36) * Removed the guides link * Correct doc link (#36) * Fix typo in LDClient.on jsdoc (#37) * add inlineUsersInEvents option in TypeScript (#37) * Filter private attributes on debug event users. Send variation for debug events. * update uuid package version (#39) * use Releaser v2 config + newer CI image * First half, add the type, create the new options, add the new util method, and add tests * Second half, call the tranform util method before calling any HTTP requests * Update the transform to work on a copy of headers instead of mutating it * add comments about removing custom event warning logic in the future * revert updating of UUID dependency (#43) * Revert "update uuid package version (#39)" This reverts commit 3b2ff6c. * update package-lock.json * better error handling for local storage operations (#44) * better error handling for local storage operations * lint * fix obsolete comments * add basic logger similar to server-side Node SDK (#45) * fix exports and add validation of custom logger (#46) * remove typedoc.js file that interferes with Releaser's docs build * update typescript version * add maintenance branch * backport sc-142333 fix Co-authored-by: Eli Bishop <eli@launchdarkly.com> Co-authored-by: Zach Davis <zach@launchdarkly.com> Co-authored-by: LaunchDarklyCI <dev@launchdarkly.com> Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com> Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com> Co-authored-by: Michael Siadak <mike.siadak@gmail.com> Co-authored-by: Jeff Wen <sinchangwen@gmail.com> Co-authored-by: Andrey Krasnov <34657799+Doesntmeananything@users.noreply.github.com> Co-authored-by: Gavin Whelan <gwhelan@launchdarkly.com> Co-authored-by: LaunchDarklyReleaseBot <launchdarklyreleasebot@launchdarkly.com> Co-authored-by: Louis Chan <lchan@launchdarkly.com> Co-authored-by: Louis Chan <91093020+louis-launchdarkly@users.noreply.github.com> * Releasing version 3.5.1 * rm obsolete file to fix merge * Releasing version 3.5.1 * make URL path concatenation work right whether base URL has a trailing slash or not (#61) * make URL path concatenation work right whether base URL has a trailing slash or not * lint * Implement support for application tags. (#55) * Fix typing of LDOptionsBase. (#63) * Implement application tags for 3.x. (#62) * lint * Add a line to refer to sendEventsOnlyForVariation * don't include deleted flags in allFlags (#66) * Enforce 64 character limit for tag value. (#68) * Enforce 64 character limit for tag value. * Lint. Comments. Remove unused param. * Remove the last seen cache, deprecate allowFrequentDuplicateEvents. (#73) * Inspector proposal V2. (#71) * Fix invoking flagUsed. (#77) * Port jitter and backoff. (#79) (#81) * remove flatMap usage to support older browsers (#82) Co-authored-by: Mateusz Sikora <mateusz.sikora@ramp.network> Co-authored-by: Eli Bishop <eli@launchdarkly.com> Co-authored-by: Zach Davis <zach@launchdarkly.com> Co-authored-by: LaunchDarklyCI <dev@launchdarkly.com> Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com> Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com> Co-authored-by: Michael Siadak <mike.siadak@gmail.com> Co-authored-by: Jeff Wen <sinchangwen@gmail.com> Co-authored-by: Andrey Krasnov <34657799+Doesntmeananything@users.noreply.github.com> Co-authored-by: Gavin Whelan <gwhelan@launchdarkly.com> Co-authored-by: LaunchDarklyReleaseBot <launchdarklyreleasebot@launchdarkly.com> Co-authored-by: Louis Chan <lchan@launchdarkly.com> Co-authored-by: Louis Chan <91093020+louis-launchdarkly@users.noreply.github.com> Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com> Co-authored-by: Mateusz Sikora <mateusz.sikora@ramp.network>
diff --git a/src/Stream.js b/src/Stream.js
@@ -1,6 +1,7 @@
 const messages = require('./messages');
 const { appendUrlPath, base64URLEncode, objectHasOwnProperty } = require('./utils');
 const { getLDHeaders, transformHeaders } = require('./headers');
+const { isHttpErrorRecoverable } = require('./errors');
 
 // The underlying event source implementation is abstracted via the platform object, which should
 // have these three properties:
@@ -16,6 +17,8 @@ const { getLDHeaders, transformHeaders } = require('./headers');
 // interval between heartbeats from the LaunchDarkly streaming server. If this amount of time elapses
 // with no new data, the connection will be cycled.
 const streamReadTimeoutMillis = 5 * 60 * 1000; // 5 minutes
+const maxRetryDelay = 30 * 1000; // Maximum retry delay 30 seconds.
+const jitterRatio = 0.5; // Delay should be 50%-100% of calculated time.
 
 function Stream(platform, config, environment, diagnosticsAccumulator) {
   const baseUrl = config.streamUrl;
@@ -24,7 +27,7 @@ function Stream(platform, config, environment, diagnosticsAccumulator) {
   const evalUrlPrefix = appendUrlPath(baseUrl, '/eval/' + environment);
   const useReport = config.useReport;
   const withReasons = config.evaluationReasons;
-  const streamReconnectDelay = config.streamReconnectDelay;
+  const baseReconnectDelay = config.streamReconnectDelay;
   const headers = getLDHeaders(platform, config);
   let firstConnectionErrorLogged = false;
   let es = null;
@@ -33,6 +36,22 @@ function Stream(platform, config, environment, diagnosticsAccumulator) {
   let user = null;
   let hash = null;
   let handlers = null;
+  let retryCount = 0;
+
+  function backoff() {
+    const delay = baseReconnectDelay * Math.pow(2, retryCount);
+    return delay > maxRetryDelay ? maxRetryDelay : delay;
+  }
+
+  function jitter(computedDelayMillis) {
+    return computedDelayMillis - Math.trunc(Math.random() * jitterRatio * computedDelayMillis);
+  }
+
+  function getNextRetryDelay() {
+    const delay = jitter(backoff());
+    retryCount += 1;
+    return delay;
+  }
 
   stream.connect = function(newUser, newHash, newHandlers) {
     user = newUser;
@@ -63,13 +82,31 @@ function Stream(platform, config, environment, diagnosticsAccumulator) {
   };
 
   function handleError(err) {
+    // The event source may not produce a status. But the LaunchDarkly
+    // polyfill can. If we can get the status, then we should stop retrying
+    // on certain error codes.
+    if (err.status && typeof err.status === 'number' && !isHttpErrorRecoverable(err.status)) {
+      // If we encounter an unrecoverable condition, then we do not want to
+      // retry anymore.
+      closeConnection();
+      logger.error(messages.unrecoverableStreamError(err));
+      // Ensure any pending retry attempts are not done.
+      if (reconnectTimeoutReference) {
+        clearTimeout(reconnectTimeoutReference);
+        reconnectTimeoutReference = null;
+      }
+      return;
+    }
+
+    const delay = getNextRetryDelay();
+
     if (!firstConnectionErrorLogged) {
-      logger.warn(messages.streamError(err, streamReconnectDelay));
+      logger.warn(messages.streamError(err, delay));
       firstConnectionErrorLogged = true;
     }
     logConnectionResult(false);
     closeConnection();
-    tryConnect(streamReconnectDelay);
+    tryConnect(delay);
   }
 
   function tryConnect(delay) {
@@ -123,6 +160,11 @@ function Stream(platform, config, environment, diagnosticsAccumulator) {
       }
 
       es.onerror = handleError;
+
+      es.onopen = () => {
+        // If the connection is a success, then reset the retryCount.
+        retryCount = 0;
+      };
     }
   }
 
diff --git a/src/__tests__/Stream-test.js b/src/__tests__/Stream-test.js
@@ -170,6 +170,7 @@ describe('Stream', () => {
     const nAttempts = 5;
     for (let i = 0; i < nAttempts; i++) {
       es.mockError('test error');
+      await sleepAsync(10);
       const created1 = await platform.testing.expectStream();
       const es1 = created1.eventSource;
 
@@ -185,6 +186,40 @@ describe('Stream', () => {
     }
   });
 
+  it.each([401, 403])('does not reconnect after an unrecoverable error', async status => {
+    const config = { ...defaultConfig, streamReconnectDelay: 1, useReport: false };
+    const stream = new Stream(platform, config, envName);
+    stream.connect(user);
+
+    const created = await platform.testing.expectStream();
+    const es = created.eventSource;
+
+    expect(es.readyState).toBe(EventSource.CONNECTING);
+    es.mockOpen();
+    expect(es.readyState).toBe(EventSource.OPEN);
+
+    es.mockError({ status });
+    await sleepAsync(10);
+    expect(platform.testing.eventSourcesCreated.length()).toEqual(0);
+  });
+
+  it.each([400, 408, 429])('does reconnect after a recoverable error', async status => {
+    const config = { ...defaultConfig, streamReconnectDelay: 1, useReport: false };
+    const stream = new Stream(platform, config, envName);
+    stream.connect(user);
+
+    const created = await platform.testing.expectStream();
+    const es = created.eventSource;
+
+    expect(es.readyState).toBe(EventSource.CONNECTING);
+    es.mockOpen();
+    expect(es.readyState).toBe(EventSource.OPEN);
+
+    es.mockError({ status });
+    await sleepAsync(10);
+    expect(platform.testing.eventSourcesCreated.length()).toEqual(1);
+  });
+
   it('logs a warning for only the first failed connection attempt', async () => {
     const config = { ...defaultConfig, streamReconnectDelay: 1 };
     const stream = new Stream(platform, config, envName);
@@ -197,6 +232,7 @@ describe('Stream', () => {
     const nAttempts = 5;
     for (let i = 0; i < nAttempts; i++) {
       es.mockError('test error');
+      await sleepAsync(10);
       const created1 = await platform.testing.expectStream();
       es = created1.eventSource;
       es.mockOpen();
@@ -221,6 +257,7 @@ describe('Stream', () => {
     const nAttempts = 5;
     for (let i = 0; i < nAttempts; i++) {
       es.mockError('test error #1');
+      await sleepAsync(10);
       const created1 = await platform.testing.expectStream();
       es = created1.eventSource;
       es.mockOpen();
@@ -232,15 +269,16 @@ describe('Stream', () => {
 
     for (let i = 0; i < nAttempts; i++) {
       es.mockError('test error #2');
+      await sleepAsync(10);
       const created1 = await platform.testing.expectStream();
       es = created1.eventSource;
       es.mockOpen();
     }
 
     // make sure there is just a single logged message rather than five (one per attempt)
     expect(logger.output.warn).toEqual([
-      messages.streamError('test error #1', 1),
-      messages.streamError('test error #2', 1),
+      expect.stringContaining('test error #1'),
+      expect.stringContaining('test error #2'),
     ]);
   });
 
diff --git a/src/messages.js b/src/messages.js
@@ -123,14 +123,16 @@ const streamError = function(err, streamReconnectDelay) {
   return (
     'Error on stream connection: ' +
     errorString(err) +
-    ', will continue retrying every ' +
+    ', will continue retrying after ' +
     streamReconnectDelay +
     ' milliseconds.'
   );
 };
 
 const unknownOption = name => 'Ignoring unknown config option "' + name + '"';
 
+const unrecoverableStreamError = err => `Error on stream connection ${errorString(err)}, giving up permanently`;
+
 const wrongOptionType = (name, expectedType, actualType) =>
   'Config option "' + name + '" should be of type ' + expectedType + ', got ' + actualType + ', using default value';
 
@@ -228,6 +230,7 @@ module.exports = {
   tagValueTooLong,
   unknownCustomEventKey,
   unknownOption,
+  unrecoverableStreamError,
   userNotSpecified,
   wrongOptionType,
   wrongOptionTypeBoolean,
