diff --git a/README.md b/README.md index e5ee0655e..9f4186ff7 100644 --- a/README.md +++ b/README.md @@ -35,3 +35,4 @@ This library is one of Segment’s most popular Flagship libraries. It is active - Contribution guidelines: [CONTRIBUTING.md](CONTRIBUTING.md) - Development instructions: [DEVELOPMENT.md](DEVELOPMENT.md) +- Releasing (to npm): [RELEASING.md](RELEASING.md) diff --git a/packages/browser/ARCHITECTURE.md b/packages/browser/ARCHITECTURE.md deleted file mode 100644 index 14341d7bd..000000000 --- a/packages/browser/ARCHITECTURE.md +++ /dev/null @@ -1,51 +0,0 @@ -# Architecture - -## Codebases - -analytics-next is comprised of two different codebases: - -- [analytics-next](https://github.com/segmentio/analytics-next): All core functionality of AJSN -- [integrations](https://github.com/segmentio/analytics.js-integrations): All existing legacy client side destinations - -This diagram outlines the relationship between these two codebases (cloudfront being where the integrations are hosted): -![Architecture](.github/architecture.png?raw=true) - -## Core - -Some of the bigger core modules are briefly outlined here - - -- arguments-resolver: Responsible for reshuffling arguments in event calls -- context: Responsible for building context objects for different events -- emitter: Responsible for emitting events and adding listeners for them -- events: Responsible for building the different Segment events (track, page, etc), and normalizing those events -- logger: Responsible for building and flushing the logs that will be carried around in an events context object -- queue: The heart of AJSN, the queue is responsible for managing events which will be delivered to plugins and destinations, and handles things such as retries, timeouts, and delivery reliability -- user: Responsible for all of the logic built around an analytics user - -The general end to end flow of analytics-next core is as follows: - -1. Legacy settings and destinations are loaded from the segment CDN -2. The Analytics object is instantiated with the loaded settings, and sets up things such as new/existing users and an event queue. -3. Events are built and queued when a user makes a Segment call (ie. analytics.track()) -4. The event is dispatched, goes through all enabled plugins, and is finally sent through the segment.io plugin to get the data into Segment - -## Everything is a Plugin - -When developing against analytics-next you will likely be writing plugins, which can augment AJSN functionality and enrich data. Plugins are isolated chunks which you can build, test, version, and deploy independently of the rest of the codebase. Extensions are bounded by AJSN which handles things such as observability, retries, and error management. - -Plugins can be of two different priorities: - -- Critical: AJSN should expect this plugin to be loaded before starting event delivery -- Non-critical: AJSN can start event delivery before this plugin has finished loading - -and can be of five different types: - -- Before: Pleguns that need to be run before any other plugins are run. An example of this would be validating events before passing them along to other plugins. -- After: Plugins that need to run after all other plugins have run. An example of this is the segment.io integration, which will wait for destinations to succeed or fail so that it can send its observability metrics. -- Destination: Destinations to send the event to (ie. legacy destinations). Does not modify the event and failure does not halt execution. -- Enrichment: Modifies an event, failure here could halt the event pipeline. -- Utility: Plugins that change AJSN functionality and don't fall into the other categories. - -## Observability - -Every event and plugin has a context object, which contains both metrics and logs that were collected throughout their lifetime. Logs can be used for debugging, and the metrics will be included when the event is sent to Segment. diff --git a/packages/browser/README.md b/packages/browser/README.md index 658b15eb4..ad99c7457 100644 --- a/packages/browser/README.md +++ b/packages/browser/README.md @@ -7,7 +7,7 @@ Analytics Next (aka Analytics 2.0) is the latest version of Segment’s JavaScri - [🏎️ Quickstart](#-quickstart) - [πŸ’‘ Using with Segment](#-using-with-segment) - [πŸ’» Using as an `npm` package](#-using-as-an-npm-package) -- [πŸ”Œ Plugins](#-plugins) +- [πŸ”Œ Architecture](#-architecture--plugins) - [πŸ’ Development](#-development) --- @@ -187,6 +187,8 @@ declare global { } ``` +## πŸ”Œ Architecture & Plugins +- See [ARCHITECTURE.md](architecture/ARCHITECTURE.md) ## πŸ’ Development @@ -207,56 +209,5 @@ Then, make your changes and test them out in the test app! Example of the development app -# πŸ”Œ Plugins - -When developing against Analytics Next you will likely be writing plugins, which can augment functionality and enrich data. Plugins are isolated chunks which you can build, test, version, and deploy independently of the rest of the codebase. Plugins are bounded by Analytics Next which handles things such as observability, retries, and error management. - -Plugins can be of two different priorities: - -1. **Critical**: Analytics Next should expect this plugin to be loaded before starting event delivery -2. **Non-critical**: Analytics Next can start event delivery before this plugin has finished loading - -and can be of five different types: - -1. **Before**: Plugins that need to be run before any other plugins are run. An example of this would be validating events before passing them along to other plugins. -2. **After**: Plugins that need to run after all other plugins have run. An example of this is the segment.io integration, which will wait for destinations to succeed or fail so that it can send its observability metrics. -3. **Destination**: Destinations to send the event to (ie. legacy destinations). Does not modify the event and failure does not halt execution. -4. **Enrichment**: Modifies an event, failure here could halt the event pipeline. -5. **Utility**: Plugins that change Analytics Next functionality and don't fall into the other categories. - -Here is an example of a simple plugin that would convert all track events event names to lowercase before the event gets sent through the rest of the pipeline: - -```ts -import type { Plugin } from '@segment/analytics-next' - -export const lowercase: Plugin = { - name: 'Lowercase Event Name', - type: 'before', - version: '1.0.0', - - isLoaded: () => true, - load: () => Promise.resolve(), - - track: (ctx) => { - ctx.event.event = ctx.event.event.toLowerCase() - return ctx - } -} - -analytics.register(lowercase) -``` - -For further examples check out our [existing plugins](/packages/browser/src/plugins). - -## πŸ§ͺ QA -Feature work and bug fixes should include tests. Run all [Jest](https://jestjs.io) tests: -``` -$ yarn test -``` -Lint all with [ESLint](https://github.com/typescript-eslint/typescript-eslint/): -``` -$ yarn lint -``` - diff --git a/packages/browser/architecture/ARCHITECTURE.md b/packages/browser/architecture/ARCHITECTURE.md new file mode 100644 index 000000000..b97e25cd9 --- /dev/null +++ b/packages/browser/architecture/ARCHITECTURE.md @@ -0,0 +1,222 @@ +## Analytics.js Plugin Architecture + +> [!IMPORTANT] +> This doc may get out-of-date. Please prefer to use and link to Segment documentation for the most up-to-date information. It would be advisable to move this doc to https://segment.com/docs/connections/sources/catalog/libraries/website/javascript, so there is a single source of truth. + + +### You can use the [vscode mermaid extension](https://marketplace.visualstudio.com/items?itemName=bierner.markdown-mermaid) to preview the following diagram code block inside of vscode, or copy and paste into the [mermaid live editor](https://mermaid.live/). + +## Table of Contents +- [Initialization Flow](#initialization-flow) +- [Plugin Types and Middleware](#plugin-types-and-middleware) + + +### Analytics.js Initialization Flow +The following diagram illustrates how Analytics.js bootstraps and initializes: + +```mermaid +graph TD + subgraph Load Phase + Start([Web Page Load]) --> LoadAJS[Fetch analytics.js
from CDN] + LoadAJS --> InitAJS[analytics.load
with writeKey] + InitAJS --> FetchSettings[Fetch settings
from cdn.segment.io] + end + + subgraph Plugin Registration Phase + FetchSettings --> RegisterCore[Register Core Plugins
and Middleware] + RegisterCore --> ParseDest[Parse Device Mode
Destinations] + end + + subgraph Destination Loading Phase + ParseDest --> LoadDests{Load Device Mode
Destinations} + LoadDests -->|Async| Dest1[Load Destination 1
+ Scripts] + LoadDests -->|Async| Dest2[Load Destination 2
+ Scripts] + LoadDests -->|Async| DestN[Load Destination N
+ Scripts] + end + + subgraph Event Processing Phase + Dest1 --> Ready1[Destination 1
Ready] + Dest2 --> Ready2[Destination 2
Ready] + DestN --> ReadyN[Destination N
Ready] + + Ready1 --> Flush1[Flush Events to
Destination 1] + Ready2 --> Flush2[Flush Events to
Destination 2] + ReadyN --> FlushN[Flush Events to
Destination N] + end + + RegisterCore -.->|Events Blocked
Until Complete| ParseDest + LoadDests -.->|Critical Plugins
Block Events| EventReady[Event Pipeline Ready] + + + + class Dest1,Dest2,DestN,Flush1,Flush2,FlushN async +``` + + +Key Points: +1. **Load Phase** + - Analytics.js is fetched from CDN + - Initialized with writeKey that identifies your Segment source + - CDN settings are fetched containing destination configurations + +2. **Plugin Registration Phase** + - Core plugins and middleware are registered synchronously + - Event queue is blocked until this phase completes + - Device mode destinations are identified from CDN settings + +3. **Destination Loading Phase** + - Device mode destinations are loaded in parallel + - Each destination loads its own third-party scripts + - Critical plugins (if any) must complete registration + +4. **Event Processing Phase** + - Destinations become ready independently + - Buffered events are flushed to each destination in parallel + - Non-critical destinations can receive events while others are still loading + + + +### Event Flow +```mermaid +graph TD + subgraph Event Creation + UserAction[analytics.track/page/identify] --> EventCreate[Event Factory] + EventCreate --> Queue[Event Queue] + end + + subgraph Plugin Pipeline + Queue --> BeforePlugins[Before Plugins e.g add page context] + BeforePlugins --> EnrichPlugins[Enrichment Plugins] + EnrichPlugins --> DestPlugins[Destination Plugins e.g Segment.io] + DestPlugins --> AfterPlugins[After Plugins] + end + + subgraph Plugin Types Details + BeforeDetails[Before Plugins
Priority: Critical
Example: Event Validation] --- BeforePlugins + EnrichDetails[Enrichment Plugins
Priority: Critical
Parallel Execution
Can Modify Events
Example: Add User Agent] --- EnrichPlugins + DestDetails[Destination Plugins
Parallel Execution
Cannot Modify Events
Example: Google Analytics] --- DestPlugins + AfterDetails[After Plugins
Example: Metrics Collection] --- AfterPlugins + end + + subgraph Notes + Priorities[Plugin Priorities] + Critical[Critical - Must load before
event delivery starts] + NonCritical[Non-Critical - Can load
after event delivery starts] + Priorities --> Critical + Priorities --> NonCritical + end +``` + + +### Plugin Types and Middleware +[This information is also available in the Segment documentation](https://segment.com/docs/connections/sources/catalog/libraries/website/javascript/#plugins-and-source-middleware) + +- **Source Middleware** (see [Example](#example-source-middleware-implementation)) + - **Source Middleware is just a light API wrapper around a "Before" type plugin Plugin** + - Source Middleware is the legacy API (pre-analytics next). It's less verbose than the full plugin API, but a bit less powerful. It is functionally equivalent to a "Before" type plugin. + +- **Before Plugins** (see [Example](#example-plugin-implementation)) + - Run before any other plugins + - Critical priority - block event pipeline until `.load()` resolves + - Use cases: Event validation, data transformation + - Example: Event validation before passing to other plugins) + + +- **Enrichment Plugins** + - Functionally Identitical to "before" plugins, but run after them. Before plugins are typically used internally (e.g adding page info), but there's no hard and fast rule. + +- **Destination Plugins** + - Run after enrichment + - Cannot modify the event + - Execute in parallel + - Failures do not halt pipeline + - Example: Segment.io, Google Analytics, Mixpanel + +- **After Plugins (uncommon)** + - Run after all other plugins complete + - Use cases: Metrics, logging + - Example: segment.io plugin for observability metrics + +- **Utility Plugins** + - Executes only once during the analytics.js bootstrap. Gives you access to the analytics instance using the plugin's load() method. This doesn't allow you to modify events. + - Do not directly process events + - Example: some plugin that registers a bunch of analytics event listeners (e.g. analytics.on('track', ...) and reports them to an external system) + +- **Destination Middleware** (See [Example](#example-destination-middleware-implementation)) + - A special type of plugin that allows you to add a plugin that only affects a specific (device mode) destination plugin. + + +### Example: Plugin Implementation +```ts +analytics.register({ + name: 'My Plugin', + type: 'before', + isLoaded: () => true, + load: () => Promise.resolve(), + // lowercase all track event names + track: (ctx) => { + ctx.event.event = ctx.event.event.toLowerCase() + return ctx + }, + // drop page events with a specific title + page: (ctx) => { + if (ctx.properties.title === 'some title') { + return null + } + return ctx + } +}) +``` +### Example: Source Middleware Implementation +```ts +analytics.addSourceMiddleware(({ payload, next }) => { + const { event } = payload.obj.context + if (event.type === 'track') { + // change the event name to lowercase + event.event = event.event.toLowerCase() + } else if (event.type === 'page') { + // drop any page events with a specific title + if (event.properties.title === 'some title') { + return null + } +} + next(payload) +}) +``` + +### Example: Destination Middleware Implementation +> [!NOTE] +> It is not currently possible to add a destination middleware to the Segment.io destination. +```ts +analytics.addDestinationMiddleware('amplitude', ({ next, payload }) => { + payload.obj.properties!.hello = 'from the other side' + next(payload) +}) +``` +or, to apply to all destinations +```ts +analytics.addDestinationMiddleware('*', ({ next, payload }) => { + ... +}) +``` + + +### Event Flow Example + +When `analytics.track()` is called: + +1. Event is created via Event Factory +2. Event enters the queue +3. Before plugins run, in order of registration +3. Enrichment plugins run, in order of registration +4. Destination plugins receive the event in parallel (including Segment.io plugin) +5. Any after plugins handle post-processing (e.g. metrics collection) + +### Plugin Priorities + +- **Critical Plugins**: Must be loaded before event delivery starts + - Example: Before plugins, Validation plugins +- **Non-Critical Plugins**: Can load after event delivery begins + - Example: Destination plugins + + diff --git a/packages/browser/architecture/architecture.png b/packages/browser/architecture/architecture.png new file mode 100644 index 000000000..1b4d77d71 Binary files /dev/null and b/packages/browser/architecture/architecture.png differ