Audience membership update - Core + multiple destinations#3658
Audience membership update - Core + multiple destinations#3658joe-ayoub-segment wants to merge 27 commits intomainfrom
Conversation
New required fields detectedWarning Your PR adds new required fields to an existing destination. Adding new required settings/mappings for a destination already in production requires updating existing customer destination configuration. Ignore this warning if this PR is for a new destination with no active customers in production. The following required fields were added in this PR:
Add these new fields as optional instead and assume default values in |
There was a problem hiding this comment.
Pull request overview
This PR introduces a first-class audienceMembership signal in the Actions Core execution context (derived from Engage and RETL inputs) and updates the Amplitude Cohorts syncAudience action to rely on that signal instead of passing Engage-specific fields through the action payload.
Changes:
- Add
resolveAudienceMembershiphelper +AudienceMembershiptype, and injectaudienceMembershipintoExecuteInputforperform/performBatch. - Update Amplitude Cohorts
syncAudienceto acceptsegment_external_audience_iddirectly and useaudienceMembershipto decide ADD vs REMOVE. - Add unit/integration tests for
resolveAudienceMembershipand update existing Amplitude Cohorts tests to the new payload shape (partially).
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/destination-actions/src/destinations/amplitude-cohorts/syncAudience/index.ts | Passes audienceMembership through to the action’s send logic. |
| packages/destination-actions/src/destinations/amplitude-cohorts/syncAudience/generated-types.ts | Removes engage_fields from payload shape; adds segment_external_audience_id top-level. |
| packages/destination-actions/src/destinations/amplitude-cohorts/syncAudience/functions.ts | Uses audienceMembership values to build ADD/REMOVE batches. |
| packages/destination-actions/src/destinations/amplitude-cohorts/syncAudience/fields.ts | Updates mapping to only pull segment_external_audience_id from context.personas. |
| packages/destination-actions/src/destinations/amplitude-cohorts/syncAudience/tests/index.test.ts | Updates main test suite payload expectations/mappings for new payload shape. |
| packages/core/src/index.ts | Re-exports AudienceMembership type and resolveAudienceMembership. |
| packages/core/src/destination-kit/types.ts | Adds audienceMembership to ExecuteInput and defines AudienceMembership. |
| packages/core/src/destination-kit/action.ts | Computes and injects audienceMembership in perform / performBatch. |
| packages/core/src/audience-membership.ts | New helper to resolve audience membership from Engage + RETL inputs. |
| packages/core/src/tests/audience-membership.test.ts | New tests for membership resolution and ExecuteInput injection. |
| if (this.definition.performBatch) { | ||
| const syncMode = this.definition.syncMode ? bundle.mapping?.['__segment_internal_sync_mode'] : undefined | ||
| const syncModeVal = this.definition.syncMode ? bundle.mapping?.['__segment_internal_sync_mode'] : undefined | ||
| const syncMode = isSyncMode(syncModeVal) ? syncModeVal : undefined | ||
| const matchingKey = bundle.mapping?.['__segment_internal_matching_key'] | ||
| const audienceMembership: AudienceMembership[] = bundle.data.map((d) => resolveAudienceMembership(d, syncMode)) | ||
|
|
There was a problem hiding this comment.
In executeBatch, audienceMembership is computed from bundle.data (original raw events) even though payloads may be filtered down after schema validation. If any payloads are dropped as invalid, the membership array will no longer line up with the payload array indices passed into performBatch, causing add/remove decisions to be applied to the wrong payloads. Consider building a filtered audienceMembership array alongside filteredPayload during the validation loop (only pushing membership for indices that remain in payloads).
| it('returns true when the user is being added to the audience', () => { | ||
| expect( | ||
| resolveAudienceMembership({ | ||
| context: { personas: { computation_class: 'audience', computation_key: 'my_audience' } }, | ||
| properties: { my_audience: true } | ||
| }) | ||
| ).toBe(true) | ||
| }) | ||
|
|
||
| it('returns false when the user is being removed from the audience', () => { | ||
| expect( | ||
| resolveAudienceMembership({ | ||
| context: { personas: { computation_class: 'audience', computation_key: 'my_audience' } }, | ||
| properties: { my_audience: false } | ||
| }) | ||
| ).toBe(false) |
There was a problem hiding this comment.
These unit tests call resolveAudienceMembership without an event type, and expect membership to be inferred from properties. The current implementation only resolves Engage membership when type is identify (from traits) or track (from properties), so these assertions will fail. Update the test inputs to include type and put the boolean on traits for identify (or switch to track + properties).
| event: { | ||
| type: 'identify', | ||
| userId: 'user-1', | ||
| context: { | ||
| personas: { computation_class: 'audience', computation_key: 'my_audience' } | ||
| }, | ||
| properties: { my_audience: true } | ||
| } |
There was a problem hiding this comment.
These integration tests use an identify event with the audience membership boolean in properties, but engageAudienceMembership reads traits[computation_key] for identify events. As written, capturedData.audienceMembership will be undefined and the expectations will fail. Put the boolean on traits for identify (or change the event type to track if you want to keep properties).
| /** | ||
| * Fields from the identify() or track() call emitted by Engage. This is used to determine whether the user should be added or removed from the Amplitude Cohort. | ||
| * Hidden field containing the Cohort ID which was returned when the Amplitude Cohort was created in the Audience Settings. | ||
| */ | ||
| engage_fields: { | ||
| /** | ||
| * Hidden field used to verify that the payload is generated by an Engage Audience. Payloads not containing computation_class = 'audience' or 'journey_step' will be dropped before the perform() fuction call. | ||
| */ | ||
| segment_computation_class: string | ||
| /** | ||
| * Traits or Properties object from the identify() or track() call emitted by Engage | ||
| */ | ||
| traits_or_properties: { | ||
| [k: string]: unknown | ||
| } | ||
| /** | ||
| * Hidden field used to determine whether to add or remove the user from the Amplitude Cohort. | ||
| */ | ||
| segment_audience_key: string | ||
| /** | ||
| * Hidden field containing the Cohort ID which was returned when the Amplitude Cohort was created in the Audience Settings. | ||
| */ | ||
| segment_external_audience_id: string | ||
| } | ||
| segment_external_audience_id: string | ||
| /** |
There was a problem hiding this comment.
Payload.engage_fields was removed from the generated types and replaced with top-level segment_external_audience_id. There are still existing tests in this action (e.g. __tests__/functions.test.ts and __tests__/getIds.test.ts) that construct Payload objects using engage_fields, which will no longer type-check/compile. Those tests need to be updated to the new payload shape (and any assertions updated accordingly).
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3658 +/- ##
==========================================
+ Coverage 80.49% 80.91% +0.42%
==========================================
Files 1320 1378 +58
Lines 24433 27508 +3075
Branches 4987 5906 +919
==========================================
+ Hits 19667 22259 +2592
- Misses 3859 4305 +446
- Partials 907 944 +37 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| const audienceMembership: AudienceMembership[] = audienceMembershipEnabled | ||
| ? bundle.data.map((d) => resolveAudienceMembership(d, syncMode)) | ||
| : [] | ||
|
|
||
| const data = { | ||
| rawData: bundle.data, |
There was a problem hiding this comment.
In executeBatch, payloads can be filtered down when schema validation fails, but audienceMembership is still built from bundle.data (the original, unfiltered events array). That can misalign membership values with payloads indices, causing users to be added/removed incorrectly when any payloads were filtered out. Build audienceMembership (and ideally rawData) from the same filtered set as payloads (e.g., filter bundle.data using invalidPayloadIndices before computing membership and passing rawData into performBatch).
| const audienceMembership: AudienceMembership[] = audienceMembershipEnabled | |
| ? bundle.data.map((d) => resolveAudienceMembership(d, syncMode)) | |
| : [] | |
| const data = { | |
| rawData: bundle.data, | |
| // Align rawData and audienceMembership with the filtered payloads by | |
| // removing entries whose indices were marked invalid during schema validation. | |
| const filteredData = Array.isArray(bundle.data) | |
| ? bundle.data.filter((_, index) => !invalidPayloadIndices.includes(index)) | |
| : bundle.data | |
| const audienceMembership: AudienceMembership[] = audienceMembershipEnabled | |
| ? filteredData.map((d) => resolveAudienceMembership(d, syncMode)) | |
| : [] | |
| const data = { | |
| rawData: filteredData, |
| const msResponse = new MultiStatusResponse() | ||
|
|
||
| if(!audience_key){ | ||
| return failAllPayloads(payloads, msResponse, isBatch, 'Segment Audience Key is a required field.') | ||
| if(!Array.isArray(audienceMemberships) ){ | ||
| return failAllPayloads(payloads, msResponse, isBatch, 'Audience membership details missing') | ||
| } |
There was a problem hiding this comment.
The error message used when audienceMemberships is missing/invalid is fairly generic and inconsistent with the later per-payload message ("Audience Membership Details missing"). Since this will surface to customers, consider using a single consistent message and making it actionable (e.g., explain that this action must be triggered by an Engage audience/journey_step computation or a RETL sync event so membership can be determined).
joe-ayoub-segment
changed the title
| @@ -57,6 +58,8 @@ export interface ExecuteInput< | |||
| readonly audienceSettings?: AudienceSettings | |||
| /** The transformed input data, based on `mapping` + `event` (or `events` if batched) */ | |||
| payload: Payload | |||
| /** Whether the user is being added to (true) or removed from (false) an audience. Undefined for non-audience events. For batch actions, this is an array matching the payload array. */ | |||
| audienceMembership?: AudienceMembershipType | |||
There was a problem hiding this comment.
We could actually remove AudienceMembershipType = AudienceMembership | AudienceMembership[] and infer from payload if this should be array or not.
audienceMembership?: Payload extends unknown[] ? AudienceMembership[] : AudienceMembership
joe-ayoub-segment
changed the title
Summary
This PR introduces a centralized audienceMembership resolution mechanism in actions-core and migrates five audience destinations to consume it. Instead of each destination independently parsing Engage
traits/properties or RETL sync-mode event names to determine add vs. remove, the Core framework now computes a typed boolean | undefined value and injects it into every action's ExecuteInput. All destination
changes are gated behind per-destination feature flags with full legacy fallback.
Core Changes (packages/core)
New: audience-membership.ts
Centralizes the logic for resolving audience membership from raw event data:
New: flags.ts
A single source-of-truth for feature flag names used across Core and destinations:
FLAGS.ACTIONS_CORE_AUDIENCE_MEMBERSHIP // master gate
FLAGS.ACTIONS_GOOGLE_EC_AUDIENCE_MEMBERSHIP
FLAGS.ACTIONS_BRAZE_COHORTS_AUDIENCE_MEMBERSHIP
FLAGS.ACTIONS_LINKEDIN_AUDIENCES_AUDIENCE_MEMBERSHIP
(Facebook Custom Audiences uses only the master flag.)
Updated: destination-kit/types.ts
Updated: destination-kit/action.ts
Updated: index.ts
Exports AudienceMembership, resolveAudienceMembership, and FLAGS for use by destinations.
Destination Changes
Amplitude Cohorts
Flag: ACTIONS_CORE_AUDIENCE_MEMBERSHIP (master flag only — amplitude has no per-destination flag; membership comes from Core unconditionally once the master flag is on)
top-level segment_external_audience_id field. This simplifies the mapping and removes the need for each payload to carry audience key / traits data.
PAYLOAD_VALIDATION_FAILED.
Braze Cohorts
Flags: ACTIONS_CORE_AUDIENCE_MEMBERSHIP AND ACTIONS_BRAZE_COHORTS_AUDIENCE_MEMBERSHIP (both required)
audienceMembership instead of event_properties[personas_audience_key] for add/remove routing. Legacy path preserved when flags are off.
Facebook Custom Audiences
Flag: ACTIONS_CORE_AUDIENCE_MEMBERSHIP (master flag only)
This destination received the most extensive refactor:
Google Enhanced Conversions (userList)
Flags: ACTIONS_CORE_AUDIENCE_MEMBERSHIP AND ACTIONS_GOOGLE_EC_AUDIENCE_MEMBERSHIP (both required)
LinkedIn Audiences
Flags: ACTIONS_CORE_AUDIENCE_MEMBERSHIP AND ACTIONS_LINKEDIN_AUDIENCES_AUDIENCE_MEMBERSHIP (both required)
Feature Flags
Amplitude Cohorts and Facebook Custom Audiences use only the master flag. All destinations fall back to their pre-existing behavior when flags are off, ensuring zero regression risk during rollout.
Unit Test Coverage
legacy.test.ts (295 lines), functions.test.ts (342 lines), audience-creation.test.ts (135 lines), canary.test.ts (175 lines)