-
Notifications
You must be signed in to change notification settings - Fork 393
More data retention policy edits #7581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -61,19 +61,23 @@ Select the default retention period for the workspace in this setting. This valu | |||||
With this data retention policy, all data beyond the retention period is unrecoverably deleted from all of Segment and impacts the following: | ||||||
|
||||||
* [Data Replays](/docs/guides/what-is-replay/) will only be available for data within the retention period. Unify, Engage and Linked customers that replay data to recreate Unify Spaces or Profiles may encounter variations in the number of profiles, as well as in the identifiers, traits and properties associated with the profiles, depending on the data available. | ||||||
* Backfill Data is only available for data within the retention period when sources are connected to your warehouse. | ||||||
* Backfill Data is only available for data within the retention period, when sources are connected to your warehouse. | ||||||
* [Data residency](/docs/guides/regional-segment/) migrations across regions (US and EU) is only available for data within the retention period. | ||||||
* Additional impacts to Object data: | ||||||
* [Object API](/docs/connections/sources/catalog/libraries/server/object-api/#set), [Bulk API](/docs/connections/sources/catalog/libraries/server/object-bulk-api/), or [SendGrid](/docs/connections/sources/catalog/cloud-apps/sendgrid/#streaming) and [Mandrill](/docs/connections/sources/catalog/cloud-apps/mandrill/#streaming) streaming sources: Any data older than 180 days is treated as a new record and may not contain any historic properties. To prevent loss of data properties, Segment recommends that you always send full objects with all properties. | ||||||
* [Object API](/docs/connections/sources/catalog/libraries/server/object-api/#set) or [Bulk API](/docs/connections/sources/catalog/libraries/server/object-bulk-api/): Object data not updated within the retention period will be deleted. Any new data will treated as a new record and may not contain any historic properties. To prevent loss of data properties, Segment recommends that you always send full objects with all properties. | ||||||
* Users and Accounts: Segment aggregates data from Identify and Group events into [Users and Account objects and tables for warehouse destinations](/docs/connections/storage/warehouses/schema/#warehouse-tables) object store records. Any object store records not updated in the last 180 days will be deleted from Segment's object stores. Any new data after object store records are deleted for inactivity is treated as a new object store record. If the source is connected to a Warehouse destination, object store entities are synced into [`<source>.users` and `<source>.accounts` tables](/docs/connections/storage/warehouses/schema/#warehouse-tables), and the existing record in the warehouse will be replaced with the new object store record, resulting in possible loss of attribute data. To prevent loss of attributes, Segment advises customers to migrate to using [Profiles Sync](/docs/unify/profiles-sync/overview/), always send complete Identify and Group calls, or back up your `<source>.users` and `<source>.accounts` tables. | ||||||
* [Computed traits](/docs/unify/Traits/computed-traits/) is built using the available data within the retention period. Recreating these traits may result in different values based on the available data. | ||||||
* [Profiles](/docs/unify/), [Engage](/docs/engage/) [Audiences](/docs/engage/audiences/) and [Journeys](/docs/engage/journeys/) that are built using Events will use available data within the retention period. Recreating these may result in different Profiles based on the available data. Depending on how the conditions are defined, Profiles may or may not exit Computed traits, Engage Audiences, and Journeys due to the data retention policy, and this may result in mismatches in counts when comparing against a preview. | ||||||
* [Profiles](/docs/unify/), [Engage](/docs/engage/) [Audiences](/docs/engage/audiences/) and [Journeys](/docs/engage/journeys/) that are built using Events will use available data within the retention period. Recreating these may result in different Profiles based on the available data. | ||||||
* [Real Time Computation](/docs/engage/audiences/#refresh-real-time-audiences-and-traits) (Audiences, Computed Traits, Journeys): When backfilling with historical data, backfill will use available data within the retention period. Once a computation is live, events that are removed due to data retention will not cause Profiles to enter/exit audiences and will not cause computed trait value changes. However, if you edit the definition or disable then re-enable them, this will cause the computation to re-backfill which will cause Profiles to enter/exit audiences and computed trait value to change. | ||||||
* [Batch Computation](/docs/engage/audiences/#real-time-compute-compared-to-batch) (Audiences, Computed Traits): Batch computation always computes based on available data, events removed due to data retention will cause Profile to enter/exit an Audience or computed trait values to change. | ||||||
|
||||||
|
||||||
### What data is not impacted? | ||||||
|
||||||
With this policy the following data is not impacted, but may be subject to other policies: | ||||||
|
||||||
* **[Object Cloud Sources](/docs/connections/sources/#object-cloud-sources)**: This involves Segment fetching object data from third party Cloud Sources. Since Segment always fetches the full objects, the retention policy will have no impact. | ||||||
* **[Object Cloud Sources](/docs/connections/sources/#object-cloud-sources)**: Segment fetches complete object data from third party Object Cloud Sources. Objects older than the retention period will be deleted. However, since Segment always fetches the complete object, Objects deleted will be fetched and made available again. | ||||||
* [Sendgrid](/docs/connections/sources/catalog/cloud-apps/sendgrid/) is both an Event Source and Object Source, therefore Events from Sendgrid have retention period applicable to Archive and Profile stores while Objects from Sendgrid have retention period applicable to the Object store retention period. | ||||||
|
* [Sendgrid](/docs/connections/sources/catalog/cloud-apps/sendgrid/) is both an Event Source and Object Source, therefore Events from Sendgrid have retention period applicable to Archive and Profile stores while Objects from Sendgrid have retention period applicable to the Object store retention period. | |
* [SendGrid](/docs/connections/sources/catalog/cloud-apps/sendgrid/) is both an Event Source and Object Source, therefore Events from Sendgrid have retention period applicable to Archive and Profile stores while Objects from Sendgrid have retention period applicable to the Object store retention period. |
forstisabella marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
forstisabella marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.