You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/connections/destinations/destination-filters.md
+41-33Lines changed: 41 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@ title: Destination Filters
3
3
rewrite: true
4
4
---
5
5
6
-
Destination Filters allow you to control the data flowing into each specific destination, by conditionally preventing data from being sent to cloud-mode destinations. You can filter out entire events, or just specific fields in the properties, traits, or in the context of your events.
6
+
Destination Filters allow you to control the data flowing into each specific destination by conditionally preventing data from being sent to cloud-mode destinations. You can filter out entire events, or just specific fields in the properties, traits, or in the context of your events.
7
7
8
8
With Destination Filters, you can:
9
9
@@ -19,25 +19,37 @@ With Destination Filters, you can:
19
19
20
20
### Destination Filtering Limitations
21
21
22
-
- Destination Filters can only be applied to Cloud-mode ("server-side") streaming destinations. Device-mode destinations are not supported.
23
-
- You cannot apply Destination Filters to Warehouses or S3 destinations.
22
+
- Destination Filters can only be applied to Cloud-mode ("server-side") streaming destinations. Device-mode destinations aren't supported.
23
+
- You can't apply Destination Filters to Warehouses or S3 destinations.
24
24
- Each filter can only apply to one source-destination pair.
25
25
26
-
If you have a compelling use case for these unsupported options, [contact us](https://segment.com/help/contact/).
26
+
If you have a compelling use case for these unsupported options, [contact Segment](https://segment.com/help/contact/).
27
+
28
+
## Create a Destination Filter
29
+
To create a Destination Filter:
30
+
1. Go to **Connections > Destinations** and select your destination.
31
+
2. Click on the **Filters** tab of your destination.
32
+
3. Click **+ New Filter**.
33
+
4. Configure the rules for your filter.
34
+
5.*(Optional)* Click **Load Sample Event** to test your filter to see if the event passes through your filter.
35
+
6. Click **Next Step**.
36
+
7. Name your filter and click the toggle to enable it.
37
+
6. Click **Save**.
27
38
28
39
## Destination Filters API
29
40
30
-
The Destination Filters API provides even more power than the Destination
41
+
The Destination Filters API provides more power than the Destination
31
42
Filters settings in the Segment dashboard. You can create complex filters
32
43
that are conditionally applied using Segment's "Filter Query Language" (FQL).
33
44
34
-
The Destination Filters API offers four different types of filters:
45
+
The Destination Filters API offers these 4 different types of filters:
35
46
36
-
1.**drop_event**: Do not send matched event to the destination.
37
-
2.**sample_event**: Send only a percentage of events through to the
38
-
destination.
39
-
3.**whitelist_fields**: Only send whitelisted properties to destination.
40
-
4.**blocklist_fields**: Do not send blocklisted properties to destination.
47
+
Filter | Details
48
+
------ | -------
49
+
`drop_event` | Don't send matched events to the destination.
50
+
`sample_event` | Send only a percentage of events through to the destination.
51
+
`whitelist_fields` | Only send whitelisted properties to the destination.
52
+
`blocklist_fields` | Don't send blocklisted properties to the destination.
41
53
42
54
Read more in the [Destination Filters API docs](https://reference.segmentapis.com/#6c12fbe8-9f84-4a6c-848e-76a2325cb3c5).
43
55
@@ -47,8 +59,7 @@ Read more in the [Destination Filters API docs](https://reference.segmentapis.co
47
59
48
60
Example: Remove email addresses from `context` and `properties`:
49
61
50
-
Property-level whitelisting is available using our API. Use this to do things
51
-
like only send certain traits you know have no PII to a destination, and block
62
+
Property-level whitelisting is available using Segment's API. Use this to perform actions like only send certain traits you know have no PII to a destination, and block
52
63
all other traits in the context or property fields.
53
64
54
65
@@ -61,80 +72,77 @@ Example: Only send user signed up and demo requested events:
61
72
62
73
### Cleaner Data
63
74
64
-
Example: only send events - only send track calls to Google Analytics:
75
+
Example: Only send events - only send track calls to Google Analytics:
65
76
66
77
67
78
68
79
### Remove Internal and Test Events From Production Tools
69
80
70
-
Example: Do not send events when email contains `@segment.com`:
81
+
Example: Don't send events when the email contains `@segment.com`:
[This video](https://www.youtube.com/watch?v=47dhAF1Hoco) shows an example of
88
-
filtering events being sent to a destination based on the name of the event.
99
+
filtering events sent to a destination based on the name of the event.
89
100
90
101
## Important Notes
91
102
92
103
**Conflicting settings**
93
104
94
105
Some destinations offer settings that also allow you to filter data. For example, the Facebook App Events destination allows you to map `Screen` events to `Track` events. Because Destination Filters are evaluated and applied _before_ the Destination settings are applied, they can conflict with your settings.
95
106
96
-
In the example in the video above, if you have a Destination Filter that only allows Track events _and_ you have the "Use Screen Events as Track Events" setting enabled, `Track` events are dropped, but `Screen` events are still processed and are transformed by the destination settings into a `Track` event - *after* the filters.
107
+
In the example in the video above, if you have a Destination Filter that only allows Track events _and_ you have the **Use Screen Events as Track Events** setting enabled, `Track` events are drop, but `Screen` events still process, and the destination settings transform it into a `Track` event - *after* the filters.
97
108
98
109
**Error handling**
99
110
100
-
We make every effort to ensure that Destination Filters handle unexpected
101
-
situations gracefully. For example, if you use the `contains()` FQL function on
102
-
a field and that field is `null`, we return`false` instead of returning an error. If we can't reasonably infer your intent, we log an internal error, and drop the event. We default to this behavior to prevent sensitive information, for example from a PII filter, from getting through.
111
+
Segment makes effort to ensure that Destination Filters handle unexpected
112
+
situations. For example, if you use the `contains()` FQL function on
113
+
a field and that field is `null`, Segment returns`false` instead of returning an error. If Segment can't reasonably infer your intent, Segment logs an internal error, and drops the event. Segment defaults to this behavior to prevent sensitive information, for example from a PII filter, from getting through.
103
114
104
-
Errors are not exposed in the Event Deliverability tab of your Destination. For help diagnosing unexpectedly missing events when using Destination Filters, [contact us](https://segment.com/help/contact/).
115
+
Errors aren't exposed in the Event Deliverability tab of your Destination. For help diagnosing unexpectedly missing events when using Destination Filters, [contact Segment](https://segment.com/help/contact/).
105
116
106
117
## FAQ
107
118
108
119
**Q: How does destination filters work with array properties?**
109
120
110
121
Destination Filters can filter properties out of objects nested in an array. For
111
-
example, you could filter out the `price` property of every object in an array at `properties.products`. You can also filter out an entire array from the payload. However, you cannot drop nested objects in an array, or filter
112
-
properties out of a single object in an array.
122
+
example, you could filter out the `price` property of every object in an array at `properties.products`. You can also filter out an entire array from the payload. However, you can't drop nested objects in an array, or filter properties out of a single object in an array.
113
123
114
124
**Q: How many filters can I create?**
115
125
116
-
You are currently limited to 10 filters per destination. If you need help
117
-
consolidating filters or would like to discuss your use case, [contact us](https://segment.com/help/contact/)!
126
+
You are limited to 10 filters per destination. If you need help
127
+
consolidating filters or would like to discuss your use case, [contact Segment](https://segment.com/help/contact/)!
118
128
119
129
**Q: When will you support warehouses?**
120
130
121
-
If you want to filter out events from warehouses, the best way to do that today
122
-
is with the existing Selective Sync feature.
131
+
If you want to filter out events from warehouses, the best way to do that is with the existing Selective Sync feature.
123
132
124
-
**Q: I don't see a "name" property at the top level of my events to filter on "event name"!**
133
+
**Q: I don't see a "name" property at the top level of my events to filter on "event name".**
125
134
126
135
Generally, only Track calls have "name" properties, which corresponds to the
127
136
"Event" field in an event.
128
137
129
138
**Q: How can I find out when new filters have been added or removed from a destination?**
130
139
131
-
The Activity Feed shows the action, date, and user who performed the action when a Destination Filter is created, modified, enabled, disabled, or deleted. You can also subscribe to notifications for any of these changes in the
132
-
Activity Feed settings page.
140
+
The Activity Feed shows the action, date, and user who performed the action when a Destination Filter is created, modified, enabled, disabled, or deleted. You can also subscribe to notifications for any of these changes in the **Activity Feed** settings page.
133
141
134
142
**Q: Why am I getting a permissions denied error when trying to save a filter?**
135
143
136
144
Anyone with `read` access on the source can view and test filters, but only users with `write` access can edit filters.
137
145
138
146
**Q: How can I test that my filter is working?**
139
147
140
-
Use the Destination Filter tester (in the destination filter set up screens) to test if you are filtering out the right events. Filtered events still show up in the schema page, but are not counted in the Event Deliverability graphs.
148
+
Use the Destination Filter tester (in the destination filter set up screens) to test if you are filtering out the right events. Filtered events still show up in the schema page, but aren't counted in the Event Deliverability graphs.
Copy file name to clipboardExpand all lines: src/connections/sources/catalog/libraries/website/javascript/index.md
+119-4Lines changed: 119 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ Analytics.js 2.0 provides a reduction in page load time, which improves site per
28
28
29
29
### Developer experience
30
30
31
-
Analytics.js 2.0 improves developer experience by introducing new ways for developers to augment events throughout the event timeline. For example, developers can augment events either before or after an event occurs, or while the event is in-flight.
31
+
Analytics.js 2.0 improves developer experience by introducing new ways for developers to augment events throughout the event timeline. For example, developers can augment events either before or after an event occurs, or while the event is in-flight.
32
32
33
33
For example, you can use the Analytics.js 2.0 to build features that:
34
34
@@ -656,6 +656,121 @@ When enabled, Analytics.js automatically retries network and server errors. With
656
656
657
657
Analytics.js stores events in `localStorage` and falls back to in-memory storage when `localStorage` is unavailable. It retries up to 10 times with an incrementally increasing back-off time between each retry. Analytics.js queues up to 100 events at a time to avoid using too much of the device's local storage. See the [destination Retries documentation](/docs/connections/destinations/#retries) to learn more.
658
658
659
+
660
+
## Batching
661
+
Batching is the ability to group multiple requests or calls into one request or API call. All requests sent within the same batch have the same `receivedAt` time. With Analytics.js, you can send events to Segment in batches. Sending events in batches enables you to have:
662
+
- Delivery of multiple events with fewer API calls
663
+
- Fewer errors if a connection is lost because an entire batch will retry at once rather than multiple calls retrying at random times.
664
+
665
+
### Setup
666
+
You can start batching by changing the `strategy` to `"batching"` and the parameters for `size` and `timeout` within the `load` method in the analytics object. Batching requires both parameters.
667
+
668
+
```js
669
+
analytics.load("<write_key>", {
670
+
integrations: {
671
+
"Segment.io": {
672
+
deliveryStrategy: {
673
+
strategy:"batching",
674
+
config: {
675
+
size:10,
676
+
timeout:5000
677
+
}
678
+
}
679
+
}
680
+
}
681
+
});
682
+
```
683
+
684
+
You can check to see if batching works by checking your source’s debugger in **Sources > Debugger**. When you select an event and view the **Raw** code, the `receivedAt` time of all the events in the batch should be the same.
685
+
686
+
#### Batch size
687
+
The batch size is the threshold that forces all batched events to be sent once it’s reached. For example, `size: 10` means that after triggering 10 events, Analytics.js sends those 10 events together as a batch to Segment.
688
+
689
+
Your total batched events can’t exceed the maximum payload size of 500 KB, with a limit of 32 KB for each event in the batch. If the 500 KB limit is reached, the batch will be split.
690
+
691
+
#### Timeout
692
+
`timeout` is the number of milliseconds that forces all events queued for batching to be sent, regardless of the batch size, once it’s reached. For example, `timeout: 5000` sends every event in the batch to Segment once 5 seconds passes.
693
+
694
+
### Implicit and explicit batching
695
+
All batching is done implicitly as it doesn’t require you to change your Analytics.js implementation. Implicit batching is a great option is you want an out of the box approach to batching as you only need to add the required parameters to your load method (mentioned above in Setup) and Analytics.js will handle the rest.
696
+
697
+
You can fire your analytics events as usual and assume they’ll be delivered the same as individual events would. This code block shows implicit batching:
698
+
699
+
```js
700
+
701
+
// Implicit Batching
702
+
703
+
analytics.page('Checkout')
704
+
analytics.identify('User ID')
705
+
analytics.track('Checkout seen')
706
+
707
+
// some time later
708
+
analytics.track('Product Added')
709
+
analytics.track('Checkout clicked')
710
+
711
+
// These 5 events will potentially be delivered in the same batch, depending on how far apart they're executed (based on batching config)
712
+
```
713
+
714
+
If you want to guarantee which specific events are delivered together in the batch, you can set up explicit batching by using `await Promise` and changing the implicit code example above to this:
715
+
716
+
```js
717
+
constcheckout=analytics.page('Checkout')
718
+
constidentify=analytics.identify('User ID')
719
+
constseen=analytics.track('Product Added')
720
+
721
+
// some time later
722
+
constadded=analytics.track('Product Added')
723
+
constclicked=analytics.track('Checkout clicked')
724
+
725
+
// Collecting all promises and awaiting on them will create an artificial batch
726
+
// This line will block execution until all events in the artificial batch have been delivered
In the case where all events are delivered without any user behavior, you can guarantee delivery with explicit batching as shown in this example code:
730
+
731
+
```js
732
+
// Will wait for all events to be delivered before moving on.
733
+
awaitPromise.all([
734
+
analytics.page('Checkout'),
735
+
analytics.identify('User ID'),
736
+
analytics.track('Product Added')
737
+
])
738
+
739
+
// some time later
740
+
analytics.track('Product Added')
741
+
analytics.track('Checkout clicked')
742
+
```
743
+
744
+
### Batching FAQs
745
+
#### Will Analytics.js deliver events that are in the queue when a user closes the browser?
746
+
Analytics.js does its best to deliver the queued events before the browser closes, but the delivery isn’t guaranteed.
747
+
748
+
Upon receiving the `beforeunload` browser event, Analytics.js attempts to flush the queue using `fetch` requests with `keepalive` set to true. Since the max size of `keepalive` payloads is limited to 64 KB, if the queue size is bigger than 64 KB at the time the browser closes, then there is a chance of losing a subset of the queued events. Reducing the batch size or timeout will alleviate this issue, but that will be a trade-off decision.
749
+
750
+
#### Is Batching supported on Analytics.js classic?
751
+
No, Batching is only supported as part of Analytics.js 2.0.
752
+
753
+
#### Can other destinations receive batched events?
754
+
No, this batching only impacts events sent to Segment. Once the batch reaches Segment, it is split up and follows the normal path of an event.
755
+
756
+
#### Will batching impact billing or throughput?
757
+
No, batching won’t impact billing or throughput.
758
+
759
+
#### Can I use batching with partner integrations?
760
+
Partner integrations don’t support batching as all other partner integrations run one event at a time. Only Segment.io events support batched delivery.
761
+
762
+
#### Does batching work on all browsers?
763
+
Batching won’t work on Internet Explorer.
764
+
765
+
#### If a source has retry enabled, does the retry behavior change when using batching?
766
+
Batching delays retries, as events that are queued for batching aren’t retried until a batch delivery fails.
767
+
768
+
#### When using Middlewares as a source and destination, is there a change in behavior when using batching?
769
+
No, there is no change in behavior to Middlewares.
770
+
771
+
#### When using Segment features (Schema filtering, integrations object, Protocols) to filter events from going to destinations (device and cloud-mode), will batching impact the filtering of events?
772
+
No, there is no impact to how events filter.
773
+
659
774
## Plugins
660
775
661
776
Segment offers video player 'plugins' so you can quickly collect video events using Analytics.js. See the specific documentation below to learn more:
License: MIT License, available at: [https://github.com/developit/unfetch/blob/master/LICENSE.md](https://github.com/developit/unfetch/blob/master/LICENSE.md)
845
+
License: MIT License, available at: [https://github.com/developit/unfetch/blob/master/LICENSE.md](https://github.com/developit/unfetch/blob/master/LICENSE.md)
0 commit comments