Skip to content

Commit f157664

Browse files
committed
More content requested by PM
1 parent ad31f7e commit f157664

File tree

1 file changed

+172
-11
lines changed

1 file changed

+172
-11
lines changed

src/connections/functions/insert-functions.md

Lines changed: 172 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,19 @@ Use Destination Insert Functions to compute, transform, and enrich your data bef
88

99
**Implement advanced data computation**: Write custom computation, operations, and business logic on streaming data that you send to downstream destinations.
1010

11-
**Enrich your data**: Use insert functions with Segment's Profile API or third party sources to add additional context to your data and create personalized customer experiences.
11+
**Enrich your data**: Use destination insert functions with Segment's Profile API or third party sources to add additional context to your data and create personalized customer experiences.
1212

13-
**Support data compliance**: Use insert functions to support data masking, encryption, decryption, improved PII data handling, and tokenization.
13+
**Support data compliance**: Use destination insert functions to support data masking, encryption, decryption, improved PII data handling, and tokenization.
1414

1515
**Customize filtration for your destinations**: Create custom logic with nested if-else statements, regex, custom business rules, and more to filter event data.
1616

17-
> info "Destination Insert Functions is in Private Beta"
17+
> info "Private Beta"
1818
> Destination Insert Functions is in Private Beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available. [Contact Segment](https://segment.com/help/contact/){:target="_blank"} with any feedback or questions.
1919
2020

21-
## Create and manage destination insert functions
21+
## Create destination insert functions
2222

23-
There are two ways you can access Destination Insert Functions from your Segment space:
23+
There are two ways you can access destination insert functions from your Segment space:
2424
- From the Connections [catalog](#using-the-catalog).
2525
- From the [Destinations](#using-the-destinations-tab) tab.
2626

@@ -44,11 +44,11 @@ For data to flow to your downstream destinations, you'll need to connect your in
4444

4545
To access insert functions through the Destinations tab, navigate to **Connections > Destinations > Functions** and select your insert function. Use this page to edit and manage insert functions in your workspace.
4646

47-
You can also use this page to [enable insert functions](#enable-the-insert-function) in your workspace.
47+
You can also use this page to [enable destination insert functions](#enable-the-insert-function) in your workspace.
4848

49-
## Code the insert function
49+
## Code the destination insert function
5050

51-
Segment invokes a separate part of the function (called a "handler") for each event type that you send to your insert function.
51+
Segment invokes a separate part of the function (called a "handler") for each event type that you send to your destination insert function.
5252

5353
> info ""
5454
> Your function isn't invoked for an event if you've configured a [destination filter](/docs/connections/destinations/destination-filters/), and the event doesn't pass the filter.
@@ -94,7 +94,81 @@ To change which event type the handler listens to, you can rename it to the name
9494
> info ""
9595
> Functions' runtime includes a `fetch()` polyfill using a `node-fetch` package. Check out the [node-fetch documentation](https://www.npmjs.com/package/node-fetch){:target="_blank"} for usage examples.
9696
97-
## Testing the insert function
97+
### Errors and error handling
98+
99+
Segment considers a function's execution successful if it finishes without error. You can also `throw` an error to create a failure on purpose. Use these errors to validate event data before processing it, to ensure the function works as expected.
100+
101+
You can `throw` the following pre-defined error types to indicate that the function ran as expected, but that data was deliverable:
102+
103+
- `EventNotSupported`
104+
- `InvalidEventPayload`
105+
- `ValidationError`
106+
- `RetryError`
107+
108+
The examples show basic uses of these error types.
109+
110+
```js
111+
async function onGroup(event) {
112+
if (!event.traits.company) {
113+
throw new InvalidEventPayload('Company name is required')
114+
}
115+
}
116+
117+
async function onPage(event) {
118+
if (!event.properties.pageName) {
119+
throw new ValidationError('Page name is required')
120+
}
121+
}
122+
123+
async function onAlias(event) {
124+
throw new EventNotSupported('Alias event is not supported')
125+
}
126+
127+
async function onTrack(event) {
128+
let res
129+
try {
130+
res = await fetch('http://example-service.com/api', {
131+
method: 'POST',
132+
headers: {
133+
'Content-Type': 'application/json'
134+
},
135+
body: JSON.stringify({ event })
136+
})
137+
} catch (err) {
138+
// Retry on connection error
139+
throw new RetryError(err.message)
140+
}
141+
if (res.status >= 500 || res.status === 429) {
142+
// Retry on 5xx and 429s (ratelimits)
143+
throw new RetryError(`HTTP Status ${res.status}`)
144+
}
145+
}
146+
147+
```
148+
If you don't supply a function for an event type, Segment throws an `EventNotSupported` error by default.
149+
150+
You can read more about [error handling](#destination-functions-logs-and-errors) below.
151+
152+
## Create settings and secrets
153+
154+
{% include content/functions/settings.md %}
155+
156+
Next, fill out this setting's value in the **Test** tab, so you can run the function and verify that the correct setting value is passed. (This value is only for testing your function.)
157+
158+
Now that you've configured a setting and entered a test value, you can add code to read its value and run the function, as in the example below:
159+
160+
```js
161+
async function onTrack(request, settings) {
162+
const apiKey = settings.apiKey
163+
//=> "super_secret_string"
164+
}
165+
```
166+
167+
When you deploy your destination insert function in your workspace, you fill out the settings on the destination configuration page, similar to how you would configure a normal destination.
168+
169+
170+
## Test the destination insert function
171+
98172
{% comment %}
99173
You can test your code directly from the editor in two ways:
100174

@@ -103,13 +177,14 @@ Use a sample event:
103177
2. Select a destination or source to use events from. Then, select a sample event to use.
104178
2. Click **Use event**, and select **Run** from the **Test** tab.
105179
{% endcomment %}
180+
106181
You can manually test your code from the functions editor:
107182
1. From the **Test** tab, click **customize the event yourself** and manually input your own JSON payload.
108183
2. If your test fails, you can check the error details and logs in the Output section.
109184
- Error messages display errors surfaced from your function.
110185
- Logs display any messages to console.log() from the function.
111186

112-
## Save and deploy the insert function
187+
## Save and deploy the destination insert function
113188

114189
Once you finish building your insert function, click **Next: Configure and deploy** to name it, then click **Create function** to save it.
115190

@@ -122,7 +197,7 @@ You can also choose to **Save & Deploy** to save the changes, then choose which
122197
> info ""
123198
> You may need additional permissions to update existing functions.
124199
125-
## Enable the insert function
200+
## Enable the destination insert function
126201

127202
You need to enable your insert function for it to process your data.
128203

@@ -151,8 +226,71 @@ Note the following limitations for batching with insert functions:
151226

152227
## Destination Insert Functions logs and errors
153228

229+
A function can throw errors, or Segment might encounter errors while invoking your function. You can view these errors in the [Event Delivery](/docs/connections/event-delivery/) tab for your Destination as in the example below.
230+
231+
![A screenshot of the event delivery tab, showing 519 failed events broken into categories explaining why they failed](images/event-delivery.png)
232+
233+
### Destination functions error types
234+
235+
- **Bad Request** - Any error thrown by the function code that is not covered by the other errors.
236+
- **Invalid Settings** - A configuration error prevented Segment from executing your code. If this error persists for more than an hour, [contact Segment Support](https://segment.com/help/contact/){:target="_blank"}.
237+
- **Message Rejected** - Your code threw `InvalidEventPayload` or `ValidationError` due to invalid input.
238+
- **Unsupported Event Type** - Your code doesn't implement a specific event type (for example, `onTrack()`) or threw an `EventNotSupported` error.
239+
- **Retry** - Your code threw `RetryError` indicating that the function should be retried.
240+
241+
Segment only attempts to send the event to your destination function again if a **Retry** error occurs.
242+
243+
You can view Segment's list of [Integration Error Codes](/docs/connections/integration_error_codes/) for more information about what might cause an error.
244+
245+
### Destination functions logs
246+
247+
If your function throws an error, execution halts immediately. Segment captures the event, any outgoing requests/responses, any logs the function might have printed, as well as the error itself.
248+
249+
Segment then displays the captured error information in the [Event Delivery](/docs/connections/event-delivery/) page for your destination insert function. You can use this information to find and fix unexpected errors.
250+
251+
You can throw [an error or a custom error](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error){:target="_blank"} and you can also add helpful context in logs using the [`console` API](https://developer.mozilla.org/en-US/docs/Web/API/console){:target="_blank"}. For example:
252+
253+
```js
254+
async function onTrack(event, settings) {
255+
const userId = event.userId
256+
257+
console.log('User ID is', userId)
258+
259+
if (typeof userId !== 'string' || userId.length < 8) {
260+
throw new ValidationError('User ID is invalid')
261+
}
262+
263+
console.log('User ID is valid')
264+
}
265+
```
266+
267+
> warning ""
268+
> Don't log sensitive data, such as personally-identifying information (PII), authentication tokens, or other secrets. Avoid logging entire request/response payloads. The **Function Logs** tab may be visible to other workspace members if they have the necessary permissions.
269+
154270
## Caching in Destination Insert Functions
155271

272+
Functions execute only in response to incoming data, but the environments that functions run in are generally long-running. Because of this, you can use global variables to cache small amounts of information between invocations. For example, you can reduce the number of access tokens you generate by caching a token, and regenerating it only after it expires. Segment cannot make any guarantees about the longevity of environments, but by using this strategy, you can improve the performance and reliability of your Functions by reducing the need for redundant API requests.
273+
274+
This example code fetches an access token from an external API and refreshes it every hour:
275+
276+
```js
277+
const TOKEN_EXPIRE_MS = 60 * 60 * 1000 // 1 hour
278+
let token = null
279+
async function getAccessToken () {
280+
const now = new Date().getTime()
281+
if (!token || now - token.ts > TOKEN_EXPIRE_MS) {
282+
const resp = await fetch('https://example.com/tokens', {
283+
method: 'POST'
284+
}).then(resp => resp.json())
285+
token = {
286+
ts: now,
287+
value: resp.token
288+
}
289+
}
290+
return token.value
291+
}
292+
```
293+
156294
## Managing insert functions
157295

158296
### Functions permissions
@@ -166,6 +304,29 @@ If you are a **Workspace Owner** or **Functions Admin**, you can manage your fun
166304

167305
## Destination Insert Functions FAQs
168306

307+
##### Can I see who made changes to a function?
308+
309+
Yes, Functions access is logged in the [Audit Trail](/docs/segment-app/iam/audit-trail/), so user activity related to functions appears in the logs.
310+
311+
##### Does Segment retry failed function invocations?
312+
313+
Yes, Segment retries invocations that throw RetryError or Timeout errors (temporary errors only). Segment's internal system retries failed functions API calls for four hours with a randomized exponential backoff after each attempt. This substantially improves delivery rates.
314+
315+
[Retries](/docs/connections/destinations/#retries-between-segment-and-destinations) work the same for both functions and cloud-mode destinations in Segment.
316+
317+
##### Are events guaranteed to send data in order?
318+
319+
No, Segment can't guarantee the order in which the events are delivered to an endpoint.
320+
321+
##### Can I create a device-mode destination?
322+
323+
No, destination insert functions are currently available as cloud-mode destinations only. Segment is in the early phases of exploration and discovery for supporting customer "web plugins" for custom device-mode destinations and other use cases, but this is unsupported today.
324+
325+
##### How do I publish a destination to the public Segment catalog?
326+
327+
If you are a partner, looking to publish your destination and distribute your app through Segment catalog, visit the [Developer Center](https://segment.com/partners/developer-center/){:target="_blank"} and check out the Segment [partner docs](/docs/partners/).
328+
329+
169330
{% comment %}
170331

171332
## Using Segment's Public API

0 commit comments

Comments
 (0)