Skip to content

Commit a457cd0

Browse files
committed
Fixing documentation for batch filtering case
1 parent a8fdb3e commit a457cd0

File tree

1 file changed

+26
-36
lines changed

1 file changed

+26
-36
lines changed

src/connections/functions/insert-functions.md

Lines changed: 26 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -371,50 +371,40 @@ The editor displays logs and request traces from the batch handler.
371371

372372
The [Public API](/docs/api/public-api) Functions/Preview endpoint also supports testing batch handlers. The payload must be a batch of events as a JSON array.
373373

374+
### Handling filtering in a batch
375+
Events in a batch can be filtered out using custom logic. The filtered events will be surfaced in the [Event Delivery](/docs/connections/event-delivery/) page with reason as `Filtered at insert function`
374376

375-
### Handling batching errors
377+
```js
378+
async function onBatch(events, settings) {
379+
let response = [];
380+
try {
381+
for (const event of events) {
382+
// some business logic to filter event. Here filtering out all the events with name `drop`
383+
if (event.properties.name === 'drop') {
384+
continue;
385+
}
376386

377-
Standard [function error types](/docs/connections/functions/destination-functions/#destination-functions-error-types) apply to batch handlers. Segment attempts to retry the batch in the case of Timeout or Retry errors. For all other error types, Segment discards the batch. It's also possible to report a partial failure by returning status of each event in the batch. Segment retries only the failed events in a batch until those events are successful or until they result in a permanent error.
387+
// some enrichments if needed
388+
event.properties.message = "Enriched from insert function";
378389

379-
```json
380-
[
381-
{
382-
"status": 200
383-
},
384-
{
385-
"status": 400,
386-
"errormessage": "Bad Request"
387-
},
388-
{
389-
"status": 200
390-
},
391-
{
392-
"status": 500,
393-
"errormessage": "Error processing request"
394-
},
395-
{
396-
"status": 500,
397-
"errormessage": "Error processing request"
398-
},
399-
{
400-
"status": 200
401-
},
402-
]
390+
// Enriched events are pushed to response
391+
response.push(event);
392+
}
393+
} catch (error) {
394+
console.log(error)
395+
throw new RetryError('Failed function', error);
396+
}
397+
398+
// return a subset of transformed event
399+
return response;
400+
}
403401
```
404402

405-
For example, after receiving the responses above from the `onBatch` handler, Segment only retries **event_4** and **event_5**.
406403

407-
| Error Type | Result |
408-
| ---------------------- | ------- |
409-
| Bad Request | Discard |
410-
| Invalid Settings | Discard |
411-
| Message Rejected | Discard |
412-
| RetryError | Retry |
413-
| Timeout | Retry |
414-
| Unsupported Event Type | Discard |
404+
### Handling batching errors
415405

406+
Standard [function error types](/docs/connections/functions/destination-functions/#destination-functions-error-types) apply to batch handlers. Segment attempts to retry the batch in the case of Timeout or Retry errors. For all other error types, Segment discards the batch.
416407

417-
{% comment %}
418408

419409
## Destination insert functions logs and errors
420410

0 commit comments

Comments
 (0)