|
1 | | -## Q4 objectives |
| 1 | +## Q1 2026 objectives |
2 | 2 |
|
3 | | -### Implement the LLM Analytics Pipeline |
| 3 | +### Fence off non-ingestion services from persons DB |
4 | 4 |
|
5 | | -- Build a new endpoint to handle LLM analytics events with large contexts |
6 | | -- Include support for multi-modal models |
| 5 | +In order to have a single point of reference for all persons DB access so that any future changes to the underlying storage are seamless to the users. |
7 | 6 |
|
8 | | -### Define the event ingestion boundaries and enforce them |
| 7 | +- Persons Read API for all non-ingestion services |
| 8 | +- Partial Persons Write API for non-ingestion services |
9 | 9 |
|
10 | | -- Make it clear which use cases and traffic patterns are supported |
11 | | -- Enforce boundaries to improve the resilience of the ingestion pipeline |
| 10 | +### Introduce rate / usage limits |
12 | 11 |
|
13 | | -### Prototype Persons API |
| 12 | +So that certain patterns of system abuse are handled without manual intervention from the team. |
14 | 13 |
|
15 | | -- Encapsulate database access to improve maintainability, scalability and performance |
16 | | -- Enable more frequent and granular updates of person profiles |
| 14 | +- Rate Limiter API integrated into capture |
| 15 | +- Fair rate limits that don't upset customers |
| 16 | +- Public documentation on the rate limits |
17 | 17 |
|
18 | | -### Prototype multi-threaded ingestion |
| 18 | +### Move Error Tracking ingestion to ingestion platform |
19 | 19 |
|
20 | | -- Increase the maximum per-partition event throughput of the ingestion pipeline in Node.js |
21 | | -- Prepare for a possible future rewrite |
| 20 | +To simplify the ingestion codebase specific to Error Tracking, allow the team to reuse generic ingestion platform tooling, and reduce their maintenance burden. |
22 | 21 |
|
23 | | -### Do at least one thing that improves data quality for customers |
| 22 | +- Production-ready ingestion pipeline for Error Tracking that uses Cymbal |
24 | 23 |
|
25 | | -Potential options: |
26 | | -- Schema management |
27 | | -- Authorized events |
28 | | -- Better timestamp handling |
| 24 | +### Implement core features for the ingestion platform |
| 25 | + |
| 26 | +- Ingestion Pipelines Framework ready for broader rollout (docs, tutorials, tooling, etc.) |
| 27 | +- Better infrastructure for spinning up new ingestion pipelines (charts, automation, observability) |
| 28 | +- Tools for managing dead letter queues |
| 29 | +- Event restrictions available to all ingestion pipelines |
| 30 | +- Ability to create temporary ingestion pipelines for testing |
0 commit comments