Description

When run in writer mode, iota-indexer launches two concurrent pipelines that rely on checkpoint data:

1. The main ingestion pipeline: Responsible for updating objects, transactions, and their indices
2. The object snapshots pipeline that captures an enriched snapshot of the ledger in the past. By default the lag of the snapshots from the current checkpoint number is 300 checkpoints.

These pipelines are coupled in using the same buffer of checkpoint data. As a consequence, the buffer size becomes an upper bound of the delay that can be sustained without one pipeline blocking the other. And hence it limits the effective lag that can be configure between snapshots and the current ledger state.

We would like to lift this restraint by using different buffers (different executors in data-ingestion-core jargon) for these two pipelines.