PCSM-273: Implement document level parallel replication

[inelpandzic]

PCSM-273

Problem

Change replication processes events sequentially in a single goroutine — reading from the change stream, parsing BSON, and writing to the target all happen on the same thread. This becomes a bottleneck under high write throughput because bulk write latency on the target blocks the reader from consuming new events, increasing replication lag.

Additionally, all replication tuning parameters (worker count, queue sizes, bulk write batch size, change stream batch size) are hardcoded constants with no way to adjust them without rebuilding the binary.

Solution

Implement document-level parallel replication using a worker pool architecture. Events are read from the change stream by a single reader goroutine, then dispatched to workers based on a consistent hash of the document key. This preserves per-document ordering while parallelizing BSON parsing and bulk writes across multiple workers.

Key components:

• Worker pool with configurable number of workers (default: runtime.NumCPU())
• Document key hashing to route events — same document always goes to the same worker, preserving operation order
• Capped collection routing by namespace instead of document key to preserve insertion order
• Barrier mechanism for DDL and transaction events — flushes all workers before applying schema changes and transactions, then resumes
• Transaction handling — collects all events in a transaction and applies them as a single ordered bulk write
• Deferred DML parsing — raw BSON is passed to workers and parsed in parallel rather than in the single-threaded reader
• Staggered flush tickers — worker flush intervals are offset to spread writes evenly and avoid contention bursts

The previously hardcoded ReplQueueSize constant was split into two independent options since it controlled two architecturally different buffers: the dispatcher queue (between change stream reader and dispatcher) and the per-worker queue (between dispatcher and each worker).

All replication tuning parameters are now configurable via env vars, CLI flags, and HTTP API, following the same pattern as existing clone options.

New configuration options

All options default to 0 which means "auto" (use the built-in default).

CLI flag	Env var	HTTP API field	Default	Description
--repl-num-workers	PCSM_REPL_NUM_WORKERS	replNumWorkers	runtime.NumCPU()	Number of replication workers
--repl-change-stream-batch-size	PCSM_REPL_CHANGE_STREAM_BATCH_SIZE	replChangeStreamBatchSize	10000	Change stream batch size
--repl-event-queue-size	PCSM_REPL_EVENT_QUEUE_SIZE	replEventQueueSize	5000	Dispatcher event queue buffer size
--repl-worker-queue-size	PCSM_REPL_WORKER_QUEUE_SIZE	replWorkerQueueSize	5000	Per-worker routed event queue buffer size
--repl-bulk-ops-size	PCSM_REPL_BULK_OPS_SIZE	replBulkOpsSize	5000	Max operations per bulk write

[chupe]

QQ: I know it's not part of this PR but I'm just wondering why are we using a fresh context for each new logger?