Skip to content

Commit d54fd72

Browse files
sczemborsczembor
authored
fix(btcindexer): isolate minted and reorged txs by network (#355)
* netowrk insolation + unit tests Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * use setup id Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * fix tests Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * formatting Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * fix markdown formatting Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * apply code review suggestions Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * apply code review suggestions Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * update tests Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * apply copilot suggestions Signed-off-by: sczembor <stanislaw.czembor@gmail.com> * copilot suggestions Signed-off-by: sczembor <stanislaw.czembor@gmail.com> --------- Signed-off-by: sczembor <stanislaw.czembor@gmail.com> Signed-off-by: sczembor <43810037+sczembor@users.noreply.github.com>
1 parent 1ca42d0 commit d54fd72

File tree

14 files changed

+695
-254
lines changed

14 files changed

+695
-254
lines changed

‎AGENTS.md‎

Lines changed: 25 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ This is a Cloudflare Workers project that implements various services for [Go Na
66
It uses [Bun](https://bun.com/) for JavaScript and Typescript runtime and package management (instead of Nodejs + npm).
77

88
The content is organized into a Bun workspace. See @README.md for:
9+
910
- High-level architecture and component interactions
1011
- Detailed setup instructions
1112
- Functional flow documentation
@@ -15,13 +16,13 @@ The content is organized into a Bun workspace. See @README.md for:
1516

1617
All packages are in the `./packages` directory:
1718

18-
| Package | Type | Purpose |
19-
|---------|------|---------|
20-
| `btcindexer` | Service (Worker) | Bitcoin-to-Sui bridging and minting |
21-
| `sui-indexer` | Service (Worker) | Sui blockchain monitoring and redemption |
22-
| `block-ingestor` | Service (Worker) | Receives Bitcoin blocks via REST API |
23-
| `compliance` | Service (Worker) | Sanctions and geo-blocking data |
24-
| `lib` | Shared Library | Common utilities and types |
19+
| Package | Type | Purpose |
20+
| ---------------- | ---------------- | ---------------------------------------- |
21+
| `btcindexer` | Service (Worker) | Bitcoin-to-Sui bridging and minting |
22+
| `sui-indexer` | Service (Worker) | Sui blockchain monitoring and redemption |
23+
| `block-ingestor` | Service (Worker) | Receives Bitcoin blocks via REST API |
24+
| `compliance` | Service (Worker) | Sanctions and geo-blocking data |
25+
| `lib` | Shared Library | Common utilities and types |
2526

2627
### Core Technologies
2728

@@ -96,7 +97,7 @@ See @README.md for Cloudflare RPC documentation and examples.
9697

9798
**Location**: `./packages/btcindexer`
9899

99-
### Architecture
100+
### BTCIndexer Architecture
100101

101102
- `src/index.ts` - Main entry with HTTP handlers and scheduled cron
102103
- `src/btcindexer.ts` - Bitcoin indexing and deposit detection logic
@@ -105,14 +106,14 @@ See @README.md for Cloudflare RPC documentation and examples.
105106
- `src/cf-storage.ts` - Cloudflare D1/KV storage layer
106107
- `src/bitcoin-merkle-tree.ts` - Merkle proof generation
107108

108-
### Key Features
109+
### BTCIndexer Key Features
109110

110111
1. **Bitcoin Block Processing**: Cron job runs every minute to scan blocks, identify nBTC deposits via OP_RETURN outputs
111112
2. **Cross-Chain Minting**: Tracks deposits and mints corresponding nBTC on Sui with Merkle proof validation
112113
3. **Data Storage**: Uses D1 for transaction data, KV for block storage
113114
4. **Queue Consumption**: Consumes blocks from `block-queue` populated by block-ingestor
114115

115-
### Configuration
116+
### BTCIndexer Configuration
116117

117118
- **Cron**: Every minute (`* * * * *`)
118119
- **D1 Database**: `btcindexer-dev`
@@ -121,7 +122,7 @@ See @README.md for Cloudflare RPC documentation and examples.
121122
- **Service Bindings**: `SuiIndexer`, `Compliance`
122123
- **Secrets**: `NBTC_MINTING_SIGNER_MNEMONIC` (via Secrets Store)
123124

124-
### Database Schema
125+
### BTCIndexer Database Schema
125126

126127
See migration files in `packages/btcindexer/db/migrations/`:
127128

@@ -138,7 +139,7 @@ See migration files in `packages/btcindexer/db/migrations/`:
138139

139140
**Location**: `./packages/sui-indexer`
140141

141-
### Architecture
142+
### Sui Indexer Architecture
142143

143144
- `src/index.ts` - Entry point with scheduled task
144145
- `src/processor.ts` - Sui event indexing
@@ -156,13 +157,13 @@ The Sui Indexer integrates with IKA (MPC service) for threshold signature operat
156157
- Manages presign objects for Bitcoin transaction signing
157158
- Implements coin selection logic for redemption transactions
158159

159-
### Key Features
160+
### Sui Indexer Key Features
160161

161162
1. **Event Monitoring**: Indexes Sui events for nBTC redemption requests
162163
2. **Redemption Processing**: Handles burn-and-redeem flow with IKA MPC
163164
3. **UTXO Management**: Manages UTXO lifecycle (available → locked → spent)
164165

165-
### Configuration
166+
### Sui Indexer Configuration
166167

167168
- **Cron**: Every minute (`* * * * *`)
168169
- **D1 Database**: Shared `btcindexer-dev`
@@ -176,18 +177,18 @@ The Sui Indexer integrates with IKA (MPC service) for threshold signature operat
176177

177178
See @packages/block-ingestor/README.md for detailed architecture.
178179

179-
### Architecture
180+
### Block Ingestor Architecture
180181

181182
- `src/index.ts` - HTTP router and handlers
182183
- `src/ingest.ts` - Block ingestion logic
183184
- `src/api/put-blocks.ts` - msgpack encoding/decoding
184185
- `src/api/client.ts` - Client for sending blocks
185186

186-
### Key Features
187+
### Block Ingestor Key Features
187188

188189
Receives Bitcoin blocks via REST API, validates them, and enqueues to `block-queue` for processing by BTCIndexer.
189190

190-
### Configuration
191+
### Block Ingestor Configuration
191192

192193
- **KV Namespace**: `BtcBlocks` (shared with btcindexer)
193194
- **Queue Producer**: `block-queue`
@@ -197,33 +198,33 @@ Receives Bitcoin blocks via REST API, validates them, and enqueues to `block-que
197198

198199
**Location**: `./packages/compliance`
199200

200-
### Architecture
201+
### Compliance Architecture
201202

202203
- `src/index.ts` - Scheduled worker entry point
203204
- `src/sanction.ts` - Sanctions list updating logic
204205
- `src/storage.ts` - D1 storage for sanctions
205206
- `src/rpc.ts` - RPC interface for other services to query compliance data
206207

207-
### Key Features
208+
### Compliance Key Features
208209

209210
1. **Sanctions List Updates**: Daily cron job fetches and updates sanctions data
210211
2. **Compliance API**: Exposes RPC methods for other services to check addresses
211212
3. **Geo-blocking**: Supports geo-blocking rules
212213

213-
### Configuration
214+
### Compliance Configuration
214215

215216
- **Cron**: Daily at 1am (`0 1 * * *`)
216217
- **D1 Database**: `compliance`
217218

218-
### Database Schema
219+
### Compliance Database Schema
219220

220221
See migration files in `packages/compliance/db/migrations/`.
221222

222223
## Lib
223224

224225
**Location**: `./packages/lib`
225226

226-
### Architecture
227+
### Lib Architecture
227228

228229
Shared library package containing utilities used across all services:
229230

@@ -237,7 +238,7 @@ Shared library package containing utilities used across all services:
237238
- `src/rpc-types.ts` - Shared RPC type definitions
238239
- `src/test-helpers/` - Test utilities including D1 initialization
239240

240-
### Key Features
241+
### Lib Key Features
241242

242243
1. **Shared Types**: Network enums, block records, transaction status types
243244
2. **Testing Support**: Mock RPC implementations and D1 test helpers
@@ -255,7 +256,7 @@ Shared library package containing utilities used across all services:
255256

256257
- **Framework**: Bun's built-in test framework with Miniflare for mocking Workers
257258
- **Mock RPC**: Each service provides `RPCMock` implementation for isolated testing
258-
- **Test Data**: Real Bitcoin regtest blocks from https://learnmeabitcoin.com/explorer/
259+
- **Test Data**: Real Bitcoin regtest blocks from <https://learnmeabitcoin.com/explorer/>
259260
- **Test Helpers**: Located in `packages/lib/src/test-helpers/`
260261

261262
### Configuration Pattern

‎packages/block-ingestor/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22

33
## Key Features
44

5-
#### 1. Block Reception
5+
### 1. Block Reception
66

77
- Exposes REST API endpoint to receive new Bitcoin blocks
88
- Uses msgpack encoding for efficient data transmission
99
- Stores blocks in Cloudflare KV for fast retrieval
1010

11-
#### 2. Queue Processing
11+
### 2. Queue Processing
1212

1313
- Enqueues blocks to Cloudflare Queue for downstream processing
1414
- Implements proper batching for efficient processing

‎packages/btcindexer/db/migrations/0001_initial_schema.sql‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ CREATE INDEX IF NOT EXISTS btc_blocks_is_scanned_height ON btc_blocks (is_scanne
1515

1616
-- This table tracks the nBTC deposit txs (minting)
1717
CREATE TABLE IF NOT EXISTS nbtc_minting (
18-
tx_id TEXT NOT NULL PRIMARY KEY,
18+
tx_id TEXT NOT NULL,
1919
address_id INTEGER NOT NULL, -- nbtc pkg is linked through address_id -> nbtc_deposit_addresses.setup_id
2020
sender TEXT NOT NULL,
2121
vout INTEGER NOT NULL,
@@ -28,6 +28,7 @@ CREATE TABLE IF NOT EXISTS nbtc_minting (
2828
updated_at INTEGER NOT NULL, -- timestamp_ms
2929
sui_tx_id TEXT,
3030
retry_count INTEGER NOT NULL DEFAULT 0,
31+
PRIMARY KEY (tx_id, address_id),
3132
FOREIGN KEY (address_id) REFERENCES nbtc_deposit_addresses(id)
3233
) STRICT;
3334

‎packages/btcindexer/src/btcindexer.helpers.test.ts‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,11 @@ export interface TestIndexerHelper {
107107

108108
expectMintingCount: (count: number) => Promise<void>;
109109
expectSenderCount: (count: number, expectedAddress?: string) => Promise<void>;
110-
expectTxStatus: (txId: string, expectedStatus: MintTxStatus | string) => Promise<void>;
110+
expectTxStatus: (
111+
txId: string,
112+
expectedStatus: MintTxStatus | string,
113+
setupId: number,
114+
) => Promise<void>;
111115
}
112116

113117
// test suite helper functions constructor.
@@ -336,11 +340,15 @@ export async function setupTestIndexerSuite(
336340
const expectTxStatus = async (
337341
txId: string,
338342
expectedStatus: MintTxStatus | string,
343+
setupId: number,
339344
): Promise<void> => {
340-
const { results } = await db
341-
.prepare("SELECT status FROM nbtc_minting WHERE tx_id = ?")
342-
.bind(txId)
343-
.all();
345+
const query = db
346+
.prepare(
347+
"SELECT status FROM nbtc_minting WHERE tx_id = ? AND address_id IN (SELECT id FROM nbtc_deposit_addresses WHERE setup_id = ?)",
348+
)
349+
.bind(txId, setupId);
350+
351+
const { results } = await query.all<{ status: string }>();
344352
expect(results.length).toEqual(1);
345353
expect(results[0]).toBeDefined();
346354
expect(results[0]!.status).toEqual(expectedStatus);

‎packages/btcindexer/src/btcindexer.test.ts‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -229,6 +229,7 @@ describe("Indexer.splitActiveInactiveTxs", () => {
229229
block_height: 100,
230230
btc_network: BtcNet.REGTEST,
231231
deposit_address: REGTEST_DATA[329]!.depositAddr,
232+
setup_id: 1,
232233
};
233234
const { activeTxIds } = indexer.splitActiveInactiveTxs([pendingTx]);
234235
expect(activeTxIds.length).toEqual(1);
@@ -247,6 +248,7 @@ describe("Indexer.splitActiveInactiveTxs (Inactive)", () => {
247248
block_height: 100,
248249
btc_network: BtcNet.REGTEST,
249250
deposit_address: REGTEST_DATA[329]!.depositAddr,
251+
setup_id: 1,
250252
};
251253
const result = indexer.splitActiveInactiveTxs([pendingTx]);
252254

@@ -264,6 +266,7 @@ describe("Indexer.splitActiveInactiveTxs (Inactive)", () => {
264266
block_height: 100,
265267
btc_network: BtcNet.REGTEST,
266268
deposit_address: "inactive_address",
269+
setup_id: 1,
267270
};
268271

269272
const originalMap = indexer.nbtcDepositAddrMap;
@@ -355,7 +358,7 @@ describe("Indexer.registerBroadcastedNbtcTx", () => {
355358

356359
describe("Indexer.hasNbtcMintTx", () => {
357360
it("should return false when transaction does not exist", async () => {
358-
const result = await indexer.hasNbtcMintTx("nonexistent_tx_id");
361+
const result = await indexer.hasNbtcMintTx("nonexistent_tx_id", 1);
359362
expect(result).toBe(false);
360363
});
361364

@@ -365,7 +368,7 @@ describe("Indexer.hasNbtcMintTx", () => {
365368

366369
await indexer.registerBroadcastedNbtcTx(txHex, BtcNet.REGTEST);
367370

368-
const result = await indexer.hasNbtcMintTx(txInfo.id);
371+
const result = await indexer.hasNbtcMintTx(txInfo.id, 1);
369372
expect(result).toBe(true);
370373
});
371374
});
@@ -579,9 +582,9 @@ describe("Indexer.detectMintedReorgs", () => {
579582
.bind(blockData.hash, blockData.height, BtcNet.REGTEST, Date.now(), 1)
580583
.run();
581584

582-
await indexer.detectMintedReorgs(blockData.height);
585+
await indexer.detectMintedReorgs(blockData.height, 1);
583586

584-
const status = await indexer.storage.getTxStatus(txData.id);
587+
const status = await indexer.storage.getTxStatus(txData.id, 1);
585588
expect(status).toEqual(MintTxStatus.Minted);
586589
});
587590
});
@@ -622,7 +625,7 @@ describe("Indexer.processBlock", () => {
622625
await suite.setupBlock(327);
623626
await indexer.processBlock(reorgBlockInfo);
624627

625-
const status = await indexer.storage.getTxStatus(txData.id);
628+
const status = await indexer.storage.getTxStatus(txData.id, 1);
626629
expect(status).toEqual(MintTxStatus.MintedReorg);
627630
});
628631
});
@@ -726,7 +729,7 @@ describe("Indexer.verifyConfirmingBlocks", () => {
726729
suite.mockSuiClient.verifyBlocks,
727730
"Verify that verifyBlocks was called with the correct block hash",
728731
).toHaveBeenCalledWith([block329.hash]);
729-
await suite.expectTxStatus(tx329.id, "reorg");
732+
await suite.expectTxStatus(tx329.id, "reorg", 1);
730733
});
731734

732735
it("should verify confirming blocks and not update status if blocks are still valid", async () => {
@@ -749,7 +752,7 @@ describe("Indexer.verifyConfirmingBlocks", () => {
749752
).toHaveBeenCalledWith([block329.hash]);
750753

751754
// Check that the transaction status remains 'confirming' since block is still valid
752-
await suite.expectTxStatus(tx329.id, "confirming");
755+
await suite.expectTxStatus(tx329.id, "confirming", 1);
753756
});
754757

755758
it("should handle empty confirming blocks list", async () => {
@@ -772,7 +775,7 @@ describe("Indexer.verifyConfirmingBlocks", () => {
772775
await indexer.verifyConfirmingBlocks();
773776

774777
expect(suite.mockSuiClient.verifyBlocks).toHaveBeenCalledWith([block329.hash]);
775-
await suite.expectTxStatus(tx329.id, "confirming");
778+
await suite.expectTxStatus(tx329.id, "confirming", 1);
776779
});
777780
});
778781

0 commit comments

Comments
 (0)