Skip to content

Commit 51e6bde

Browse files
daithihearndaithihearn
authored
feat(docs): add Agent Guide for indexer-core usage and integration (#113)
* feat(docs): add Agent Guide for `indexer-core` usage and integration * fix(docs): update links in AGENTS.md and CLAUDE.md for consistency
1 parent 9c5bb5e commit 51e6bde

File tree

2 files changed

+146
-132
lines changed

2 files changed

+146
-132
lines changed

‎.claude/CLAUDE.md‎

Lines changed: 2 additions & 132 deletions
Original file line numberDiff line numberDiff line change
@@ -1,135 +1,5 @@
11
# CLAUDE.md
22

3-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
3+
Use [`AGENTS.md`](../AGENTS.md) as the canonical instruction file for this repository.
44

5-
## Project Overview
6-
7-
This is a Kotlin library for building blockchain indexers for VeChainThor. It provides parallel processing, dependency management, and automatic retry logic for indexing blockchain data.
8-
9-
## Build Commands
10-
11-
```bash
12-
# Build the project
13-
./gradlew build
14-
15-
# Run tests
16-
./gradlew test
17-
18-
# Run a specific test class
19-
./gradlew test --tests "org.vechain.indexer.utils.IndexerOrderUtilsTest"
20-
21-
# Check code coverage
22-
./gradlew jacocoTestReport
23-
# Coverage report: build/reports/jacoco/test/html/index.html
24-
25-
# Format code (ktfmt with Google style)
26-
./gradlew spotlessApply
27-
28-
# Check formatting
29-
./gradlew spotlessCheck
30-
31-
# Clean build artifacts
32-
./gradlew clean
33-
```
34-
35-
## Core Architecture
36-
37-
### Indexer Types
38-
39-
The library provides two main indexer implementations via `IndexerFactory.build()`:
40-
41-
1. **BlockIndexer**: Full block-by-block processing
42-
- Used when `includeFullBlock()` is set or when `dependsOn()` is configured
43-
- Can inspect transaction call data via `callDataClauses()`
44-
- Processes reverted transactions
45-
- Required for dependent indexers
46-
47-
2. **LogsIndexer**: Fast event-based syncing
48-
- Used by default when no dependencies and full block not required
49-
- Fetches only event logs and transfer logs via Thor API
50-
- More efficient for event-driven indexing
51-
- Implements `fastSync()` to quickly catch up to finalized blocks
52-
53-
### Dependency Management & Sequential Processing
54-
55-
The `IndexerRunner` orchestrates multiple indexers using topological sorting (`IndexerOrderUtils.topologicalOrder()`):
56-
57-
- All indexers placed in single group, ordered by dependencies (dependencies before dependents)
58-
- Indexers process same block **sequentially** within group, honoring dependency order
59-
- IndexerRunner uses channels to buffer blocks and coordinate processing
60-
- Example: If `IndexerA` depends on `IndexerB`, then `IndexerB` processes block N before `IndexerA` processes block N
61-
- Only single-dependency chains supported (each indexer can depend on at most one other)
62-
63-
### Lifecycle & States
64-
65-
Indexer states (defined in `Status` enum):
66-
- `NOT_INITIALISED` → `INITIALISED` → `FAST_SYNCING` → `SYNCING` → `FULLY_SYNCED`
67-
- `SHUT_DOWN`: Terminal state
68-
69-
Initialization flow:
70-
1. `initialise()`: Determines starting block, calls `rollback()` on processor
71-
2. `fastSync()`: (LogsIndexer only) Catches up to finalized block using log events
72-
3. `processBlock()`: Main processing loop with reorg detection
73-
74-
### Reorg Detection
75-
76-
Reorg detection in `BlockIndexer.checkForReorg()`:
77-
- Compares `block.parentID` with `previousBlock.id`
78-
- On detection: logs error, calls `rollback()`, throws `ReorgException`
79-
- Only checks when `currentBlockNumber > startBlock` and `previousBlock != null`
80-
81-
### Event Processing
82-
83-
Event processing pipeline (`CombinedEventProcessor`):
84-
1. **ABI Events**: Configured via `abis()` - loads JSON ABI files
85-
2. **Business Events**: Configured via `businessEvents()` - custom event definitions with conditional logic
86-
3. **VET Transfers**: Included by default unless `excludeVetTransfers()` is called
87-
88-
Events are decoded and returned as `IndexedEvent` objects to the `IndexerProcessor.process()` method.
89-
90-
### IndexerProcessor Interface
91-
92-
Implementations must provide:
93-
- `getLastSyncedBlock()`: Returns last successfully processed block (or null)
94-
- `rollback(blockNumber)`: Reverts data for specified block
95-
- `process(entry)`: Handles `IndexingResult.BlockResult` (full block) or `IndexingResult.LogResult` (log batch)
96-
97-
## Code Style
98-
99-
- **Formatting**: ktfmt with Google style, 4-space indents (enforced by Spotless)
100-
- **Language**: Kotlin with Java 21 target
101-
- **Testing**: JUnit 5, MockK for mocking, Strikt for assertions
102-
103-
## Important Implementation Details
104-
105-
### IndexerFactory Configuration
106-
107-
The factory uses a builder pattern. Key methods:
108-
- `name()`, `thorClient()`, `processor()`: Required
109-
- `startBlock()`: Default is 0
110-
- `dependsOn()`: Forces BlockIndexer (needed for dependency coordination). Single-parent only.
111-
- `includeFullBlock()`: Forces BlockIndexer (enables access to gas, reverted txs)
112-
- `blockBatchSize()`: For LogsIndexer, controls log fetch batch size (default 100). For IndexerRunner, controls channel buffer (default 1).
113-
- `logFetchLimit()`: Pagination limit for Thor API calls (default 1000)
114-
115-
### Retry Logic
116-
117-
`IndexerRunner.retryUntilSuccess()` wraps:
118-
- Indexer initialization
119-
- Block fetching
120-
- Block processing
121-
122-
On failure: logs error, waits 1 second, retries indefinitely (until success or cancellation).
123-
124-
## Testing Notes
125-
126-
- Mock Thor client for unit tests
127-
- Use `TestableLogsIndexer` pattern to test internal sync logic
128-
- Verify topological ordering for dependency chains in `IndexerOrderUtilsTest`
129-
- Test reorg scenarios by providing blocks with mismatched `parentID`
130-
131-
## Preferences
132-
Be extremely concise. Sacrifice grammar for the sake of concision.
133-
Always prefer simple solution over complex ones.
134-
When unsure, ask for clarification.
135-
run `make format` after making code changes to ensure proper formatting.
5+
Do not duplicate or extend project guidance here unless Claude-specific behavior genuinely requires it.

‎AGENTS.md‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Agent Guide
2+
3+
This file is the canonical instruction entry point for coding agents working with `indexer-core`.
4+
5+
It is intentionally lean. Use it to build a correct mental model quickly, then open the linked repo docs instead of inferring behavior from scattered source files.
6+
7+
## What This Library Is
8+
9+
`indexer-core` is a Kotlin library for building VeChainThor indexers.
10+
11+
At a high level it provides:
12+
13+
- `IndexerProcessor` as the application persistence boundary
14+
- `IndexerFactory` as the only supported way to configure and build indexers
15+
- `IndexerRunner` to initialise, fast-sync when possible, coordinate dependencies, and keep indexers running through retries and reorg recovery
16+
- two runtime modes:
17+
- `LogsIndexer` for fast log-based catch-up when you only need decoded events / transfers
18+
- `BlockIndexer` when you need full block context or dependency ordering
19+
20+
Do not ask users to construct indexers manually from implementation classes unless they are working on the library internals themselves. For normal usage, all indexers should be built with `IndexerFactory`.
21+
22+
## Who This Guide Is For
23+
24+
This guide is for both:
25+
26+
- agents changing `indexer-core` itself
27+
- agents helping a consumer integrate `indexer-core` into another service
28+
29+
If the task is library maintenance, preserve public behavior documented in the repo docs unless the change explicitly updates that behavior.
30+
31+
If the task is consumer guidance, optimize for correct mode selection and integration advice before discussing internals.
32+
33+
## Required Onboarding Path
34+
35+
Before making claims about library behavior, read in this order:
36+
37+
1. [`README.md`](README.md)
38+
2. [`docs/README.md`](docs/README.md)
39+
3. one targeted guide based on the task:
40+
- runtime model and lifecycle: [`docs/IndexerOverview.md`](docs/IndexerOverview.md)
41+
- log-based mode and fast sync: [`docs/LogsIndexerOverview.md`](docs/LogsIndexerOverview.md)
42+
- ABI loading and decoded events: [`docs/EventsAndABIHandling.md`](docs/EventsAndABIHandling.md)
43+
- business event design: [`docs/BusinessEvents.md`](docs/BusinessEvents.md)
44+
- upgrade / compatibility questions: [`docs/MIGRATION-8.0.0.md`](docs/MIGRATION-8.0.0.md)
45+
46+
The repo markdown docs are the source of truth. Prefer them over memory, ad hoc code reading, or external copies.
47+
48+
## Mental Model To Keep In Mind
49+
50+
- `IndexerProcessor` is where consumers persist progress and domain data.
51+
- The runtime may emit either `IndexingResult.LogResult` or `IndexingResult.BlockResult`; processors should handle both when relevant to the configuration.
52+
- Startup rollback is intentional. It is a data-integrity feature, not a bug.
53+
- Reorg recovery is part of the runtime contract. Consumers are expected to implement deterministic rollback behavior.
54+
- Dependencies affect execution semantics, not just throughput. Adding `dependsOn(...)` changes how the runtime must coordinate indexers.
55+
56+
## Mode Selection Checklist
57+
58+
Use this checklist before recommending or editing indexer configuration.
59+
60+
Choose the default factory-built log mode when:
61+
62+
- the consumer only needs decoded ABI events, business events, or VET transfers
63+
- fastest catch-up is the priority
64+
- there is no same-block dependency on another indexer
65+
66+
Choose `includeFullBlock()` when the consumer needs:
67+
68+
- full block contents
69+
- reverted transaction visibility
70+
- gas / fee metadata from full block processing
71+
- clause inspection results from `callDataClauses(...)`
72+
73+
Choose `dependsOn(...)` when:
74+
75+
- one indexer must finish a block before another processes that same block
76+
77+
Important:
78+
79+
- `LogsIndexer` and `BlockIndexer` are not interchangeable modes.
80+
- `dependsOn(...)` forces block-based execution semantics.
81+
- `includeFullBlock()` forces block-based execution semantics.
82+
83+
Choose business events when:
84+
85+
- downstream consumers care about higher-level actions rather than every raw event
86+
87+
Choose raw ABI events when:
88+
89+
- the consumer needs each decoded event individually
90+
- there is no stable semantic grouping worth encoding as a business event
91+
92+
## Guardrails
93+
94+
- Build indexers through `IndexerFactory`, not by manually wiring implementation classes in application code.
95+
- Do not describe `LogsIndexer` and `BlockIndexer` as equivalent choices with different performance profiles. They expose different runtime behavior and different data.
96+
- Do not treat startup rollback as suspicious behavior. It is part of the library’s safety model.
97+
- Do not rely on stale documentation copies. The repo docs are authoritative.
98+
- Do not present internal implementation details as stable public API unless they are explicitly documented as such.
99+
100+
## Common Agent Tasks
101+
102+
Optimize guidance for these common tasks:
103+
104+
- explaining how to integrate `indexer-core` into another service
105+
- changing the library itself
106+
- debugging behavior differences between `LogsIndexer` and `BlockIndexer`
107+
- designing ABI-driven or business-event-driven indexing setups
108+
109+
Documentation updates matter, but they are secondary to preserving correct runtime behavior and public guidance.
110+
111+
## Verification Expectations
112+
113+
When changing this library:
114+
115+
- run targeted tests for the touched behavior as a minimum
116+
- run broader `./gradlew test` when the change is cross-cutting or affects shared runtime behavior
117+
- run formatting checks or formatting fixes when Kotlin code changes
118+
119+
Minimum standard before claiming completion:
120+
121+
- the changed behavior is covered by tests or an existing test path was exercised
122+
- any affected public guidance remains consistent with the repo docs
123+
- the response states clearly if full verification was not run
124+
125+
Useful commands:
126+
127+
```bash
128+
./gradlew test
129+
./gradlew test --tests "org.vechain.indexer.SomeTest"
130+
./gradlew spotlessCheck
131+
./gradlew spotlessApply
132+
```
133+
134+
## When Working From Source
135+
136+
The codebase is useful for confirmation, but agents should not need to reverse-engineer the library from source just to understand its purpose.
137+
138+
Read source after the docs when you need to:
139+
140+
- confirm an implementation detail
141+
- debug a behavioral discrepancy
142+
- update internals while preserving the documented contract
143+
144+
If source and docs appear to disagree, call that out explicitly instead of silently choosing one.

0 commit comments

Comments
 (0)