Restructure mdBook docs into sectioned chapters

Author: Okm165

docs/book.toml

@@ -3,7 +3,8 @@ authors = ["Okm165"]
 language = "en"
 multilingual = false
 src = "src"
-title = "Herodotus Data Processor Documentation"
+title = "HDP Cairo Documentation"
+description = "Provable historical blockchain data in Cairo"
 
 [output.html]
 git-repository-url = "https://github.com/HerodotusDev/hdp-cairo"

docs/src/SUMMARY.md

@@ -1,4 +1,69 @@
-- [Introduction](./introduction.md)
-- [Getting Started](./getting_started.md)
-- [Architecture](./architecture.md)
-- [Debugging](./debugging.md)
+# Summary
+
+[Introduction](introduction.md)
+[Getting Started](getting_started.md)
+
+# Architecture
+
+- [Overview](architecture/overview.md)
+- [Rust Crates](architecture/crates.md)
+- [Rust-Cairo Integration](architecture/rust_cairo_integration.md)
+
+# Pipeline
+
+- [Overview](pipeline/overview.md)
+- [Stage 1: Dry Run](pipeline/dry_run.md)
+- [Stage 2: Fetcher](pipeline/fetcher.md)
+- [Stage 3: Sound Run](pipeline/sound_run.md)
+
+# Verification
+
+- [Overview](verification/overview.md)
+- [MPT Proofs](verification/mpt_proofs.md)
+- [MMR Proofs](verification/mmr_proofs.md)
+
+# Syscall Handlers
+
+- [Overview](syscall_handlers/overview.md)
+- [Dry vs Sound](syscall_handlers/dry_vs_sound.md)
+- [EVM Handlers](syscall_handlers/evm_handlers.md)
+- [Starknet Handlers](syscall_handlers/starknet_handlers.md)
+
+# State Management
+
+- [Memorizers](state/memorizers.md)
+- [Injected State](state/injected_state.md)
+- [Unconstrained Data](state/unconstrained.md)
+
+# Output
+
+- [Overview](output/overview.md)
+- [Task Hash](output/task_hash.md)
+- [Output Root](output/output_root.md)
+
+# Cairo Library
+
+- [Overview](cairo_library/overview.md)
+- [EVM API](cairo_library/evm.md)
+- [Starknet API](cairo_library/starknet.md)
+- [Injected State API](cairo_library/injected_state.md)
+- [eth_call](cairo_library/eth_call.md)
+
+# Proving
+
+- [STWO Integration](proving/stwo_integration.md)
+
+# Examples
+
+- [StarkGate Solvency](examples/starkgate_solvency.md)
+- [Multi-Chain](examples/multichain.md)
+- [Injected State](examples/injected_state_example.md)
+
+# Reference
+
+- [CLI](reference/cli.md)
+- [Configuration](reference/configuration.md)
+- [Types](reference/types.md)
+
+[Glossary](glossary.md)
+[Debugging](debugging.md)

docs/src/architecture.md

@@ -0,0 +1,19 @@
+# Rust Crates
+
+This repository is organized into focused Rust crates. The most important ones are listed below.
+
+| Crate | Purpose | Notes |
+| --- | --- | --- |
+| `cli` | User-facing CLI (`hdp`) | Subcommands: dry-run, fetch-proofs, sound-run |
+| `dry_run` | Stage 1 execution | Runs Cairo VM with RPC-backed handlers |
+| `fetcher` | Stage 2 proof collection | Builds `ProofsData` from key sets |
+| `sound_run` | Stage 3 execution | Offline run with verified proofs |
+| `dry_hint_processor` | Dry-run syscall handlers | RPC reads and key collection |
+| `sound_hint_processor` | Sound-run syscall handlers | Reads from memorizers |
+| `syscall_handler` | Shared syscall dispatch | Routes to EVM/Starknet/injected state |
+| `hints` | Cairo hint implementations | 250+ hints for VM integration |
+| `types` | Shared types and schemas | Inputs/outputs, keys, proofs |
+| `indexer_client` | Herodotus Indexer API client | MMR proof requests |
+| `state_server` | Injected state server | Patricia trie + proof API |
+
+If you are new to the codebase, start with `cli`, `dry_run`, `fetcher`, and `sound_run`, then follow their dependencies.

docs/src/architecture/crates.md

@@ -0,0 +1,35 @@
+# Architecture Overview
+
+HDP Cairo is a layered system that combines Cairo0 verifiers, Cairo1 user modules, and Rust-based hint processors. The goal is to make historical blockchain data verifiable and deterministic.
+
+## Layers
+
+- **Cairo1 user layer**: your module uses the `hdp_cairo` library to request data.
+- **Bootloader layer (Cairo0)**: loads the compiled Cairo1 class and executes it.
+- **Verification layer (Cairo0)**: validates MMR/MPT/Patricia proofs before data is used.
+- **Hint processors (Rust)**: bridge the Cairo VM with external data structures.
+- **Orchestration (Rust)**: CLI commands for dry run, fetcher, and sound run.
+
+## Data flow
+
+```
++-----------+    +-----------+    +-----------+
+| Dry Run   | -> | Fetcher   | -> | Sound Run |
+| RPC keys  |    | Proofs    |    | Verify run|
++-----------+    +-----------+    +-----------+
+
+dry_run_output.json -> proofs.json -> outputs
+```
+
+## Cairo0 vs Cairo1 boundary
+
+- Cairo1 code calls `call_contract_syscall` via the `hdp_cairo` library.
+- The Cairo VM forwards syscalls to the Rust `SyscallHandler`.
+- The handler either performs RPC reads (dry run) or memorizer reads (sound run).
+
+## Entry points
+
+- `hdp dry-run`: stage 1, generates `dry_run_output.json`
+- `hdp fetch-proofs`: stage 2, generates `proofs.json`
+- `hdp sound-run`: stage 3, returns `task_hash`, `output_root`, and `mmr_metas`
+

docs/src/architecture/overview.md

@@ -0,0 +1,48 @@
+# Rust-Cairo Integration
+
+HDP uses the Cairo VM syscall interface to connect Cairo1 code to Rust logic. The same Cairo1 module runs in both dry run and sound run; only the Rust handlers change.
+
+## Call flow
+
+1. A Cairo1 module calls a method from `hdp_cairo`, which internally uses `call_contract_syscall`.
+2. The Cairo VM forwards the syscall to the Rust `SyscallHandler`.
+3. The handler decodes a `CallContractRequest`, routes it, and writes a `CallContractResponse`.
+4. The result is decoded back in Cairo1 and returned to the caller.
+
+## Syscall selectors
+
+HDP handles two syscall selectors:
+
+- `CallContract` for all data access paths.
+- `Keccak` for Cairo keccak hashing.
+
+The `Keccak` selector is handled by `KeccakHandler` inside the Rust `SyscallHandler`.
+
+## Routing logic
+
+The Rust `SyscallHandler` dispatches requests based on:
+
+- **Contract address** for special handlers (debug, injected_state, unconstrained).
+- **Chain ID** in calldata for EVM vs Starknet routing.
+
+Relevant code paths:
+
+- `crates/syscall_handler/src/lib.rs` (routing and execution)
+- `crates/types/src/cairo/new_syscalls.rs` (request/response layout)
+- `hdp_cairo/src/evm/*.cairo` and `hdp_cairo/src/starknet/*.cairo` (Cairo API)
+
+## Dict manager and hints
+
+The Cairo VM uses a shared `DictManager` for memorizers. During execution:
+
+- Cairo hints can read/write dicts via the manager.
+- Rust hint processors populate dicts during verification.
+
+This is how verified data becomes available to Cairo1 calls without network access.
+
+## Dry vs sound
+
+- **Dry run** uses `dry_hint_processor` handlers that make RPC calls and collect keys.
+- **Sound run** uses `sound_hint_processor` handlers that read from memorizers filled during verification.
+
+This separation keeps Cairo code deterministic and identical across stages.

docs/src/architecture/rust_cairo_integration.md

@@ -0,0 +1,44 @@
+# eth_call
+
+HDP includes a provable EVM interpreter that can execute `eth_call`-style requests inside Cairo.
+
+## Entry point
+
+`hdp_cairo` re-exports the helper:
+
+- `execute_eth_call(hdp, time_and_space, sender, target, calldata)`
+
+Source: `hdp_cairo/src/eth_call/execute_call.cairo`
+
+## What it does
+
+- Builds an EIP-1559 transaction wrapper for the call
+- Executes an EVM interpreter in Cairo
+- Reads required state through HDP memorizers
+- Returns a `TransactionResult` with success flag and return data
+
+## Supported instructions
+
+The interpreter is implemented in `hdp_cairo/src/eth_call/evm/instructions/`. If you need to verify opcode coverage, start there and in `hdp_cairo/src/eth_call/evm/instructions.cairo`.
+
+## Limitations
+
+- The interpreter is designed for `eth_call`-style execution (no state mutation).
+- Opcode coverage is intentionally scoped; check the instruction modules for details.
+
+## Example
+
+```cairo
+use hdp_cairo::{HDP, execute_eth_call};
+use hdp_cairo::eth_call::hdp_backend::TimeAndSpace;
+use starknet::EthAddress;
+
+let time_and_space = TimeAndSpace { chain_id: 0x1, block_number: 18_500_000 };
+let result = execute_eth_call(
+    @hdp,
+    @time_and_space,
+    EthAddress::from(0x01),
+    EthAddress::from(0x02),
+    array![0x12, 0x34].span(),
+);
+```