From 759a28b85316ed28f11756ac815a116b06cefc05 Mon Sep 17 00:00:00 2001 From: pthmas <9058370+pthmas@users.noreply.github.com> Date: Wed, 17 Jun 2026 10:33:05 +0200 Subject: [PATCH 1/2] docs: update all documentation to match current codebase --- README.md | 122 ++++++---- backend/README.md | 214 +++++++++--------- docs/API.md | 367 +++++++++++++++++++----------- docs/ARCHITECTURE.md | 178 ++++++++++++--- docs/FEATURES.md | 129 +++++++++++ docs/FEATURES_V2.md | 353 ----------------------------- docs/PRD.md | 503 +++++++---------------------------------- docs/WHITE_LABELING.md | 172 +++++++------- frontend/README.md | 170 +++++++------- 9 files changed, 950 insertions(+), 1258 deletions(-) create mode 100644 docs/FEATURES.md delete mode 100644 docs/FEATURES_V2.md diff --git a/README.md b/README.md index 6bbd22e..e7fd854 100644 --- a/README.md +++ b/README.md @@ -1,81 +1,121 @@ # Atlas -A lightweight Ethereum L2 blockchain explorer. +Atlas is an EVM blockchain explorer for ev-node based chains. It runs a Rust indexer, REST API, background workers, PostgreSQL database, and Vite frontend as one deployable stack. + +## Current Architecture + +- `atlas-server`: one Rust binary that runs the indexer, API, live SSE stream, metrics endpoint, optional faucet, optional Data Availability tracking, NFT metadata worker, missed-block gap-fill worker, and optional snapshot scheduler. +- `postgres`: stores indexed blocks, transactions, addresses, ERC-20 data, NFT data, logs, contract verification data, proxy metadata, and indexer state. +- `atlas-frontend`: Vite app served by unprivileged nginx on port 8080 inside the container, exposed as host port 80 by Docker Compose. ## Quick Start -### Prerequisites +Requirements: -- `just` 1.0+ - Docker and Docker Compose -- Bun 1.0+ -- Rust 1.75+ - -### Running with Docker +- An EVM JSON-RPC endpoint ```bash cp .env.example .env -docker-compose up -d +# Edit RPC_URL and CHAIN_NAME in .env +docker compose up -d ``` -Access the explorer at http://localhost:3000 +Default service URLs: + +| Service | URL | +| --- | --- | +| Explorer UI | `http://localhost` | +| API through frontend nginx | `http://localhost/api` | +| Backend API when running `atlas-server` locally | `http://localhost:3000/api` | +| Prometheus metrics when backend port is reachable | `http://localhost:3000/metrics` | +| Liveness probe when backend port is reachable | `http://localhost:3000/health/live` | +| Readiness probe when backend port is reachable | `http://localhost:3000/health/ready` | -### Local Development +The `postgres` service binds to `127.0.0.1:5432` for local development access. Docker Compose does not publish `atlas-server:3000` to the host by default; the frontend proxies browser API traffic through `http://localhost/api`. + +## Local Development + +Start only PostgreSQL: ```bash -cp .env.example .env -docker-compose up -d postgres -just frontend-install +docker compose up -d postgres ``` -Start the backend: +Run the backend: ```bash -just backend-run +cd backend +export DATABASE_URL=postgres://atlas:atlas@localhost:5432/atlas +export RPC_URL=http://localhost:8545 +cargo run --bin atlas-server -- run ``` -Start frontend: +Run the frontend: ```bash -just frontend-dev +cd frontend +bun install +bun run dev ``` -### Useful Commands +The Vite dev server runs on `http://localhost:5173` and proxies `/api` to `http://localhost:3000`. + +## Operator Commands + +Run these from `backend/` unless noted otherwise: ```bash -just --list -just frontend-lint -just frontend-build -just backend-fmt -just backend-clippy -just backend-test -just ci +cargo run --bin atlas-server -- check +cargo run --bin atlas-server -- migrate +cargo run --bin atlas-server -- run +cargo run --bin atlas-server -- db dump /tmp/atlas.dump +cargo run --bin atlas-server -- db restore /tmp/atlas.dump +cargo run --bin atlas-server -- db reset --confirm ``` -## Configuration +Useful validation commands: -Copy `.env.example` to `.env` and set `RPC_URL`. Common options: +```bash +cd backend && cargo test --workspace +cd frontend && bun run build +cd frontend && bun run lint +``` -| Variable | Description | Default | -|----------|-------------|---------| -| `RPC_URL` | Ethereum JSON-RPC endpoint | Required | -| `DATABASE_URL` | PostgreSQL connection string | `postgres://atlas:atlas@localhost:5432/atlas` (local dev) | -| `START_BLOCK` | Block to start indexing from | `0` | -| `BATCH_SIZE` | Blocks per indexing batch | `100` | -| `RPC_REQUESTS_PER_SECOND` | RPC rate limit | `100` | -| `FETCH_WORKERS` | Parallel block fetch workers | `10` | -| `RPC_BATCH_SIZE` | Blocks per RPC batch request | `20` | -| `IPFS_GATEWAY` | Gateway for NFT metadata | `https://ipfs.io/ipfs/` | -| `REINDEX` | Wipe and reindex from start | `false` | +## Configuration -See [White Labeling](docs/WHITE_LABELING.md) for branding customization (chain name, logo, colors). +Copy `.env.example` to `.env`. `RPC_URL` is required. When running without Docker, `DATABASE_URL` is also required. + +Common variables: + +| Variable | Purpose | Default | +| --- | --- | --- | +| `RPC_URL` | EVM JSON-RPC endpoint | required | +| `DATABASE_URL` | PostgreSQL connection URL for local backend runs | required outside Docker | +| `CHAIN_NAME` | Display name served by `/api/config` and `/api/status` | `Unknown` | +| `START_BLOCK` | First block to index | `0` | +| `BATCH_SIZE` | Blocks written per DB batch | `100` | +| `FETCH_WORKERS` | Concurrent block fetch workers | `10` | +| `RPC_BATCH_SIZE` | Blocks fetched per RPC batch call | `20` | +| `RPC_REQUESTS_PER_SECOND` | RPC request rate limit | `100` | +| `DB_MAX_CONNECTIONS` | Indexer pool size | `20` | +| `API_DB_MAX_CONNECTIONS` | API pool size | `20` | +| `IPFS_GATEWAY` | Gateway for IPFS NFT metadata | `https://ipfs.io/ipfs/` | +| `METADATA_FETCH_WORKERS` | NFT metadata worker concurrency | `4` | +| `METADATA_RETRY_ATTEMPTS` | Retry attempts for retryable metadata failures | `3` | +| `REINDEX` | Wipe indexed data and restart from `START_BLOCK` | `false` | + +Optional features include Data Availability tracking, faucet support, white-label branding, scheduled snapshots, JSON logging, CORS restriction, and a Solidity compiler cache. See [.env.example](.env.example) and [Backend](backend/README.md) for the full operator reference. ## Documentation -- [API Reference](docs/API.md) - [Architecture](docs/ARCHITECTURE.md) +- [API Reference](docs/API.md) +- [Features](docs/FEATURES.md) - [White Labeling](docs/WHITE_LABELING.md) -- [Product Requirements](docs/PRD.md) +- [Product and Roadmap](docs/PRD.md) +- [Backend](backend/README.md) +- [Frontend](frontend/README.md) ## License diff --git a/backend/README.md b/backend/README.md index 509e635..8332afc 100644 --- a/backend/README.md +++ b/backend/README.md @@ -1,131 +1,129 @@ # Atlas Backend -Rust backend services for the Atlas blockchain explorer. +The backend is a Rust workspace containing `atlas-common` and the combined `atlas-server` binary. `atlas-server` runs the HTTP API, chain indexer, live update stream, background workers, and operator utilities. ## Crates -### atlas-common +| Crate | Purpose | +| --- | --- | +| `atlas-common` | Shared SQLx models, error types, pagination, and database helpers | +| `atlas-server` | Axum API, indexer, CLI, metrics, faucet, snapshots, DA tracking, NFT metadata, and database utilities | -Shared types and utilities: -- Database models with SQLx `FromRow` derives -- Error types -- Pagination helpers -- Database connection pool +There are no separate backend service binaries for the API and indexer. Use `atlas-server` for all backend operations. -### atlas-indexer +## Commands -Blockchain indexer that: -- Connects to L2 RPC endpoint -- Indexes blocks and transactions -- Detects ERC-721 Transfer events -- Manages NFT ownership tracking -- Fetches and caches NFT metadata - -### atlas-api - -REST API server (Axum) providing: -- Block/transaction/address endpoints -- NFT collection and token endpoints -- ERC-20 token endpoints -- Event log endpoints -- Address labels -- Proxy contract detection -- Contract verification -- Etherscan-compatible API -- Universal search -- Health check endpoint - -## Building +From `backend/`: ```bash -# Development build -cargo build - -# Release build -cargo build --release +# Validate configuration and DB/RPC connectivity +cargo run --bin atlas-server -- check -# Run checks -cargo check +# Run migrations and exit +cargo run --bin atlas-server -- migrate -# Run tests -cargo test +# Start API, indexer, and background workers +cargo run --bin atlas-server -- run -# Run clippy -cargo clippy +# Database utilities +cargo run --bin atlas-server -- db dump /tmp/atlas.dump +cargo run --bin atlas-server -- db restore /tmp/atlas.dump +cargo run --bin atlas-server -- db reset --confirm ``` -## Running +`db dump` uses `pg_dump --format=custom --no-owner --no-acl`. `db restore` resets the public schema and runs `pg_restore --exit-on-error`. `pg_dump`, `pg_restore`, and `psql` must be available on the host for these utilities. -```bash -# Required environment variables -export DATABASE_URL=postgres://atlas:atlas@localhost/atlas -export RPC_URL=https://your-l2-rpc.example.com - -# Run indexer -cargo run --bin atlas-indexer - -# Run API server -cargo run --bin atlas-api -``` +## Required Environment -## Project Structure +When running outside Docker: +```bash +export DATABASE_URL=postgres://atlas:atlas@localhost:5432/atlas +export RPC_URL=http://localhost:8545 +cargo run --bin atlas-server -- run ``` -backend/ -├── Cargo.toml # Workspace definition -├── Dockerfile # Multi-stage Docker build -├── migrations/ # SQLx migrations -│ └── *.sql -└── crates/ - ├── atlas-common/ - │ └── src/ - │ ├── lib.rs - │ ├── types.rs # Database models - │ ├── error.rs # Error types - │ └── db.rs # Connection pool - ├── atlas-indexer/ - │ └── src/ - │ ├── main.rs - │ ├── config.rs - │ ├── indexer.rs - │ └── metadata.rs - └── atlas-api/ - └── src/ - ├── main.rs - ├── error.rs - └── handlers/ - ├── mod.rs - ├── blocks.rs - ├── transactions.rs - ├── addresses.rs - ├── nfts.rs - ├── tokens.rs - ├── logs.rs - ├── labels.rs - ├── proxy.rs - ├── contracts.rs - ├── etherscan.rs - └── search.rs -``` - -## Dependencies - -Key dependencies: -- `tokio` - Async runtime -- `axum` - Web framework -- `sqlx` - Async PostgreSQL client -- `alloy` - Ethereum types and RPC -- `reqwest` - HTTP client for metadata fetching -- `tracing` - Structured logging -## Database Migrations - -Migrations run automatically on startup. To run manually: +Docker Compose sets `DATABASE_URL` for the `atlas-server` container. `RPC_URL` must be provided in `.env`. + +## CLI and Environment Reference + +Most runtime options are available as environment variables and CLI flags. + +| Area | Env var | CLI flag | Default | +| --- | --- | --- | --- | +| Database | `DB_MAX_CONNECTIONS` | `--atlas.db.max-connections` | `20` | +| Database | `API_DB_MAX_CONNECTIONS` | `--atlas.db.api-max-connections` | `20` | +| RPC | `RPC_URL` | `--atlas.rpc.url` | required | +| RPC | `RPC_REQUESTS_PER_SECOND` | `--atlas.rpc.requests-per-second` | `100` | +| RPC | `RPC_BATCH_SIZE` | `--atlas.rpc.batch-size` | `20` | +| API | `API_HOST` | `--atlas.api.host` | `127.0.0.1` | +| API | `API_PORT` | `--atlas.api.port` | `3000` | +| API | `CORS_ORIGIN` | `--atlas.api.cors-origin` | allow all | +| API | `SSE_REPLAY_BUFFER_BLOCKS` | `--atlas.api.sse-replay-buffer-blocks` | `4096` | +| Contracts | `SOLC_CACHE_DIR` | `--atlas.api.solc-cache-dir` | `/tmp/solc-cache` | +| Indexer | `START_BLOCK` | `--atlas.indexer.start-block` | `0` | +| Indexer | `BATCH_SIZE` | `--atlas.indexer.batch-size` | `100` | +| Indexer | `FETCH_WORKERS` | `--atlas.indexer.fetch-workers` | `10` | +| Indexer | `REINDEX` | `--atlas.indexer.reindex` | `false` | +| Indexer | `IPFS_GATEWAY` | `--atlas.indexer.ipfs-gateway` | `https://ipfs.io/ipfs/` | +| NFT metadata | `METADATA_FETCH_WORKERS` | `--atlas.indexer.metadata-fetch-workers` | `4` | +| NFT metadata | `METADATA_RETRY_ATTEMPTS` | `--atlas.indexer.metadata-retry-attempts` | `3` | +| Chain | `CHAIN_NAME` | `--atlas.chain.name` | `Unknown` | +| Chain | `CHAIN_LOGO_URL` | `--atlas.chain.logo-url` | unset | +| Chain | `CHAIN_LOGO_URL_LIGHT` | `--atlas.chain.logo-url-light` | unset | +| Chain | `CHAIN_LOGO_URL_DARK` | `--atlas.chain.logo-url-dark` | unset | +| DA tracking | `ENABLE_DA_TRACKING` | `--atlas.da.enabled` | `false` | +| DA tracking | `EVNODE_URL` | `--atlas.da.evnode-url` | required when enabled | +| DA tracking | `DA_WORKER_CONCURRENCY` | `--atlas.da.worker-concurrency` | `50` | +| DA tracking | `DA_RPC_REQUESTS_PER_SECOND` | `--atlas.da.rpc-requests-per-second` | `50` | +| Faucet | `FAUCET_ENABLED` | `--atlas.faucet.enabled` | `false` | +| Faucet | `FAUCET_AMOUNT` | `--atlas.faucet.amount` | required when enabled | +| Faucet | `FAUCET_COOLDOWN_MINUTES` | `--atlas.faucet.cooldown-minutes` | required when enabled | +| Logging | `RUST_LOG` | `--atlas.log.level` | `atlas_server=info,tower_http=debug,sqlx=warn` | +| Logging | `LOG_FORMAT` | `--atlas.log.format` | `text` | + +`FAUCET_PRIVATE_KEY` is env-only by design and must be set when `FAUCET_ENABLED=true`. + +Snapshot settings are env-only: + +| Env var | Purpose | Default | +| --- | --- | --- | +| `SNAPSHOT_ENABLED` | Enable daily `pg_dump` snapshots | `false` | +| `SNAPSHOT_TIME` | UTC time in `HH:MM` format | `03:00` | +| `SNAPSHOT_RETENTION` | Number of completed snapshot files to keep | `7` | +| `SNAPSHOT_DIR` | Container path for snapshot files | `/snapshots` | +| `SNAPSHOT_HOST_DIR` | Host bind mount used by Docker Compose | `./snapshots` | + +## Runtime Behavior + +- Migrations run on startup through a dedicated migration connection without the API query timeout. +- The API and indexer use separate SQLx pools. API connections apply a 10 second `statement_timeout`; the indexer pool uses the configured max connection count. +- Bulk writes use a separate `tokio-postgres` binary COPY connection so high-volume indexing does not consume the API pool. +- The indexer fetches blocks in batches, writes blocks/transactions/logs/token data, and publishes committed heads to the in-process `HeadTracker`. +- The gap-fill worker retries rows in `failed_blocks` with backoff and clears recovered failures atomically. +- The NFT metadata worker processes `pending` and retryable tokens, resolves IPFS/Arweave/data/HTTP URIs, classifies failures as retryable or permanent, and stores status fields on `nft_tokens`. +- Optional DA tracking queries ev-node for Celestia header/data inclusion heights and publishes `da_batch` SSE events. +- Optional snapshots write portable custom-format dumps named `atlas_snapshot_.dump`. + +## API and Probes + +The Axum router exposes the REST API under `/api`, Prometheus metrics at `/metrics`, and health endpoints: + +| Path | Purpose | +| --- | --- | +| `/health` | Legacy plain-text OK response | +| `/health/live` | Process liveness JSON | +| `/health/ready` | DB connectivity plus fresh indexer state | +| `/metrics` | Prometheus text format | + +Most routes are wrapped in a 10 second HTTP timeout and return `408 Request Timeout` when exceeded. `/api/events` and `/api/contracts/{address}/verify` are excluded because SSE streams stay open and Solidity compilation can take longer. + +## Development ```bash -# Install sqlx-cli -cargo install sqlx-cli - -# Run migrations -sqlx migrate run +cargo fmt --all +cargo clippy --workspace --all-targets +cargo test --workspace ``` + +Integration tests live under `backend/crates/atlas-server/tests/integration` and use testcontainers and wiremock where needed. diff --git a/docs/API.md b/docs/API.md index 3e53168..b044985 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1,39 +1,60 @@ # API Reference -Base URL: `http://localhost:3000` +Atlas exposes native REST endpoints under `/api`, an Etherscan-compatible query API at exact `/api`, Prometheus metrics at `/metrics`, and health probes under `/health`. -## Pagination +When using Docker Compose through the frontend, use base URL `/api`. When running the backend directly, use `http://localhost:3000/api`. -All list endpoints support pagination: +## Shared Behavior + +### Pagination + +List endpoints use: | Parameter | Default | Max | Description | -|-----------|---------|-----|-------------| -| `page` | 1 | - | Page number | -| `limit` | 20 | 100 | Items per page | +| --- | --- | --- | --- | +| `page` | `1` | none | One-based page number | +| `limit` | `20` | `100` | Items per page | -Response format: +Paginated response: ```json { - "data": [...], + "data": [], "page": 1, "limit": 20, - "total": 1000, - "total_pages": 50 + "total": 0, + "total_pages": 0 } ``` -## Endpoints +Blocks use keyset-style pagination internally to avoid large `OFFSET` scans. Other list endpoints may still use offset pagination. + +### Timeouts and Errors + +Most routes are wrapped in a 10 second HTTP timeout and return `408 Request Timeout` if exceeded. `/api/events` and `/api/contracts/{address}/verify` are excluded because they can legitimately run longer. + +Application errors are returned as JSON with an `error` field. Rate-limited faucet responses include a retry hint through `retry-after` and/or `retry_after_seconds`. + +Timeout-generated `408` responses come from the HTTP timeout middleware, not the application error handler, so they return an empty body rather than the JSON error envelope. -### Status +### Address and Hash Parameters + +Address and transaction hash parameters accept values with or without `0x` where the handler normalizes them. Addresses are stored lowercase. + +## Status, Config, Health, Metrics | Method | Path | Description | -|--------|------|-------------| -| GET | `/api/status` | Current indexed block height and index timestamp (lightweight, safe to poll frequently) | -| GET | `/api/events` | SSE stream of committed `new_block` events | -| GET | `/health` | Health check (returns "OK") | +| --- | --- | --- | +| GET | `/api/height` | Lightweight indexed-head endpoint for frequent polling | +| GET | `/api/status` | Chain ID, chain name, indexed height, and estimated totals | +| GET | `/api/config` | Branding, feature flags, and faucet config | +| GET | `/metrics` | Prometheus text format | +| GET | `/health` | Legacy plain-text `OK` | +| GET | `/health/live` | Liveness JSON | +| GET | `/health/ready` | Readiness JSON: DB reachable and indexer state fresh | + +`GET /api/height`: -**`/api/status` response:** ```json { "block_height": 1000000, @@ -41,12 +62,52 @@ Response format: } ``` -`block_height` and `indexed_at` refer to the latest committed/indexed head. +`GET /api/status`: + +```json +{ + "chain_id": "31337", + "chain_name": "Devnet", + "block_height": 1000000, + "total_transactions": 5000000, + "total_addresses": 250000, + "indexed_at": "2026-01-01T00:00:00+00:00" +} +``` + +`GET /api/config`: + +```json +{ + "chain_name": "Devnet", + "logo_url": "/branding/logo.svg", + "logo_url_light": "/branding/logo-light.svg", + "logo_url_dark": "/branding/logo-dark.svg", + "accent_color": "#3b82f6", + "background_color_dark": "#050505", + "background_color_light": "#f4ede6", + "success_color": "#22c55e", + "error_color": "#dc2626", + "features": { "da_tracking": true }, + "faucet": { + "enabled": true, + "amount_wei": "10000000000000000", + "cooldown_minutes": 30 + } +} +``` + +Unset optional branding fields are omitted. + +## Live Events + +| Method | Path | Description | +| --- | --- | --- | +| GET | `/api/events` | Server-Sent Events stream for committed block heads and DA updates | -**`/api/events` SSE details:** +The stream is excluded from the 10 second timeout and has nginx buffering disabled in the production frontend image. -- Event name: `new_block` -- Payload: +Event `new_block`: ```json { @@ -57,177 +118,221 @@ Response format: "timestamp": 1700000000, "gas_used": 21000, "gas_limit": 30000000, + "base_fee_per_gas": "1000000000", "transaction_count": 1, "indexed_at": "2026-01-01T00:00:00+00:00" } } ``` -The stream represents the committed indexed head, not a speculative node-observed head. It is a head/tail stream, not a history replay API: new or reconnected clients resume from the current live tail, while canonical catch-up stays on `/api/blocks` and `/api/status`. +Event `da_batch`: + +```json +{ + "updates": [ + { + "block_number": 1000000, + "header_da_height": 12345, + "data_da_height": 12346 + } + ] +} +``` -### Blocks +Event `da_resync`: + +```json +{ + "required": true +} +``` + +This event is emitted when a client falls behind the DA update stream and should refetch visible DA state instead of relying on incremental updates. + +The stream represents committed indexed data, not speculative node head data. Clients should use `/api/blocks` and `/api/height` for catch-up and polling fallback. + +## Blocks | Method | Path | Description | -|--------|------|-------------| -| GET | `/api/blocks` | List blocks (newest first) | -| GET | `/api/blocks/:number` | Get block by number | -| GET | `/api/blocks/:number/transactions` | Get transactions in block | +| --- | --- | --- | +| GET | `/api/blocks` | List blocks newest first | +| GET | `/api/blocks/{number}` | Get one block | +| GET | `/api/blocks/{number}/transactions` | List transactions in a block | -### Transactions +Block objects include `base_fee_per_gas` and, when DA tracking has data for that block, `da_status`. + +## Transactions | Method | Path | Description | -|--------|------|-------------| -| GET | `/api/transactions` | List transactions (newest first) | -| GET | `/api/transactions/:hash` | Get transaction details | -| GET | `/api/transactions/:hash/logs` | Get event logs | -| GET | `/api/transactions/:hash/logs/decoded` | Get decoded event logs with signatures | -| GET | `/api/transactions/:hash/erc20-transfers` | Get ERC-20 transfers in transaction | -| GET | `/api/transactions/:hash/nft-transfers` | Get NFT transfers in transaction | - -### Addresses - -| Method | Path | Parameters | Description | -|--------|------|------------|-------------| +| --- | --- | --- | +| GET | `/api/transactions` | List transactions newest first | +| GET | `/api/transactions/{hash}` | Get transaction details | +| GET | `/api/transactions/{hash}/logs` | Get raw event logs | +| GET | `/api/transactions/{hash}/logs/decoded` | Get decoded event logs when signatures or verified ABIs are available | +| GET | `/api/transactions/{hash}/erc20-transfers` | ERC-20 transfers in a transaction | +| GET | `/api/transactions/{hash}/nft-transfers` | NFT transfers in a transaction | + +## Addresses + +| Method | Path | Query | Description | +| --- | --- | --- | --- | | GET | `/api/addresses` | `is_contract`, `from_block`, `to_block`, `address_type` | List addresses | -| GET | `/api/addresses/:address` | - | Get address details | -| GET | `/api/addresses/:address/transactions` | - | Get address transactions | -| GET | `/api/addresses/:address/transfers` | `transfer_type` (erc20/nft) | Get all transfers | -| GET | `/api/addresses/:address/nfts` | - | Get NFTs owned | -| GET | `/api/addresses/:address/tokens` | - | Get ERC-20 balances | -| GET | `/api/addresses/:address/logs` | `topic0` | Get event logs | -| GET | `/api/addresses/:address/label` | - | Get address with label | +| GET | `/api/addresses/{address}` | none | Address details | +| GET | `/api/addresses/{address}/transactions` | pagination | Address transaction history | +| GET | `/api/addresses/{address}/transfers` | `transfer_type=erc20|nft`, pagination | Token and NFT transfers | +| GET | `/api/addresses/{address}/nfts` | pagination | NFTs owned by address | +| GET | `/api/addresses/{address}/tokens` | pagination | ERC-20 balances | +| GET | `/api/addresses/{address}/logs` | `topic0`, pagination | Logs emitted by address | -**Address Types**: `eoa`, `contract`, `erc20`, `nft` +`address_type` can identify EOAs, contracts, ERC-20 tokens, and NFT contracts where indexed data is available. -### NFT Collections +## ERC-20 Tokens -| Method | Path | Description | -|--------|------|-------------| -| GET | `/api/nfts/collections` | List NFT collections | -| GET | `/api/nfts/collections/:address` | Get collection details | -| GET | `/api/nfts/collections/:address/tokens` | List tokens in collection | -| GET | `/api/nfts/collections/:address/transfers` | Get collection transfers | -| GET | `/api/nfts/collections/:address/tokens/:token_id` | Get token details | -| GET | `/api/nfts/collections/:address/tokens/:token_id/transfers` | Get token transfer history | +| Method | Path | Query | Description | +| --- | --- | --- | --- | +| GET | `/api/tokens` | pagination | List ERC-20 tokens | +| GET | `/api/tokens/{address}` | none | Token detail with holder and transfer counts | +| GET | `/api/tokens/{address}/holders` | pagination | Token holder balances | +| GET | `/api/tokens/{address}/transfers` | pagination | Token transfer history | +| GET | `/api/tokens/{address}/chart` | `window` | Token chart data | -### ERC-20 Tokens +Chart `window` values are `1h`, `6h`, `24h`, `7d`, `1m`, `6m`, and `1y`. -| Method | Path | Description | -|--------|------|-------------| -| GET | `/api/tokens` | List ERC-20 tokens | -| GET | `/api/tokens/:address` | Get token details (includes holder/transfer counts) | -| GET | `/api/tokens/:address/holders` | Get token holders with balances | -| GET | `/api/tokens/:address/transfers` | Get token transfers | +## NFTs -### Event Logs +| Method | Path | Query | Description | +| --- | --- | --- | --- | +| GET | `/api/nfts/collections` | pagination | List NFT collections | +| GET | `/api/nfts/collections/{address}` | none | Collection detail | +| GET | `/api/nfts/collections/{address}/tokens` | pagination | Tokens in a collection | +| GET | `/api/nfts/collections/{address}/transfers` | pagination | Collection transfer history | +| GET | `/api/nfts/collections/{address}/tokens/{token_id}` | none | NFT token detail | +| GET | `/api/nfts/collections/{address}/tokens/{token_id}/transfers` | pagination | Token transfer history | -| Method | Path | Parameters | Description | -|--------|------|------------|-------------| -| GET | `/api/logs` | `topic0` (required) | Filter logs by event signature | +NFT token responses include metadata state fields such as `metadata_status`, `metadata_retry_count`, `next_retry_at`, `last_metadata_error`, `last_metadata_attempted_at`, and `metadata_updated_at`. -### Address Labels +## Event Logs -| Method | Path | Parameters | Description | -|--------|------|------------|-------------| -| GET | `/api/labels` | `tag`, `search` | List labels | -| GET | `/api/labels/:address` | - | Get label for address | -| GET | `/api/labels/tags` | - | Get all tags with counts | -| POST | `/api/labels` | Body: `{address, name, tags[]}` | Create/update label | -| POST | `/api/labels/bulk` | Body: `{labels: [...]}` | Bulk import labels | -| DELETE | `/api/labels/:address` | - | Delete label | +| Method | Path | Query | Description | +| --- | --- | --- | --- | +| GET | `/api/transactions/{hash}/logs` | pagination | Logs in a transaction | +| GET | `/api/transactions/{hash}/logs/decoded` | pagination | Decoded logs in a transaction | +| GET | `/api/addresses/{address}/logs` | `topic0`, pagination | Logs emitted by address | -### Contract Verification +## Contracts and Proxies | Method | Path | Description | -|--------|------|-------------| -| GET | `/api/contracts/:address/abi` | Get verified ABI | -| GET | `/api/contracts/:address/source` | Get verified source code | -| POST | `/api/contracts/verify` | Verify contract source | +| --- | --- | --- | +| GET | `/api/contracts/{address}` | Verified contract metadata, source, ABI, and verification status | +| POST | `/api/contracts/{address}/verify` | Verify source for a contract address | +| GET | `/api/contracts/{address}/proxy` | Proxy detection result for an address | +| GET | `/api/contracts/{address}/combined-abi` | Merged proxy and implementation ABI when available | +| GET | `/api/proxies` | List detected proxy contracts | + +`POST /api/contracts/{address}/verify` accepts JSON for single-file Solidity source or standard JSON compiler input. The route allows up to 50 MiB request bodies and is not subject to the 10 second timeout. + +Example body: -**Verification Body:** ```json { - "address": "0x...", - "source_code": "...", - "contract_name": "MyContract", + "source_code": "contract C {}", + "standard_json_input": null, + "contract_name": "C", "compiler_version": "v0.8.19+commit.7dd6d404", "optimization_enabled": true, "optimization_runs": 200, - "constructor_args": "0x...", + "constructor_args": "0x", "evm_version": "paris", - "license_type": "MIT", - "is_standard_json": false + "license_type": "MIT" } ``` -### Proxy Contracts +## Search + +| Method | Path | Query | Description | +| --- | --- | --- | --- | +| GET | `/api/search` | `q` required | Universal search | + +Search result types: + +- `block` +- `transaction` +- `address` +- `nft_collection` +- `nft` +- `erc20_token` + +Hex queries search addresses, transactions, and block hashes. Numeric queries search block numbers. Text queries of at least three characters search NFT collections, NFT token names, and ERC-20 names/symbols with wildcard escaping. + +## Stats + +| Method | Path | Query | Description | +| --- | --- | --- | --- | +| GET | `/api/stats/blocks-chart` | `window` | Bucketed transaction count and average gas used | +| GET | `/api/stats/daily-txs` | none | Daily transaction counts for the last 14 indexed days | +| GET | `/api/stats/gas-price` | `window` | Bucketed average gas price with base-fee fallback | + +Stats windows support `1h`, `6h`, `24h`, `7d`, `1m`, `6m`, and `1y`. + +## Faucet + +Faucet routes are registered only when `FAUCET_ENABLED=true` and required faucet settings are valid. | Method | Path | Description | -|--------|------|-------------| -| GET | `/api/proxies` | List detected proxy contracts | -| GET | `/api/contracts/:address/proxy` | Get proxy info | -| GET | `/api/contracts/:address/combined-abi` | Get merged proxy + implementation ABI | -| POST | `/api/contracts/:address/detect-proxy` | Trigger proxy detection | +| --- | --- | --- | +| GET | `/api/faucet/info` | Faucet amount, backend balance, and cooldown | +| POST | `/api/faucet` | Request funds for an address | -**Proxy Types**: `eip1967`, `eip1822`, `transparent`, `custom` +`POST /api/faucet` body: + +```json +{ + "address": "0x0000000000000000000000000000000000000001" +} +``` -### Search +Success: -| Method | Path | Parameters | Description | -|--------|------|------------|-------------| -| GET | `/api/search` | `q` (required) | Universal search | +```json +{ + "tx_hash": "0x..." +} +``` -Searches across: -- Block numbers -- Transaction hashes -- Addresses -- Contract/token names +Cooldown failures return `429 Too Many Requests` with retry metadata. ## Etherscan-Compatible API -For tooling compatibility, the following Etherscan-style endpoints are supported: - -### Account Module +The exact path `/api` supports Etherscan-style query parameters and response shape: +```json +{ + "status": "1", + "message": "OK", + "result": {} +} ``` + +Supported modules and actions: + +```text GET /api?module=account&action=balance&address=0x... GET /api?module=account&action=balancemulti&address=0x...,0x... GET /api?module=account&action=txlist&address=0x... GET /api?module=account&action=txlistinternal&address=0x... GET /api?module=account&action=tokentx&address=0x... GET /api?module=account&action=tokenbalance&address=0x...&contractaddress=0x... -``` -### Contract Module - -``` GET /api?module=contract&action=getabi&address=0x... GET /api?module=contract&action=getsourcecode&address=0x... -POST /api?module=contract&action=verifysourcecode -``` - -### Transaction Module -``` GET /api?module=transaction&action=gettxreceiptstatus&txhash=0x... -``` - -### Block Module -``` GET /api?module=block&action=getblockreward&blockno=123 -``` - -### Proxy Module (RPC) -``` GET /api?module=proxy&action=eth_blockNumber -GET /api?module=proxy&action=eth_getBlockByNumber&tag=0x...&boolean=true +GET /api?module=proxy&action=eth_getBlockByNumber&blockno=0x1 GET /api?module=proxy&action=eth_getTransactionByHash&txhash=0x... ``` -## Notes - -- All address parameters accept with or without `0x` prefix -- Addresses are case-insensitive (normalized to lowercase) -- Transaction hashes accept with or without `0x` prefix +Unsupported modules or actions return an Etherscan-style error payload. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 91ffdc6..ab59a54 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,46 +1,150 @@ # Architecture -## Overview +Atlas is a single-chain EVM explorer for ev-node based chains. The backend is one Rust process, `atlas-server`, that combines the indexer, HTTP API, live event stream, and background workers. -Atlas is a modular Ethereum L2 blockchain indexer and API server built in Rust. The `atlas-server` process runs both the indexer and the HTTP API, indexing blocks, transactions, ERC-20 tokens, and NFTs from any EVM-compatible chain. +## System Overview -## System Diagram - -``` -┌────────────────────────────────────────────────────────────────────┐ -│ atlas-server process │ -│ │ -│ ┌────────────────────┐ post-commit publish ┌──────────┐ │ -│ │ Indexer │ ────────────────────────────► │HeadTracker│ │ -│ │ • RPC block fetch │ │ latest │ │ -│ │ • Batch assembly │ │ live tail │ │ -│ │ • DB writes │ └─────┬────┘ │ -│ └─────────┬──────────┘ │ │ -│ │ │ │ -│ ▼ ▼ │ -│ ┌────────────────────┐ ┌────────────────┐│ -│ │ PostgreSQL │ │ HTTP API ││ -│ │ canonical history │ │ • REST ││ -│ │ blocks/indexes │ │ • SSE events ││ -│ └────────────────────┘ └────────────────┘│ -└────────────────────────────────────────────────────────────────────┘ - ▲ - │ - ┌─────────────────────┐ - │ Ethereum Node │ - │ (JSON-RPC) │ - └─────────────────────┘ +```text + JSON-RPC + +----------------+ + | EVM / ev-node | + +-------+--------+ + | + v ++-------------------------------------------------------------------+ +| atlas-server | +| | +| +--------------+ +----------------+ +-------------+ | +| | Block fetch | ----> | Batch builder | ----> | Binary COPY | | +| | workers | | logs/tokens | | writer | | +| +--------------+ +----------------+ +------+------+ | +| | | +| +----------------+ +----------------+ +-------------+-------+| +| | Metadata | | Gap fill | | DA worker || +| | worker | | worker | | optional ev-node || +| +----------------+ +----------------+ +-------------+-------+| +| | | +| +----------------+ +----------------+ +-------------+-------+| +| | Axum REST API | | SSE /api/events| | Prometheus metrics || +| +----------------+ +----------------+ +---------------------+| ++-----------------------------+-------------------------------------+ + | + v + +----------+ + | Postgres | + +----------+ + ^ + | + +---------+----------+ + | atlas-frontend | + | nginx + Vite build | + +--------------------+ ``` -## Project Structure +## Repository Layout -``` +```text atlas/ -├── backend/ -│ ├── crates/ -│ │ ├── atlas-common/ # Shared types, DB models, error handling -│ │ └── atlas-server/ # Combined indexer + API server (Axum) -│ └── migrations/ # PostgreSQL migrations -├── frontend/ # React frontend (Vite + Tailwind) -└── docker-compose.yml ++-- backend/ +| +-- Cargo.toml +| +-- crates/ +| | +-- atlas-common/ # Shared models, errors, pagination, DB helpers +| | +-- atlas-server/ # API, indexer, CLI, workers, metrics +| +-- migrations/ # SQLx migrations ++-- frontend/ # Vite app, Bun, Tailwind, Preact compatibility ++-- branding/ # Optional mounted logo/static branding assets ++-- docs/ ++-- docker-compose.yml ++-- .env.example ``` + +## Backend Process + +`atlas-server run` starts: + +- A migration pass before serving traffic. +- Separate SQLx pools for API and indexer work. +- The block fetch pipeline with configurable batch size, fetch workers, RPC batch size, and RPC rate limit. +- A binary COPY writer using a direct `tokio-postgres` connection. +- An Axum router with REST endpoints, Etherscan-compatible API, metrics, and health probes. +- An in-process `HeadTracker` used by `/api/height` and `/api/events`. +- The missed-block gap-fill worker. +- The NFT metadata worker. +- Optional DA tracking, faucet backend, and snapshot scheduler. + +The `check` subcommand validates configuration and DB/RPC connectivity. The `migrate` subcommand runs migrations and exits. The `db` subcommands provide dump, restore, and indexed-data reset operations. + +## Indexing Flow + +1. The indexer resumes from stored state or `START_BLOCK`. +2. Fetch workers pull blocks and receipts from the configured EVM JSON-RPC endpoint. +3. The batch builder extracts blocks, transactions, addresses, event logs, ERC-20 transfers/balances, NFT transfers/ownership, contract metadata, and failed block state. +4. The writer persists batches with binary COPY and ordinary SQL where appropriate. +5. Committed batches update the `HeadTracker` and publish live block events. +6. Failed blocks are recorded for the gap-fill worker, which retries with backoff and clears recovered rows atomically. + +The fresh-chain block-zero path is supported; recent underflow fixes ensure initial local-chain indexing works from `START_BLOCK=0`. + +## Background Workers + +### NFT Metadata + +NFT tokens move through a metadata state machine: + +- `pending` +- `fetched` +- `retryable_error` +- `permanent_error` + +The worker resolves IPFS, Arweave, `data:` URI, direct image, and HTTP metadata references, applies SSRF and payload protections, classifies failures, stores retry timestamps, and exposes metadata status through NFT APIs. + +### Data Availability Tracking + +When `ENABLE_DA_TRACKING=true`, Atlas queries ev-node using `EVNODE_URL` for Celestia header and data inclusion heights. The worker backfills known blocks, updates pending entries, stores results in `block_da_status`, and emits `da_batch` SSE events for visible UI updates. + +### Snapshots + +When `SNAPSHOT_ENABLED=true`, a scheduler runs daily `pg_dump` snapshots at `SNAPSHOT_TIME` UTC, writes custom-format dump files into `SNAPSHOT_DIR`, and keeps the newest `SNAPSHOT_RETENTION` completed dumps. + +## API Layer + +The API is built with Axum. Most routes have: + +- 10 second HTTP timeout returning `408`. +- API pool `statement_timeout = '10s'`. +- HTTP request metrics collected by route. +- CORS allowing all origins by default or a single `CORS_ORIGIN` when configured. + +Routes excluded from the 10 second HTTP timeout: + +- `/api/events`, because SSE connections are long-lived. +- `/api/contracts/{address}/verify`, because Solidity compilation can take longer and accepts up to 50 MiB request bodies. + +## Database + +PostgreSQL stores canonical explorer history and derived views. Large tables are optimized for explorer access patterns: + +- Blocks avoid large `OFFSET` scans by using a cursor derived from the highest indexed block and the clamped page limit. +- Large table counts use `pg_class.reltuples` estimates where appropriate, with exact counts for small tables. +- Transactions use lookup tables and partition-aware queries where needed. +- Migrations run with a dedicated one-connection pool that is not constrained by the API statement timeout. + +## Frontend and Deployment + +The frontend is a Vite build served by unprivileged nginx: + +- Container port `8080`, exposed as host port `80` in Docker Compose. +- `/api/` and exact `/api` are proxied to `atlas-server:3000`. +- `/api/events` is proxied with buffering disabled. +- `/branding/` serves mounted branding assets. +- SPA routes fall back to `index.html`. + +In development, Vite serves on `5173` and proxies browser `/api/...` requests to `localhost:3000`. + +## Observability + +- `/metrics` exposes Prometheus text format. +- `/health/live` reports process liveness. +- `/health/ready` checks DB connectivity and recent indexer state. +- Logs support text or JSON format through `LOG_FORMAT`. +- Metrics include indexer head, missing block state, HTTP route metrics, and SSE connection accounting. diff --git a/docs/FEATURES.md b/docs/FEATURES.md new file mode 100644 index 0000000..c5eba1a --- /dev/null +++ b/docs/FEATURES.md @@ -0,0 +1,129 @@ +# Features + +This document describes the implemented Atlas feature set on `origin/main` through merged PR #79. + +## Explorer Core + +- Latest blocks and block details, including base fee, gas, transaction count, and optional DA status. +- Transaction lists and transaction details with status, value, gas, input data, transfers, and logs. +- Address pages with transaction history, ERC-20 balances, NFT ownership, transfers, and a contract tab for verified contracts. +- Universal search for block numbers, block hashes, transaction hashes, addresses, NFT collections, NFT token names, and ERC-20 names/symbols. +- Status page with chain ID, chain name, indexed height, estimated totals, and charts. +- Live block updates through Server-Sent Events with `/api/height` polling fallback. + +## Tokens and NFTs + +### ERC-20 + +- Detects ERC-20 transfers from event logs. +- Stores token contracts, transfers, balances, holders, and indexed supply history where available. +- Exposes token list, token detail, holders, transfer history, address balances, transaction token transfers, and token chart data. +- Recomputes indexed total supply from balances when complete history is available. + +### NFTs + +- Detects NFT contracts and transfers. +- Tracks token ownership and transfer history. +- Fetches and stores token metadata, image URL, name, full JSON metadata, retry status, and error state. +- Shows pending/unavailable metadata states in the frontend. +- NFT token pages display common fields and provide a raw JSON toggle for full metadata inspection. +- NFT token name search uses trigram indexing for fast text lookup. + +## Event Logs and Decoding + +- Indexes raw event logs for transactions and emitting addresses. +- Provides transaction logs and decoded transaction logs. +- Uses known event signatures and verified ABIs where available. +- Supports address log pagination with explicit `page` and `limit` query parsing. + +## Contract Verification and Proxies + +- Verifies Solidity contracts through `POST /api/contracts/{address}/verify`. +- Supports standard JSON input and single-source verification. +- Downloads and caches solc binaries in `SOLC_CACHE_DIR`. +- Stores source, compiler settings, ABI, bytecode metadata, and verification status. +- Displays verified source and ABI in the frontend contract tab. +- Decodes transaction input against verified ABIs where possible. +- Detects proxy contracts and exposes proxy info plus combined proxy/implementation ABI. + +## Etherscan-Compatible API + +Atlas supports Etherscan-style responses at exact `/api` for tooling compatibility: + +- `account`: balance, multi-balance, tx list, internal tx placeholder, token transfers, token balance. +- `contract`: get ABI, get source code. +- `transaction`: receipt status. +- `block`: block reward. +- `proxy`: selected RPC passthrough actions. + +The native contract verification route is `/api/contracts/{address}/verify`; the current Etherscan-compatible contract module does not implement `verifysourcecode`. + +## Live Updates + +- `/api/events` streams committed indexed blocks as `new_block` events. +- When DA tracking is enabled, the same stream emits `da_batch` updates. +- The frontend buffers and de-duplicates live blocks, then falls back to polling when SSE disconnects. +- `SSE_REPLAY_BUFFER_BLOCKS` controls the in-memory block replay buffer for connected clients. + +## Data Availability Tracking + +Optional DA tracking uses ev-node to record Celestia inclusion heights: + +- Enable with `ENABLE_DA_TRACKING=true`. +- Configure ev-node with `EVNODE_URL`. +- Tune with `DA_WORKER_CONCURRENCY` and `DA_RPC_REQUESTS_PER_SECOND`. +- API and frontend expose header/data DA heights for blocks. +- DA UI is feature-gated by `/api/config.features.da_tracking`. + +## Faucet + +Optional faucet support: + +- Enable with `FAUCET_ENABLED=true`. +- Configure `FAUCET_PRIVATE_KEY`, `FAUCET_AMOUNT`, and `FAUCET_COOLDOWN_MINUTES`. +- `/api/config` exposes whether the faucet is enabled and its public amount/cooldown metadata. +- `/api/faucet/info` returns faucet balance and settings. +- `/api/faucet` sends funds and returns a transaction hash. +- Cooldown failures return `429` with retry metadata. + +## White Labeling + +Branding is runtime configurable through environment variables: + +- Chain name. +- Default, light-theme, and dark-theme logos. +- Accent, background, success, and error colors. +- Mounted `/branding/` assets in Docker. +- `/api/config` as the frontend runtime source of truth. + +No frontend rebuild is required for branding changes when using runtime environment configuration and mounted assets. + +## Operations + +- `atlas-server check` validates DB/RPC configuration. +- `atlas-server migrate` runs SQLx migrations. +- `atlas-server run` starts API, indexer, and workers. +- `atlas-server db dump` creates portable `pg_dump` custom-format backups. +- `atlas-server db restore` restores backups after resetting the public schema. +- `atlas-server db reset --confirm` truncates indexed data while preserving schema and migrations. +- Scheduled snapshots can run daily through env-only `SNAPSHOT_*` settings. + +## Observability and Reliability + +- Prometheus metrics at `/metrics`. +- Liveness and readiness probes at `/health/live` and `/health/ready`. +- HTTP route metrics with matched route labels. +- 10 second API timeout and database statement timeout for request-serving pools. +- Timeout-free migration pool for long index builds. +- Missed-block gap-fill worker with retry/backoff. +- RPC request rate limiting and batch fetching. +- Binary COPY batch writes for high-throughput indexing. + +## Current Non-Goals and Gaps + +- Multi-chain indexing in one Atlas deployment. +- User accounts or per-user saved settings. +- ERC-1155 indexing. +- Full trace/internal transaction indexing. +- Browser-based contract write interactions. +- Native label management API routes. diff --git a/docs/FEATURES_V2.md b/docs/FEATURES_V2.md deleted file mode 100644 index 5d6d70a..0000000 --- a/docs/FEATURES_V2.md +++ /dev/null @@ -1,353 +0,0 @@ -# Atlas v2 Features - -Post-MVP feature additions based on Blockscout analysis. Implement after core explorer is stable. - ---- - -## Priority 1: Essential - -### 1.1 ERC-20 Token Support - -ERC-721-only is insufficient for a general-purpose explorer. - -**Data Model** - -``` -ERC20Contract: -- address: bytes20 (PK) -- name: string -- symbol: string -- decimals: u8 -- total_supply: u256 (optional, not all tokens have this) -- first_seen_block: u64 - -ERC20Transfer: -- id: serial (PK) -- tx_hash: bytes32 (FK) -- log_index: u32 -- contract_address: bytes20 (FK) -- from_address: address -- to_address: address -- value: u256 -- block_number: u64 -- timestamp: u64 - -ERC20Balance: -- address: bytes20 (PK) -- contract_address: bytes20 (PK) -- balance: u256 -- last_updated_block: u64 -``` - -**API Endpoints** - -- `GET /api/tokens` - List ERC-20 tokens -- `GET /api/tokens/:address` - Token detail (name, symbol, decimals, holders count) -- `GET /api/tokens/:address/holders` - Paginated holder list -- `GET /api/tokens/:address/transfers` - Token transfer history -- `GET /api/addresses/:address/tokens` - ERC-20 balances for address - -**Indexer Changes** - -- Detect ERC-20 via Transfer(address,address,uint256) events (same signature as ERC-721, differentiate by topic count) -- Call `name()`, `symbol()`, `decimals()` on detection -- Track balances incrementally from transfer events - ---- - -### 1.2 Event Log Decoding - -Show all emitted events, not just Transfer. - -**Data Model** - -``` -EventLog: -- id: serial (PK) -- tx_hash: bytes32 (FK) -- log_index: u32 -- address: bytes20 -- topic0: bytes32 (event signature) -- topic1: bytes32 (nullable) -- topic2: bytes32 (nullable) -- topic3: bytes32 (nullable) -- data: bytes -- block_number: u64 -- decoded: jsonb (nullable, populated if ABI known) -``` - -**API Endpoints** - -- `GET /api/transactions/:hash/logs` - All logs for transaction -- `GET /api/addresses/:address/logs` - Logs emitted by contract (paginated) -- `GET /api/logs?topic0=:sig` - Filter logs by event signature - -**Decoding Strategy** - -1. Maintain a table of known event signatures (4bytes.directory or curated list) -2. For verified contracts, use stored ABI -3. Store decoded JSON when possible, raw otherwise - ---- - -### 1.3 Etherscan-Compatible API - -Required for tooling ecosystem compatibility. - -**Endpoints to Implement** - -``` -# Account -/api?module=account&action=balance&address=:addr -/api?module=account&action=balancemulti&address=:addr1,:addr2 -/api?module=account&action=txlist&address=:addr -/api?module=account&action=txlistinternal&address=:addr -/api?module=account&action=tokentx&address=:addr -/api?module=account&action=tokenbalance&address=:addr&contractaddress=:token - -# Contract -/api?module=contract&action=getabi&address=:addr -/api?module=contract&action=getsourcecode&address=:addr -/api?module=contract&action=verifysourcecode (POST) - -# Transaction -/api?module=transaction&action=gettxreceiptstatus&txhash=:hash - -# Block -/api?module=block&action=getblockreward&blockno=:num - -# Proxy (pass-through to RPC) -/api?module=proxy&action=eth_blockNumber -/api?module=proxy&action=eth_getBlockByNumber -/api?module=proxy&action=eth_getTransactionByHash -``` - -**Implementation Notes** - -- Response format must match Etherscan exactly (status, message, result fields) -- Hardhat and Foundry verify plugins depend on specific response shapes -- Consider rate limiting per API key (optional) - ---- - -### 1.4 Address Labels - -Zero-complexity UX improvement. - -**Implementation** - -Config file (`labels.json` or database table): - -```json -{ - "0x1234...": { - "name": "Bridge Contract", - "tags": ["infrastructure", "bridge"] - }, - "0x5678...": { - "name": "Governance", - "tags": ["governance"] - } -} -``` - -**Features** - -- Display label instead of/alongside raw address -- Filter addresses by tag -- Admin endpoint to add/update labels (or file reload) -- No user accounts needed - curated list only - ---- - -## Priority 2: High Value - -### 2.1 Internal Transactions (Traces) - -Required for accurate value tracking in contract interactions. - -**Prerequisites** - -- L2 node must support `debug_traceTransaction` or `trace_transaction` -- Significantly increases indexing time and storage - -**Data Model** - -``` -InternalTransaction: -- id: serial (PK) -- tx_hash: bytes32 (FK) -- trace_address: int[] (e.g., [0, 1, 2] for nested calls) -- call_type: enum (call, delegatecall, staticcall, create, create2, selfdestruct) -- from_address: address -- to_address: address -- value: u256 -- gas: u64 -- gas_used: u64 -- input: bytes -- output: bytes -- error: string (nullable) -- block_number: u64 -``` - -**API Endpoints** - -- `GET /api/transactions/:hash/internal` - Internal txs for transaction -- `GET /api/addresses/:address/internal` - Internal txs involving address - -**Indexer Changes** - -- Add trace fetching step after receipt processing -- Consider making this optional (config flag) due to performance impact - ---- - -### 2.2 Read/Write Contract Interaction - -Direct contract interaction from explorer UI. - -**Read Functions** - -- Parse ABI from verified contracts -- Generate UI form for view/pure functions -- Execute via `eth_call` and display results -- No wallet connection needed - -**Write Functions** - -- Generate UI form for state-changing functions -- Connect wallet (WalletConnect, injected provider) -- Estimate gas, submit transaction -- Show pending status, link to tx page on confirmation - -**API Support** - -- `POST /api/contracts/:address/call` - Execute read function - ```json - { - "function": "balanceOf", - "args": ["0x1234..."] - } - ``` - ---- - -### 2.3 Proxy Contract Detection - -Critical if L2 uses upgradeable contracts. - -**Patterns to Detect** - -| Pattern | Storage Slot | -|---------|--------------| -| EIP-1967 | `0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bbc` | -| EIP-1822 (UUPS) | `0xc5f16f0fcc639fa48a6947836d9850f504798523bf8c9a3a87d5876cf622bcf7` | -| OpenZeppelin Transparent | Admin slot at `0xb53127684a568b3173ae13b9f8a6016e243e63b6e8ee1178d6a717850b5d6103` | - -**Implementation** - -1. On contract detection, check known proxy storage slots -2. If implementation address found, link contracts -3. When viewing proxy, show implementation source/ABI -4. Handle nested proxies (proxy -> proxy -> impl) - -**UI Changes** - -- Badge indicating "Proxy Contract" -- Link to implementation -- Show combined ABI (proxy + implementation) for interaction - ---- - -## Priority 3: Nice to Have - -### 3.1 Basic Chain Analytics - -Minimal analytics without a full dashboard system. - -**Metrics to Track** - -``` -DailyStats: -- date: date (PK) -- block_count: u32 -- tx_count: u32 -- unique_addresses: u32 (senders + receivers) -- gas_used: u256 -- avg_gas_price: u256 -``` - -**API Endpoints** - -- `GET /api/stats/daily?from=:date&to=:date` - Daily stats -- `GET /api/stats/summary` - Current chain state (latest block, total txs, etc.) - -**Implementation** - -- Aggregate during indexing or via scheduled job -- Simple line charts on homepage - ---- - -### 3.2 State Diffs - -Show storage changes per transaction. - -**Data Model** - -``` -StateDiff: -- id: serial (PK) -- tx_hash: bytes32 (FK) -- address: bytes20 -- slot: bytes32 -- previous_value: bytes32 -- new_value: bytes32 -``` - -**Requirements** - -- Requires `debug_traceTransaction` with `prestateTracer` or `stateDiffTracer` -- High storage cost - consider storing only for recent blocks or on-demand - ---- - -## Implementation Order - -``` -Phase 1 (Post-MVP Stabilization) -├── 1.4 Address Labels (trivial, immediate UX win) -├── 1.1 ERC-20 Token Support -└── 1.2 Event Log Decoding - -Phase 2 -├── 1.3 Etherscan-Compatible API -├── 2.3 Proxy Contract Detection -└── 2.2 Read/Write Contract Interaction - -Phase 3 (If Needed) -├── 2.1 Internal Transactions -├── 3.1 Basic Chain Analytics -└── 3.2 State Diffs -``` - ---- - -## Dependencies & Risks - -| Feature | Dependency | Risk | -|---------|------------|------| -| Internal Transactions | `debug_*` RPC methods | Node may not support, 10x+ indexing time | -| State Diffs | `debug_traceTransaction` | Same as above | -| Write Contract | Wallet integration | Frontend complexity, security surface | -| ERC-20 Balances | Accurate transfer indexing | Reorg handling critical for balance accuracy | - ---- - -## Non-Goals (Unchanged from v1) - -- Multi-chain support -- User accounts / authentication -- Gas price oracles -- Full analytics dashboards -- ERC-1155 support (reconsider if needed) diff --git a/docs/PRD.md b/docs/PRD.md index 3ef50a7..0e5e8e8 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -1,449 +1,108 @@ -# Atlas - Lightweight Ethereum L2 Block Explorer +# Product and Roadmap -## Overview +This document describes the current Atlas product state and the remaining roadmap. It replaces the older MVP checklist, which is now mostly implemented. -Atlas is a minimal, fast blockchain explorer for custom Ethereum L2 networks. It replaces Blockscout with a focused feature set, trading breadth for simplicity and performance. +## Product Positioning -**Target users:** Development team, community members, general public. +Atlas is a lightweight EVM explorer for one ev-node based chain per deployment. It prioritizes operational simplicity, high-throughput indexing, useful public APIs, and white-label deployment over Blockscout-scale breadth. ---- +Primary audiences: -## Goals +- Chain operators deploying an explorer for an ev-node based chain. +- Developers inspecting blocks, transactions, contracts, ERC-20 tokens, and NFTs. +- Community users browsing chain activity. -1. **Simplicity** - Lean codebase, minimal dependencies, easy to operate -2. **Performance** - Fast queries, responsive UI, efficient indexing -3. **Full history** - Complete chain indexing with reindex capability -4. **NFT-first** - Rich NFT metadata indexing and display +## Implemented Product Areas -## Non-Goals +### Core Explorer -- Multi-chain support (single L2 only) -- ERC-1155 support (ERC-721 only) -- User accounts or authentication -- Smart contract IDE/debugging -- Gas price oracles or analytics dashboards +- Blocks, block details, and block transaction lists. +- Transactions, transaction details, logs, ERC-20 transfers, and NFT transfers. +- Address list and address detail pages. +- Universal search for chain entities and token names. +- Status dashboard with chain metadata, indexed height, totals, and charts. +- Live block updates through SSE with polling fallback. ---- +### Indexing -## Architecture +- Batch block and receipt fetching from EVM JSON-RPC. +- Configurable start block, batch size, fetch workers, RPC batch size, and RPC rate limit. +- Binary COPY bulk writes for high indexing throughput. +- Resume from persisted indexer state. +- Reindex mode for wiping indexed data and rebuilding. +- Failed-block recording and gap-fill recovery worker. -``` -┌─────────────┐ ┌─────────────┐ ┌─────────────┐ -│ React │────▶│ Rust API │────▶│ Postgres │ -│ Frontend │ │ (Axum) │ │ │ -└─────────────┘ └──────┬──────┘ └─────────────┘ - │ ▲ - │ │ - ┌──────▼──────┐ │ - │ Indexer │────────────┘ - │ (async) │ - └──────┬──────┘ - │ - ┌──────▼──────┐ - │ L2 Node │ - │ (RPC) │ - └─────────────┘ -``` +### Tokens and NFTs -### Components +- ERC-20 detection, contracts, balances, holders, transfers, and charts. +- NFT collection/token indexing, ownership, transfer history, and token detail pages. +- NFT metadata state machine with retryable and permanent error handling. +- IPFS/Arweave/data/HTTP metadata resolution with safety checks. +- Full NFT metadata display with raw JSON inspection. +- NFT token name search backed by trigram indexing. -| Component | Tech | Purpose | -|-----------|------|---------| -| **API Server** | Rust (Axum) | REST API, serves frontend | -| **Indexer** | Rust (tokio) | Polls RPC, indexes blocks/txs/NFTs | -| **Database** | PostgreSQL | Persistent storage, full-text search | -| **Frontend** | React | UI, can be SSR or SPA | -| **Metadata Fetcher** | Rust (async) | Resolves and caches NFT tokenURIs | +### Contracts ---- +- Solidity source verification via native API and frontend contract tab. +- solc compiler download/cache support. +- Verified source, ABI, compiler settings, and metadata storage. +- Transaction input decoding against available ABIs. +- Proxy detection and combined ABI exposure. -## Data Model +### Operations -### Core Entities +- Single `atlas-server` binary for API, indexer, workers, and CLI utilities. +- Docker Compose deployment with `postgres`, `atlas-server`, and `atlas-frontend`. +- Health probes, Prometheus metrics, structured logging option, and request timeouts. +- Scheduled database snapshots and manual dump/restore/reset commands. +- Runtime white-label branding and optional faucet support. +- Optional DA inclusion tracking from ev-node. -#### Block +## Current Non-Goals -``` -- number: u64 (PK) -- hash: bytes32 -- parent_hash: bytes32 -- timestamp: u64 -- gas_used: u64 -- gas_limit: u64 -- transaction_count: u32 -- indexed_at: timestamp -``` +- Multi-chain indexing in one deployment. +- Account systems, user preferences, or explorer login. +- ERC-1155 indexing. +- Full EVM trace/internal transaction indexing. +- Gas oracle service or mempool analytics. +- Browser-based contract write UI. +- Hosted image caching/proxying for NFT media. -#### Transaction +## Roadmap Candidates -``` -- hash: bytes32 (PK) -- block_number: u64 (FK) -- block_index: u32 -- from_address: address -- to_address: address (nullable, for contract creation) -- value: u256 -- gas_price: u256 -- gas_used: u64 -- input_data: bytes -- status: bool -- contract_created: address (nullable) -- timestamp: u64 -``` +These are not committed interfaces; they are candidate future work. -#### Address +| Area | Candidate | +| --- | --- | +| ERC-1155 | Index multi-token transfers, balances, metadata, and UI pages | +| Traces | Optional `debug_traceTransaction` or `trace_transaction` ingestion for internal calls | +| Labels | Native address label management API and import/export workflow | +| Contract interaction | Read-only contract calls from verified ABIs; later wallet-backed write flows | +| Search | Attribute search for NFTs and richer ranked text search | +| Media | Optional NFT image proxy/cache for unreliable upstream metadata | +| Operations | More deployment examples for Kubernetes and managed Postgres | +| API compatibility | Additional Etherscan-compatible actions where tools need them | -``` -- address: bytes20 (PK) -- is_contract: bool -- first_seen_block: u64 -- tx_count: u32 -- balance: u256 (optional, can be fetched live) -``` +## Success Criteria -#### NFT Contract (ERC-721) +Atlas is healthy when: -``` -- address: bytes20 (PK) -- name: string -- symbol: string -- total_supply: u64 (if enumerable) -- first_seen_block: u64 -``` +- `atlas-server check` passes against the target DB and RPC endpoint. +- `/health/live` returns `200`. +- `/health/ready` returns `200` while the indexer is fresh. +- `/metrics` exposes Prometheus data. +- `/api/height` advances as blocks are indexed. +- The frontend can load `/api/config`, show the configured chain name, and browse blocks. +- Reindexing from `START_BLOCK=0` works on a fresh local chain. +- NFT metadata failures are visible as retryable or permanent states rather than silent missing data. -#### NFT Token +## Acceptance Baseline for New Features -``` -- contract_address: bytes20 (PK) -- token_id: u256 (PK) -- owner: address -- token_uri: string -- metadata_fetched: bool -- metadata: jsonb (nullable) -- image_url: string (nullable) -- last_transfer_block: u64 -``` +New product features should include: -#### NFT Transfer - -``` -- id: serial (PK) -- tx_hash: bytes32 (FK) -- log_index: u32 -- contract_address: bytes20 -- token_id: u256 -- from_address: address -- to_address: address -- block_number: u64 -- timestamp: u64 -``` - ---- - -## Features - -### P0 - Core (MVP) - -#### Block Explorer - -- [ ] View latest blocks with pagination -- [ ] Block detail page (hash, txs, gas, timestamp) -- [ ] Navigate between blocks (prev/next) - -#### Transaction Explorer - -- [ ] View transactions in a block -- [ ] Transaction detail page (hash, from, to, value, gas, input data, status) -- [ ] Decode common function selectors (transfer, approve, etc.) - -#### Address Pages - -- [ ] View address balance (live RPC call) -- [ ] List transactions for address (sent/received) -- [ ] Identify contract vs EOA -- [ ] List NFTs owned by address - -#### NFT Explorer - -- [ ] List all indexed ERC-721 contracts -- [ ] View NFT collection page (name, symbol, tokens) -- [ ] View individual NFT (image, metadata, owner, transfer history) -- [ ] NFT transfer history per token - -#### Search - -- [ ] Search by transaction hash -- [ ] Search by address -- [ ] Search by block number/hash -- [ ] Search by NFT token name (from metadata) - -#### Indexer - -- [ ] Index blocks from genesis or configured start block -- [ ] Index transactions and receipts -- [ ] Detect and index ERC-721 contracts (via Transfer events) -- [ ] Queue NFT metadata fetching -- [ ] Support reindex from scratch (wipe and rebuild) -- [ ] Resume from last indexed block on restart - -### P1 - Enhanced - -#### NFT Metadata - -- [ ] Fetch and cache tokenURI metadata -- [ ] Handle IPFS URIs (via gateway) -- [ ] Handle HTTP URIs -- [ ] Store and display NFT attributes -- [ ] Retry failed metadata fetches - -#### Search Enhancements - -- [ ] Full-text search on NFT names/descriptions -- [ ] Search NFTs by attribute values - -#### UI Polish - -- [ ] Real-time block updates (websocket or polling) -- [ ] Copy-to-clipboard for addresses/hashes -- [ ] Mobile-responsive design -- [ ] Loading states and error handling - -### P2 - Nice to Have - -#### Contract Verification - -- [ ] Upload and verify Solidity source -- [ ] Display verified source code -- [ ] Decode transaction input against verified ABI - -#### API - -- [ ] Documented REST API -- [ ] Rate limiting -- [ ] API usage stats - ---- - -## Search Implementation - -| Query Type | Detection | Implementation | -|------------|-----------|----------------| -| Tx hash | 66 chars, starts with 0x | Direct lookup | -| Address | 42 chars, starts with 0x | Direct lookup | -| Block number | Numeric | Direct lookup | -| Block hash | 66 chars, starts with 0x | Direct lookup | -| NFT/Token name | String | Postgres full-text search on metadata | - ---- - -## Indexer Behavior - -### Startup - -1. Check last indexed block in DB -2. If reindex flag set, truncate tables and start from genesis/config -3. Otherwise, resume from last indexed block + 1 - -### Indexing Loop - -1. Fetch block via `eth_getBlockByNumber` (with txs) -2. Fetch receipts via `eth_getBlockReceipts` (or individual) -3. Parse Transfer events for ERC-721 detection -4. Insert block, transactions, addresses, NFT data -5. Queue metadata fetch jobs for new NFTs -6. Commit transaction -7. Update last indexed block -8. Sleep or continue based on head distance - -### Metadata Fetching - -- Separate async task pool -- Fetch tokenURI from contract -- Resolve URI (IPFS via configurable gateway, HTTP direct) -- Parse JSON metadata -- Store in database -- Retry with exponential backoff on failure - ---- - -## Configuration - -```toml -[rpc] -url = "https://your-l2-rpc.example.com" -requests_per_second = 100 # rate limit - -[database] -url = "postgres://user:pass@localhost/atlas" -max_connections = 20 - -[indexer] -start_block = 0 # or "genesis" -batch_size = 100 -reindex = false - -[metadata] -ipfs_gateway = "https://ipfs.io/ipfs/" -fetch_workers = 4 -retry_attempts = 3 - -[server] -host = "0.0.0.0" -port = 3000 -``` - ---- - -## API Endpoints (Draft) - -### Blocks - -- `GET /api/blocks` - List blocks (paginated) -- `GET /api/blocks/:number` - Block detail -- `GET /api/blocks/:number/transactions` - Transactions in block - -### Transactions - -- `GET /api/transactions/:hash` - Transaction detail - -### Addresses - -- `GET /api/addresses/:address` - Address detail -- `GET /api/addresses/:address/transactions` - Address transactions -- `GET /api/addresses/:address/nfts` - NFTs owned by address - -### NFTs - -- `GET /api/nfts/collections` - List NFT contracts -- `GET /api/nfts/collections/:address` - Collection detail -- `GET /api/nfts/collections/:address/tokens` - Tokens in collection -- `GET /api/nfts/collections/:address/tokens/:id` - Token detail -- `GET /api/nfts/collections/:address/tokens/:id/transfers` - Token transfers - -### Search - -- `GET /api/search?q=:query` - Universal search - ---- - -## Tech Stack - -| Layer | Choice | Rationale | -|-------|--------|-----------| -| Language | Rust | Performance, safety, low resource usage | -| Web framework | Axum | Async, ergonomic, well-maintained | -| Database | PostgreSQL | Full-text search, JSONB, reliable | -| ORM/Query | SQLx | Compile-time checked queries, async | -| HTTP client | reqwest | Async, well-maintained | -| RPC | ethers-rs or alloy | Ethereum types and RPC | -| Frontend | React | Familiar, adequate for scope | -| Styling | Tailwind CSS | Fast iteration, no custom CSS overhead | - ---- - -## Deployment - -### Minimum Requirements - -- 2 vCPU, 4GB RAM (indexer + API) -- PostgreSQL 14+ -- Network access to L2 RPC - -### Docker Compose (Development) - -```yaml -services: - postgres: - image: postgres:16 - environment: - POSTGRES_DB: atlas - POSTGRES_USER: atlas - POSTGRES_PASSWORD: atlas - volumes: - - pgdata:/var/lib/postgresql/data - ports: - - "5432:5432" - - atlas: - build: . - environment: - DATABASE_URL: postgres://atlas:atlas@postgres/atlas - RPC_URL: ${RPC_URL} - ports: - - "3000:3000" - depends_on: - - postgres - -volumes: - pgdata: -``` - ---- - -## Risks & Mitigations - -| Risk | Impact | Mitigation | -|------|--------|------------| -| RPC rate limiting | Slow indexing | Configurable rate limits, batch requests | -| Large metadata (images) | Storage bloat | Store URLs only, don't cache images | -| IPFS gateway unreliable | Missing metadata | Multiple gateway fallbacks, retry queue | -| Chain reorgs | Data inconsistency | Track finalized blocks, reorg handling | -| Full history = slow initial sync | Long bootstrap | Configurable start block, progress API | - ---- - -## Open Questions - -1. **Image caching** - Should we proxy/cache NFT images or just link to source? - - Recommendation: Link to source initially, add caching later if needed - -2. **Reorg handling** - How deep can reorgs be on your L2? - - Need to know finality guarantees to set confirmation depth - -3. **Token standards** - Any custom ERC-721 extensions in use? - - May need custom event parsing - -4. **Internal transactions** - Do you need to trace internal calls? - - Requires debug/trace APIs, significantly more complex - ---- - -## Timeline Estimate - -Not providing time estimates per instructions. Work breakdown: - -**Phase 1: Foundation** - -- Database schema and migrations -- Indexer core (blocks, transactions) -- Basic API endpoints - -**Phase 2: NFT Indexing** - -- ERC-721 detection and indexing -- Metadata fetching pipeline -- NFT API endpoints - -**Phase 3: Frontend** - -- Block/transaction views -- Address pages -- NFT gallery and detail views -- Search - -**Phase 4: Polish** - -- Real-time updates -- Error handling -- Performance optimization -- Documentation - ---- - -## Success Metrics - -- Indexer keeps up with chain head (< 10 block lag) -- API p99 latency < 200ms -- Search returns results < 500ms -- 95%+ NFT metadata successfully fetched -- Zero data loss on restart/crash +- Backend behavior documented in `docs/API.md` or `backend/README.md` when it changes public interfaces. +- Frontend behavior documented in `frontend/README.md` when it changes routes or runtime assumptions. +- Environment variables added to `.env.example` and the backend README. +- Unit tests for new Rust logic in the same file where practical. +- Integration tests for API behavior when the route or query semantics change. +- No use of large-table `OFFSET` pagination for block-scale data. diff --git a/docs/WHITE_LABELING.md b/docs/WHITE_LABELING.md index 9edf813..1280f3b 100644 --- a/docs/WHITE_LABELING.md +++ b/docs/WHITE_LABELING.md @@ -1,47 +1,86 @@ # White Labeling -Atlas supports white-labeling so each L2 chain can customize the explorer's appearance — name, logo, and color scheme — without rebuilding the frontend. +Atlas branding is configured at runtime through `atlas-server` environment variables and served to the frontend by `GET /api/config`. A frontend rebuild is not required for normal branding changes. -All branding is configured through environment variables. When none are set, the explorer uses the default ev-node branding (red accent, dark/warm-beige backgrounds). `CHAIN_NAME` defaults to "Unknown" — deployers should always set it. +## Runtime Config -## Configuration +| Variable | Description | Default | +| --- | --- | --- | +| `CHAIN_NAME` | Displayed in page title, navbar, welcome page, and `/api/status` | `Unknown` | +| `CHAIN_LOGO_URL` | Default logo URL or path | bundled logo | +| `CHAIN_LOGO_URL_LIGHT` | Logo used in light theme | falls back to `CHAIN_LOGO_URL` | +| `CHAIN_LOGO_URL_DARK` | Logo used in dark theme | falls back to `CHAIN_LOGO_URL` | +| `ACCENT_COLOR` | Primary interactive color | `#dc2626` | +| `BACKGROUND_COLOR_DARK` | Dark theme base background | `#050505` | +| `BACKGROUND_COLOR_LIGHT` | Light theme base background | `#f4ede6` | +| `SUCCESS_COLOR` | Success state color | `#22c55e` | +| `ERROR_COLOR` | Error state color | `#dc2626` | -Add these variables to your `.env` file alongside `RPC_URL`: +Example: -| Variable | Description | Default (ev-node) | -|----------|-------------|--------------------| -| `CHAIN_NAME` | Displayed in the navbar, page title, and welcome page | `Unknown` | -| `CHAIN_LOGO_URL` | URL or path to your logo (e.g. `/branding/logo.svg`) | Bundled ev-node logo | -| `ACCENT_COLOR` | Primary accent hex for links, buttons, active states | `#dc2626` | -| `BACKGROUND_COLOR_DARK` | Dark mode base background hex | `#050505` | -| `BACKGROUND_COLOR_LIGHT` | Light mode base background hex | `#f4ede6` | -| `SUCCESS_COLOR` | Success indicator hex (e.g. confirmed badges) | `#22c55e` | -| `ERROR_COLOR` | Error indicator hex (e.g. failed badges) | `#dc2626` | +```env +CHAIN_NAME=My Chain +CHAIN_LOGO_URL=/branding/logo.svg +CHAIN_LOGO_URL_LIGHT=/branding/logo-light.svg +CHAIN_LOGO_URL_DARK=/branding/logo-dark.svg +ACCENT_COLOR=#3b82f6 +BACKGROUND_COLOR_DARK=#070a12 +BACKGROUND_COLOR_LIGHT=#f5f7fb +SUCCESS_COLOR=#22c55e +ERROR_COLOR=#ef4444 +``` + +## `/api/config` + +The backend returns branding, feature flags, and faucet display metadata: + +```json +{ + "chain_name": "My Chain", + "logo_url": "/branding/logo.svg", + "logo_url_light": "/branding/logo-light.svg", + "logo_url_dark": "/branding/logo-dark.svg", + "accent_color": "#3b82f6", + "background_color_dark": "#070a12", + "background_color_light": "#f5f7fb", + "success_color": "#22c55e", + "error_color": "#ef4444", + "features": { + "da_tracking": false + }, + "faucet": { + "enabled": false + } +} +``` -All variables are optional. Unset variables use the ev-node defaults shown above. +Unset optional fields are omitted. The frontend normalizes this response, applies CSS custom properties, updates the favicon/title, and gates optional UI such as DA indicators and faucet navigation. -## Custom Logo +## Branding Assets -To use a custom logo, place your image file in a `branding/` directory at the project root and set `CHAIN_LOGO_URL` to its path: +Place assets in the repository-level `branding/` directory: ```text atlas/ -├── branding/ -│ └── logo.svg # Your custom logo -├── .env -├── docker-compose.yml -└── ... ++-- branding/ +| +-- logo.svg +| +-- logo-light.svg +| +-- logo-dark.svg ++-- docker-compose.yml ++-- .env ``` +Set URLs relative to the frontend origin: + ```env CHAIN_LOGO_URL=/branding/logo.svg +CHAIN_LOGO_URL_LIGHT=/branding/logo-light.svg +CHAIN_LOGO_URL_DARK=/branding/logo-dark.svg ``` -The logo appears in the navbar, the welcome page, and as the browser favicon. +## Docker -### Docker - -In Docker, the `branding/` directory is mounted into the frontend container as a read-only volume. This is configured automatically in `docker-compose.yml`: +Docker Compose mounts branding assets into the frontend container: ```yaml atlas-frontend: @@ -49,79 +88,48 @@ atlas-frontend: - ${BRANDING_DIR:-./branding}:/usr/share/nginx/html/branding:ro ``` -To use a different directory, set `BRANDING_DIR` in your `.env`: +To use another host directory: ```env -BRANDING_DIR=/path/to/my/assets +BRANDING_DIR=/path/to/branding-assets ``` -### Local Development +Restart the frontend container after changing mounted files if nginx or browser cache prevents immediate refresh. + +## Local Vite Development -For `bun run dev`, create a symlink so Vite's dev server can serve the branding files: +Expose repository branding assets through Vite's public directory: ```bash +mkdir -p branding cd frontend/public ln -s ../../branding branding ``` -## Color System - -### Accent Color - -`ACCENT_COLOR` sets the primary interactive color used for links, buttons, focus rings, and active indicators throughout the UI. - -### Background Colors - -Each theme (dark and light) takes a single base color. The frontend automatically derives a full surface palette from it: - -- **5 surface shades** (from darkest to lightest for dark mode, reversed for light mode) -- **Border color** -- **Text hierarchy** (primary, secondary, muted, subtle, faint) - -This means you only need to set one color per theme to get a cohesive palette. +Run the backend with the branding env vars set, then start Vite: -### Success and Error Colors - -`SUCCESS_COLOR` and `ERROR_COLOR` control status badges and indicators. For example, "Success" transaction badges use the success color, and "Failed" badges use the error color. - -## Examples - -### Blue theme - -```env -CHAIN_NAME=MegaChain -CHAIN_LOGO_URL=/branding/logo.png -ACCENT_COLOR=#3b82f6 -BACKGROUND_COLOR_DARK=#0a0a1a -BACKGROUND_COLOR_LIGHT=#e6f0f4 -``` - -### Green theme (Eden) +```bash +cd backend +cargo run --bin atlas-server -- run -```env -CHAIN_NAME=Eden -CHAIN_LOGO_URL=/branding/logo.svg -ACCENT_COLOR=#4ade80 -BACKGROUND_COLOR_DARK=#0a1f0a -BACKGROUND_COLOR_LIGHT=#e8f5e8 -SUCCESS_COLOR=#22c55e -ERROR_COLOR=#dc2626 +cd ../frontend +bun run dev ``` -### Minimal — just rename - -```env -CHAIN_NAME=MyChain -``` +## Color Behavior -Everything else stays default ev-node branding. +- `ACCENT_COLOR` drives links, active states, focus rings, and primary buttons. +- `BACKGROUND_COLOR_DARK` and `BACKGROUND_COLOR_LIGHT` are base colors; the frontend derives related surface, border, and text colors. +- `SUCCESS_COLOR` and `ERROR_COLOR` drive status badges and indicators. +- Theme-specific logos let deployers use different marks for light and dark backgrounds. -## How It Works +Use valid CSS hex colors. Invalid or empty values are ignored by the frontend normalization layer. -1. The backend reads branding env vars at startup and serves them via `GET /api/config` -2. The frontend fetches this config once on page load -3. CSS custom properties are set on the document root, overriding the defaults -4. Background surface shades are derived automatically using HSL color manipulation -5. The page title, navbar logo, and favicon are updated dynamically +## Deployment Checklist -No frontend rebuild is needed — just change the env vars and restart the API. +- Set `CHAIN_NAME`. +- Add logo assets to `branding/` or use absolute hosted image URLs. +- Set `CHAIN_LOGO_URL_LIGHT` and `CHAIN_LOGO_URL_DARK` when one logo does not work in both themes. +- Set accent/background colors and verify both light and dark mode. +- Confirm `GET /api/config` returns expected values. +- Open the frontend and verify navbar, welcome page, favicon, charts, buttons, badges, and optional feature-gated nav items. diff --git a/frontend/README.md b/frontend/README.md index 90fbe4e..24644da 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -1,110 +1,112 @@ # Atlas Frontend -React frontend for the Atlas blockchain explorer. +The frontend is a Vite app written with React-compatible components and bundled through Preact compatibility aliases. It is built with Bun and served in production by `nginxinc/nginx-unprivileged:alpine`. ## Tech Stack -- React 19 +- Vite 7 - TypeScript -- Vite +- Preact via `@preact/preset-vite` and `preact/compat` aliases for React imports +- React Router - Tailwind CSS -- Bun (package manager) +- Recharts +- Bun ## Development ```bash -# Install dependencies bun install - -# Start dev server bun run dev - -# Build for production bun run build - -# Run linter bun run lint - -# Preview production build bun run preview ``` -## Project Structure +The dev server listens on `http://localhost:5173` and proxies `/api` to `http://localhost:3000`, so browser requests still use the `/api/...` paths. -``` -frontend/ -├── src/ -│ ├── api/ # API client functions -│ │ ├── blocks.ts -│ │ ├── transactions.ts -│ │ ├── addresses.ts -│ │ ├── nfts.ts -│ │ ├── tokens.ts -│ │ ├── logs.ts -│ │ ├── labels.ts -│ │ ├── proxies.ts -│ │ └── search.ts -│ ├── components/ # Reusable UI components -│ │ ├── Layout.tsx -│ │ ├── Pagination.tsx -│ │ ├── AddressLink.tsx -│ │ ├── TxHashLink.tsx -│ │ ├── BlockLink.tsx -│ │ ├── EventLogs.tsx -│ │ ├── LabeledAddress.tsx -│ │ └── ProxyBadge.tsx -│ ├── hooks/ # React hooks for data fetching -│ │ ├── useBlocks.ts -│ │ ├── useTransactions.ts -│ │ ├── useAddresses.ts -│ │ ├── useNFTs.ts -│ │ ├── useTokens.ts -│ │ ├── useLogs.ts -│ │ ├── useLabels.ts -│ │ └── useProxies.ts -│ ├── pages/ # Page components -│ │ ├── HomePage.tsx -│ │ ├── BlocksPage.tsx -│ │ ├── BlockDetailPage.tsx -│ │ ├── TransactionsPage.tsx -│ │ ├── TransactionDetailPage.tsx -│ │ ├── AddressPage.tsx -│ │ ├── NFTsPage.tsx -│ │ ├── NFTContractPage.tsx -│ │ ├── NFTTokenPage.tsx -│ │ ├── TokensPage.tsx -│ │ ├── TokenDetailPage.tsx -│ │ └── SearchPage.tsx -│ ├── types/ # TypeScript type definitions -│ │ └── index.ts -│ ├── utils/ # Utility functions -│ │ └── format.ts -│ ├── App.tsx # Router configuration -│ └── main.tsx # Entry point -├── public/ # Static assets -├── index.html -├── package.json -├── bun.lock -├── vite.config.ts -├── tailwind.config.js -└── tsconfig.json +## API Base URL + +The API client reads: + +```env +VITE_API_BASE_URL=http://localhost:3000/api ``` -## Environment Variables +If unset, development defaults to `http://localhost:3000/api`. The Docker production build sets `VITE_API_BASE_URL=/api`, so browser requests go through nginx to `atlas-server:3000`. -Create a `.env` file for local development: +## Production nginx -```env -VITE_API_URL=http://localhost:3000 +The frontend image: + +- Builds with `bunx vite build` +- Serves static assets on container port `8080` +- Uses SPA fallback routing through `index.html` +- Proxies `/api/` and exact `/api` to `atlas-server:3000` +- Proxies `/api/events` separately with buffering disabled for SSE +- Allows contract verification uploads up to `50m` +- Serves mounted branding assets from `/branding/` + +Docker Compose exposes the frontend as host port `80:8080`. + +## Routes + +Current app routes: + +| Route | Page | +| --- | --- | +| `/` | Welcome dashboard | +| `/blocks` | Blocks list with live updates and optional DA status | +| `/blocks/:number` | Block details | +| `/blocks/:number/transactions` | Block transactions | +| `/transactions` | Transactions list | +| `/tx/:hash` | Transaction details, logs, token/NFT transfers, decoded input where ABI is available | +| `/addresses` | Address list | +| `/address/:address` | Address details, transactions, tokens, NFTs, transfers, contract tab | +| `/tokens` | ERC-20 token list | +| `/tokens/:address` | Token details, holders, transfers, charts | +| `/nfts` | NFT collection list | +| `/nfts/:contract` | NFT collection tokens and transfers | +| `/nfts/:contract/:tokenId` | NFT token detail, transfer history, full metadata, raw JSON toggle | +| `/status` | Chain status and charts | +| `/faucet` | Faucet page, feature-gated by `/api/config` | +| `/search` | Search results | + +## Runtime Data Flow + +- `BrandingProvider` fetches `/api/config` once at startup and applies chain name, logos, theme colors, feature flags, and faucet metadata. +- The layout uses `/api/events` for live block updates and `/api/height` as a lightweight polling fallback. +- Pages use typed API wrappers in `src/api`. +- NFT views use metadata status fields to distinguish pending, fetched, retryable error, and permanent error states. +- DA UI is shown only when `/api/config.features.da_tracking` is true. + +## Branding Assets + +In Docker, `${BRANDING_DIR:-./branding}` is mounted to `/usr/share/nginx/html/branding:ro`. + +For Vite development, expose the same directory through `frontend/public`: + +```bash +mkdir -p branding +cd frontend/public +ln -s ../../branding branding ``` -## Features +Then set paths such as `CHAIN_LOGO_URL=/branding/logo.svg`. + +## Project Structure -- **Blocks**: Browse and search blocks -- **Transactions**: View transaction details with decoded event logs -- **Addresses**: Address pages with transaction history, token balances, NFTs -- **NFTs**: Browse ERC-721 collections and tokens with metadata -- **Tokens**: Browse ERC-20 tokens with holders and transfers -- **Labels**: Display curated address labels -- **Proxy Detection**: Show proxy contract indicators -- **Search**: Universal search across all entities +```text +frontend/ ++-- src/ +| +-- api/ # Typed fetch clients +| +-- components/ # Shared UI components +| +-- context/ # Theme, branding, live block stats +| +-- hooks/ # Data-fetching and SSE hooks +| +-- pages/ # Route components +| +-- types/ # Shared TypeScript API types +| +-- utils/ # Formatting, ABI decode, metadata helpers ++-- nginx.conf ++-- Dockerfile ++-- package.json ++-- vite.config.ts +``` From 2e4bf82f4b9220bf61637105ae47b6ad9c37e24c Mon Sep 17 00:00:00 2001 From: pthmas <9058370+pthmas@users.noreply.github.com> Date: Wed, 17 Jun 2026 11:08:15 +0200 Subject: [PATCH 2/2] docs: remove PRD.md --- docs/PRD.md | 108 ---------------------------------------------------- 1 file changed, 108 deletions(-) delete mode 100644 docs/PRD.md diff --git a/docs/PRD.md b/docs/PRD.md deleted file mode 100644 index 0e5e8e8..0000000 --- a/docs/PRD.md +++ /dev/null @@ -1,108 +0,0 @@ -# Product and Roadmap - -This document describes the current Atlas product state and the remaining roadmap. It replaces the older MVP checklist, which is now mostly implemented. - -## Product Positioning - -Atlas is a lightweight EVM explorer for one ev-node based chain per deployment. It prioritizes operational simplicity, high-throughput indexing, useful public APIs, and white-label deployment over Blockscout-scale breadth. - -Primary audiences: - -- Chain operators deploying an explorer for an ev-node based chain. -- Developers inspecting blocks, transactions, contracts, ERC-20 tokens, and NFTs. -- Community users browsing chain activity. - -## Implemented Product Areas - -### Core Explorer - -- Blocks, block details, and block transaction lists. -- Transactions, transaction details, logs, ERC-20 transfers, and NFT transfers. -- Address list and address detail pages. -- Universal search for chain entities and token names. -- Status dashboard with chain metadata, indexed height, totals, and charts. -- Live block updates through SSE with polling fallback. - -### Indexing - -- Batch block and receipt fetching from EVM JSON-RPC. -- Configurable start block, batch size, fetch workers, RPC batch size, and RPC rate limit. -- Binary COPY bulk writes for high indexing throughput. -- Resume from persisted indexer state. -- Reindex mode for wiping indexed data and rebuilding. -- Failed-block recording and gap-fill recovery worker. - -### Tokens and NFTs - -- ERC-20 detection, contracts, balances, holders, transfers, and charts. -- NFT collection/token indexing, ownership, transfer history, and token detail pages. -- NFT metadata state machine with retryable and permanent error handling. -- IPFS/Arweave/data/HTTP metadata resolution with safety checks. -- Full NFT metadata display with raw JSON inspection. -- NFT token name search backed by trigram indexing. - -### Contracts - -- Solidity source verification via native API and frontend contract tab. -- solc compiler download/cache support. -- Verified source, ABI, compiler settings, and metadata storage. -- Transaction input decoding against available ABIs. -- Proxy detection and combined ABI exposure. - -### Operations - -- Single `atlas-server` binary for API, indexer, workers, and CLI utilities. -- Docker Compose deployment with `postgres`, `atlas-server`, and `atlas-frontend`. -- Health probes, Prometheus metrics, structured logging option, and request timeouts. -- Scheduled database snapshots and manual dump/restore/reset commands. -- Runtime white-label branding and optional faucet support. -- Optional DA inclusion tracking from ev-node. - -## Current Non-Goals - -- Multi-chain indexing in one deployment. -- Account systems, user preferences, or explorer login. -- ERC-1155 indexing. -- Full EVM trace/internal transaction indexing. -- Gas oracle service or mempool analytics. -- Browser-based contract write UI. -- Hosted image caching/proxying for NFT media. - -## Roadmap Candidates - -These are not committed interfaces; they are candidate future work. - -| Area | Candidate | -| --- | --- | -| ERC-1155 | Index multi-token transfers, balances, metadata, and UI pages | -| Traces | Optional `debug_traceTransaction` or `trace_transaction` ingestion for internal calls | -| Labels | Native address label management API and import/export workflow | -| Contract interaction | Read-only contract calls from verified ABIs; later wallet-backed write flows | -| Search | Attribute search for NFTs and richer ranked text search | -| Media | Optional NFT image proxy/cache for unreliable upstream metadata | -| Operations | More deployment examples for Kubernetes and managed Postgres | -| API compatibility | Additional Etherscan-compatible actions where tools need them | - -## Success Criteria - -Atlas is healthy when: - -- `atlas-server check` passes against the target DB and RPC endpoint. -- `/health/live` returns `200`. -- `/health/ready` returns `200` while the indexer is fresh. -- `/metrics` exposes Prometheus data. -- `/api/height` advances as blocks are indexed. -- The frontend can load `/api/config`, show the configured chain name, and browse blocks. -- Reindexing from `START_BLOCK=0` works on a fresh local chain. -- NFT metadata failures are visible as retryable or permanent states rather than silent missing data. - -## Acceptance Baseline for New Features - -New product features should include: - -- Backend behavior documented in `docs/API.md` or `backend/README.md` when it changes public interfaces. -- Frontend behavior documented in `frontend/README.md` when it changes routes or runtime assumptions. -- Environment variables added to `.env.example` and the backend README. -- Unit tests for new Rust logic in the same file where practical. -- Integration tests for API behavior when the route or query semantics change. -- No use of large-table `OFFSET` pagination for block-scale data.