ERC20 Token Balances Service

A NestJS service for indexing and tracking ERC20 token balances across multiple blockchains in real-time.

📋 Table of Contents

Overview
System Architecture
Features
Tech Stack
Getting Started
API Documentation
Monitoring Multiple Tokens
Job Types
Developer Guide
Testing
Deployment

🎯 Overview

The ERC20 Token Balances Service is a backend service designed to index and track ERC20 token transfer events across multiple blockchain networks. It provides real-time balance tracking, historical data indexing, and scheduled recurring indexing with built-in caching and rate limiting for RPC calls.

Key Capabilities

Multi-Chain Support: Index tokens from any EVM-compatible blockchain
Historical Data: Backfill historical transfer events from any block range
Scheduled Indexing: Automatically sync new blocks at configurable intervals
Caching: In-memory caching + PostgreSQL for optimal read/write performance
Rate Limiting: Built-in protection against RPC provider rate limits
Scalable: Queue-based architecture with BullMQ for horizontal scaling

Usage instructions

See Track ERC20 balances for instructions to start tracking ERC20 token balances specifically one the service is setup. Same steps can be applied to any other token.

🏗️ System Architecture

High-Level Architecture

graph TB
    subgraph "Client Layer"
        A[Admin Client] -->|JWT Auth| B[Admin API]
        C[User Client] -->|Public Access| D[Token API]
    end
    
    subgraph "Application Layer"
        B --> E[NestJS Application]
        D --> E
        E --> F[Token Service]
        E --> G[Indexer Manager]
    end
    
    subgraph "Job Processing Layer"
        G -->|Enqueue Jobs| H[BullMQ Queue]
        H --> I[Indexer Worker ]
    end
    
    subgraph "Data Layer"
        I --> L[(PostgreSQL)]
        F --> M[( Cache)]
        I --> M
    end
    
    subgraph "Blockchain Layer"
        I -->|RPC Calls| N[Ethereum Node]
        I -->|RPC Calls| O[Polygon Node]
        I -->|RPC Calls| P[Other EVM Chains]
    end
    
    L -->|Persist| Q[Events Table]
    L -->|Persist| R[Token Balances Table]
    L -->|Persist| S[Indexer Jobs Table]
    L -->|Persist| T[Tokens Table]

Data Flow

sequenceDiagram
    participant Admin
    participant API
    participant Queue
    participant Worker
    participant RPC
    participant DB
    participant Cache
    
    Admin->>API: POST /admin/indexer/schedule
    API->>Queue: Enqueue Scheduled Job
    Queue->>Worker: Process Job
    
    loop Every Interval
        Worker->>DB: Check Last Processed Block
        Worker->>RPC: Get Latest Block Number
        Worker->>RPC: Get Logs (chunked)
        RPC-->>Worker: Transfer Events
        Worker->>DB: Save Events
        Worker->>Cache: Update Balances
        Worker->>DB: Update Balances
        Worker->>DB: Update Job Progress
    end
    
    Worker-->>Queue: Job Complete

Worker Architecture

graph LR
    A[BullMQ Queue] --> B{Job Type}
    B -->|Historical| C[Historical Indexing Worker]
    B -->|Scheduled| D[Scheduled Indexing Worker]
    
    C --> E[Chunk Block Range]
    D --> F[Get Latest Block]
    
    E --> G[Fetch Logs via RPC]
    F --> G
    
    G --> H[Parse Events]
    H --> I[Save to PostgreSQL]
    H --> J[Update Cache]
    I --> K[Update Job Status]
    J --> K

Features

Indexing Features

Multi-chain Multi-token support: Admin can add multiple token addresses to track balances of
Progress Tracking: Real-time job progress updates
Resume Capability: Scheduled jobs resume from last processed block
Logs: Adequate logging for viewing progress and status

Data Management

Event Storage: All transfer events stored in PostgreSQL
Balance Tracking: Balance calculations with caching
Token Registry: Track multiple tokens across multiple chains

Tech Stack

Core Framework

NestJS - Progressive Node.js framework
TypeScript - Type-safe development
Node.js - Runtime environment

Queue & Workers

BullMQ - Redis-based queue for job processing

Database

PostgreSQL - Primary data store for events, balances, and jobs
TypeORM - Database ORM with TypeScript support

Blockchain Interaction

Viem - TypeScript library for Ethereum interactions

Authentication & Security

JWT - JSON Web Tokens for admin authentication
Role-Based Access Control - Admin vs. public endpoints

Getting Started

Prerequisites

Node.js v22+
PostgreSQL v14+
Redis v7+
Docker & Docker Compose (optional, for easy setup)

Installation

Clone the repository

git clone https://github.com/yourusername/erc20-token-balances-service.git
cd erc20-token-balances-service

Install dependencies
```
npm install
```
Set up environment variables

Create a .env file in the root directory. Refer .sample.env file.
Start infrastructure with Docker (recommended)
```
docker-compose up postgres redis
```
Or manually start PostgreSQL and Redis.
Generate admin authentication token
```
npm run generate:admin-token 86400
```
This generates a JWT token valid for 86400 seconds (24 hours). Save this token for making admin API calls.

Start the application

# Development mode with hot-reload
npm run start:dev

# Production mode
npm run build
npm run start:prod

📚 API Documentation

Swagger API

http://localhost:3000/api/v1

Swagger Documentation

Once the application is running, access interactive API documentation at:

http://localhost:3000/api

Admin APIs

All admin endpoints require JWT authentication. Include the token in the Authorization header:

Authorization: Bearer <your-jwt-token>

Token Management

Method	Endpoint	Description	Request Body	Response
`POST`	`/admin/token`	Add a new token to track	`{ chainId: number, tokenAddress: string }`	Token object
`GET`	`/admin/tokens`	Get all tracked tokens	-	Array of tokens
`DELETE`	`/admin/token`	Remove a tracked token	`{ chainId: number, tokenAddress: string }`	Success message

Example: Add Token

curl -X POST http://localhost:3000/api/v1/admin/token \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "chainId": 1,
    "tokenAddress": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"
  }'

Indexer Management

Method	Endpoint	Description	Request Body	Response
`POST`	`/admin/indexer`	Start one-time historical indexing	`{ chainId, tokenAddress, rpcUrl, fromBlock, toBlock, maxBlockRange?, minIntervalBetweenRequestsMs? }`	Success message
`POST`	`/admin/indexer/schedule`	Schedule recurring indexing	`{ chainId, tokenAddress, rpcUrl, fromBlock, repeatEveryMs, maxBlockRangePerRequest?, minIntervalBetweenRequestsMs? }`	Success message
`GET`	`/admin/indexer`	Get all indexing jobs	-	Array of jobs

Example: Start Historical Indexing

curl -X POST http://localhost:3000/api/v1/admin/indexer \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "chainId": 1,
    "tokenAddress": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "rpcUrl": "https://eth-mainnet.g.alchemy.com/v2/YOUR_KEY",
    "fromBlock": 10000000,
    "toBlock": 10001000,
    "maxBlockRange": 100,
    "minIntervalBetweenRequestsMs": 1000
  }'

Example: Schedule Recurring Indexing

curl -X POST http://localhost:3000/api/v1/admin/indexer/schedule \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "chainId": 1,
    "tokenAddress": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "rpcUrl": "https://eth-mainnet.g.alchemy.com/v2/YOUR_KEY",
    "fromBlock": 10000000,
    "repeatEveryMs": 60000,
    "maxBlockRangePerRequest": 100,
    "minIntervalBetweenRequestsMs": 1000
  }'

User APIs

Public endpoints - no authentication required.

Get User Token Balance

Retrieve the ERC20 token balance for a specific user address.

Endpoint: GET /api/v1/token/balance

Query Parameters:

Parameter	Type	Required	Description	Example
`chainId`	`number`	Yes	The blockchain network ID	`1` (Ethereum Mainnet)
`tokenAddress`	`string`	Yes	The ERC20 token contract address	`0xdAC17F958D2ee523a2206206994597C13D831ec7`
`userAddress`	`string`	Yes	The user wallet address to check balance for	`0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb`

Example Request:

curl "http://localhost:3000/api/v1/token/balance?chainId=1&tokenAddress=0xdAC17F958D2ee523a2206206994597C13D831ec7&userAddress=0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb"

Success Response (200 OK):

{
  "balance": "1500000000000000000000",
  "blockNumber": "18571358"
}

Response Fields:

Field	Type	Description
`balance`	`string`	Token balance in smallest unit (wei for most tokens). Use appropriate decimals for display.
`blockNumber`	`string \| null`	The block number at which this balance was last updated. `null` if never indexed.

Error Response (400 Bad Request):

{
  "statusCode": 400,
  "message": "Invalid address format",
  "error": "Bad Request"
}

Notes:

Balance is returned as a string to preserve precision for large numbers
If the user has never held this token, balance will be "0"
The blockNumber indicates when this balance was last updated from blockchain events

Get All Balances for a Token

Retrieve all user balances for a specific token with pagination support.

Endpoint: GET /api/v1/token/balances

Query Parameters:

Parameter	Type	Required	Description	Default	Example
`chainId`	`number`	Yes	The blockchain network ID	-	`1`
`tokenAddress`	`string`	Yes	The ERC20 token contract address	-	`0xdAC17F958D2ee523a2206206994597C13D831ec7`
`limit`	`number`	No	Maximum number of results per page (1-100)	`20`	`50`
`skip`	`number`	No	Number of records to skip for pagination	`0`	`100`

Example Request:

curl "http://localhost:3000/api/v1/token/balances?chainId=1&tokenAddress=0xdAC17F958D2ee523a2206206994597C13D831ec7&limit=50&skip=0"

Success Response (200 OK):

{
  "data": [
    {
      "userAddress": "0x0123456789abcdef0123456789abcdef01234567",
      "balance": "2500000000000000000000",
      "blockNumber": "18571358"
    },
    {
      "userAddress": "0x1234567890abcdef1234567890abcdef12345678",
      "balance": "1500000000000000000000",
      "blockNumber": "18571400"
    }
  ],
  "total": 2,
  "limit": 50,
  "skip": 0
}

Response Fields:

Field	Type	Description
`data`	`array`	Array of balance records, sorted by user address (ascending)
`data[].userAddress`	`string`	The user's wallet address
`data[].balance`	`string`	Token balance in smallest unit
`data[].blockNumber`	`string \| null`	Block number of last balance update
`total`	`number`	Number of records returned in this response
`limit`	`number`	The limit used for this query
`skip`	`number`	The skip offset used for this query

Pagination Example:

# Get first 20 results
curl "http://localhost:3000/api/v1/token/balances?chainId=1&tokenAddress=0x4c9...&limit=20&skip=0"

# Get next 20 results
curl "http://localhost:3000/api/v1/token/balances?chainId=1&tokenAddress=0x4c9...&limit=20&skip=20"

# Get results 100-150
curl "http://localhost:3000/api/v1/token/balances?chainId=1&tokenAddress=0x4c9...&limit=50&skip=100"

Error Response (400 Bad Request):

{
  "statusCode": 400,
  "message": ["limit must not be greater than 100"],
  "error": "Bad Request"
}

Notes:

Results are always sorted by userAddress in ascending order for consistent pagination
Maximum limit is 100 to prevent excessive database load
Use skip to paginate through large result sets
Only returns addresses with non-zero balances
Balance and blockNumber have the same meaning as in the single balance endpoint

🎯 Monitoring Multiple Tokens

The service is designed to efficiently monitor multiple tokens across multiple chains simultaneously.

Setup Process

Add Tokens to Track

# Add USDC on Ethereum
curl -X POST http://localhost:3000/api/v1/admin/token \
  -H "Authorization: Bearer TOKEN" \
  -d '{"chainId": 1, "tokenAddress": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"}'

# Add USDC on Polygon
curl -X POST http://localhost:3000/api/v1/admin/token \
  -H "Authorization: Bearer TOKEN" \
  -d '{"chainId": 137, "tokenAddress": "0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174"}'

# Add DAI on Ethereum
curl -X POST http://localhost:3000/api/v1/admin/token \
  -H "Authorization: Bearer TOKEN" \
  -d '{"chainId": 1, "tokenAddress": "0x6B175474E89094C44Da98b954EedeAC495271d0F"}'

Schedule Recurring Indexing for Each Token

# Schedule USDC Ethereum - check every 30 seconds
curl -X POST http://localhost:3000/api/v1/admin/indexer/schedule \
  -H "Authorization: Bearer TOKEN" \
  -d '{
    "chainId": 1,
    "tokenAddress": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "rpcUrl": "https://eth-mainnet.g.alchemy.com/v2/KEY",
    "fromBlock": 18000000,
    "repeatEveryMs": 30000
  }'

# Schedule USDC Polygon - check every 15 seconds (faster block time)
curl -X POST http://localhost:3000/api/v1/admin/indexer/schedule \
  -H "Authorization: Bearer TOKEN" \
  -d '{
    "chainId": 137,
    "tokenAddress": "0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174",
    "rpcUrl": "https://polygon-mainnet.g.alchemy.com/v2/KEY",
    "fromBlock": 45000000,
    "repeatEveryMs": 15000
  }'

Monitor Job Status

curl http://localhost:3000/api/v1/admin/indexer \
  -H "Authorization: Bearer TOKEN"

Best Practices

Different Chains: Use separate RPC URLs for each chain
Rate Limiting: Adjust minIntervalBetweenRequestsMs based on RPC provider limits
Block Range: Use smaller maxBlockRangePerRequest for chains with high transaction volume
Scheduling: Set repeatEveryMs based on average block time of the chain

Job Types

1. Historical Indexing (One-Time)

Purpose: Backfill historical transfer events for a specific block range.

Use Cases:

Initial data population
Fill gaps in historical data

Characteristics:

Runs once and completes
Processes specified block range (fromBlock → toBlock)
Does not repeat automatically

Database Table: indexer_jobs

2. Scheduled Indexing (Recurring)

Purpose: Continuously sync new blocks as they are produced.

Use Cases:

Real-time balance tracking
Ongoing monitoring of token transfers
Live dashboard updates

Characteristics:

Runs repeatedly at configured intervals
Automatically resumes from last processed block
Persists progress in database

Database Table: indexer_jobs

Developer Guide

Project Structure

src/
├── app.module.ts                    # Main application module
├── main.ts                          # Application entry point
├── common/                          # Shared utilities
│   ├── decorators/                  # Custom decorators
│   ├── filters/                     # Exception filters
│   ├── guards/                      # Auth guards
│   └── validators/                  # Custom validators
├── config/                          # Configuration
│   └── configuration.ts             # App configuration
├── datasources/                     # Database setup
│   └── db/
│       ├── database.module.ts
│       └── database.provider.ts
├── middleware/                      # HTTP middleware
│   └── http-logging.middleware.ts
├── routes/                          # API Controllers
│   ├── admin/                       # Admin endpoints
│   │   ├── admin.controller.ts
│   │   └── entities/                # DTOs
│   └── token/                       # User endpoints
│       ├── token.controller.ts
│       └── entities/                # DTOs
├── services/                        # Business logic
│   ├── cache/                       # Caching service
│   │   ├── cache.module.ts
│   │   └── token-balance.cache.ts
│   ├── events/                      # Event management
│   │   ├── event.module.ts
│   │   ├── event.service.ts
│   │   └── repository/
│   ├── indexer-manager/             # Job queue management
│   │   ├── indexer-manager.module.ts
│   │   ├── indexer-manager.service.ts
│   │   └── respository/
│   ├── indexer-worker/              # Job processing
│   │   ├── indexer-worker.module.ts
│   │   ├── indexer-worker.processor.ts
│   │   └── respository/
│   ├── logging/                     # Logging service
│   ├── token/                       # Token registry
│   └── token-balance/               # Balance service
│       ├── token-balance.module.ts
│       ├── token.service.ts
│       └── repository/
├── scripts/                         # Utility scripts
│   └── generate-admin-token.ts      # JWT token generator
└── utils/                           # Helper functions
    └── constants.ts

Database Schema

Tables

tokens - Registry of tracked tokens

CREATE TABLE tokens (
  id UUID PRIMARY KEY,
  chain_id INTEGER NOT NULL,
  token_address VARCHAR(42) NOT NULL,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(chain_id, token_address)
);

events - Transfer events

CREATE TABLE events (
  id SERIAL PRIMARY KEY,
  event_type VARCHAR(50) NOT NULL,
  chain_id INTEGER NOT NULL,
  token_address VARCHAR(42) NOT NULL,
  from_address VARCHAR(42) NOT NULL,
  to_address VARCHAR(42) NOT NULL,
  amount VARCHAR(78) NOT NULL,
  block_number BIGINT NOT NULL,
  block_hash VARCHAR(66) NOT NULL,
  tx_hash VARCHAR(66) NOT NULL,
  log_index INTEGER NOT NULL,
  confirmed BOOLEAN DEFAULT FALSE,
  created_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(tx_hash, block_hash, block_number, log_index)
);

token_balances - Current balances

CREATE TABLE token_balances (
  id UUID PRIMARY KEY,
  token_id UUID REFERENCES tokens(id),
  user_address VARCHAR(42) NOT NULL,
  balance VARCHAR(78) NOT NULL,
  updated_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(token_id, user_address)
);

indexer_jobs - Job tracking

CREATE TABLE indexer_jobs (
  id UUID PRIMARY KEY,
  job_id VARCHAR(255) UNIQUE NOT NULL,
  status VARCHAR(50) NOT NULL,
  from_block BIGINT NOT NULL,
  last_block BIGINT,
  last_block_hash VARCHAR(66),
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

Key Components

1. Indexer Manager Service

Manages job queue
Enqueues historical and scheduled jobs
Provides job status and control

2. Indexer Worker Processor

Processes jobs from queue
Fetches logs from RPC
Saves events to database
Updates cache
Tracks job progress

3. Token Balance Cache

Multi-tier caching (memory + Redis)
Balance add/remove operations
Automatic cache invalidation

4. Event Service

Bulk event insertion
Duplicate handling
Event querying

Adding a New Chain

Ensure the chain is supported by Viem (or add custom chain config)
No code changes needed - just use the chain ID in API calls
Configure appropriate RPC URL for the chain

Environment Variables

Variable	Description	Default	Required
`NODE_ENV`	Environment (development/production)	`development`	No
`PORT`	Application port	`3000`	No
`DATABASE_URL`	PostgreSQL connection string	-	Yes
`INDEXER_CONNECTION_HOST`	Redis host	`localhost`	Yes
`INDEXER_CONNECTION_PORT`	Redis port	`6379`	Yes
`JWT_SECRET`	Secret for JWT signing	-	Yes

Generate Admin Token

The service uses JWT tokens for admin authentication. Generate a token using:

npm run generate:admin-token <expiresInSeconds>

Example:

# Generate token valid for 24 hours (86400 seconds)
npm run generate:admin-token 86400

# Output:
# Generated Admin JWT Token:
# eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Store this token securely and use it in the Authorization header for admin API calls.

Testing

Run Tests

# Unit tests
npm run test

# Watch mode
npm run test:watch

# Test coverage
npm run test:cov

# E2E tests
npm run test:e2e

Test Structure

src/
└── **/__tests__/          # Tests co-located with source
test/
└── *.e2e-spec.ts          # End-to-end tests

Monitoring & Observability

Logs

Structured logging with Winston:

All logs in JSON format
Log levels: error, warn, info, debug
Job progress tracking
RPC call monitoring

License

This project is licensed under the MIT License.

Project setup

$ npm install

Compile and run the project

# development
$ npm run start

# watch mode
$ npm run start:dev

Run tests

# unit tests
$ npm run test

License

Nest is MIT licensed.

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.github/workflows		.github/workflows
docs		docs
scripts		scripts
src		src
test		test
.gitignore		.gitignore
.prettierrc		.prettierrc
.sample.env		.sample.env
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
eslint.config.mjs		eslint.config.mjs
nest-cli.json		nest-cli.json
package-lock.json		package-lock.json
package.json		package.json
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json

License

akshay-ap/erc20-token-balances-service

Folders and files

Latest commit

History

Repository files navigation

ERC20 Token Balances Service

📋 Table of Contents

🎯 Overview

Key Capabilities

Usage instructions

🏗️ System Architecture

High-Level Architecture

Data Flow

Worker Architecture

Features

Indexing Features

Data Management

Tech Stack

Core Framework

Queue & Workers

Database

Blockchain Interaction

Authentication & Security

Getting Started

Prerequisites

Installation

📚 API Documentation

Swagger API

Swagger Documentation

Admin APIs

Token Management

Indexer Management

User APIs

Get User Token Balance

Get All Balances for a Token

🎯 Monitoring Multiple Tokens

Setup Process

Best Practices

Job Types

1. Historical Indexing (One-Time)

2. Scheduled Indexing (Recurring)

Developer Guide

Project Structure

Database Schema

Tables

Key Components

1. Indexer Manager Service

2. Indexer Worker Processor

3. Token Balance Cache

4. Event Service

Adding a New Chain

Environment Variables

Generate Admin Token

Testing

Run Tests

Test Structure

Monitoring & Observability

Logs

License

Project setup

Compile and run the project

Run tests

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages