Skip to content

Latest commit

 

History

History
72 lines (62 loc) · 3.92 KB

README.md

File metadata and controls

72 lines (62 loc) · 3.92 KB

Quant IPO RL Agent: Portfolio Allocation for Simulated IPOs

Project Overview

This repo provides a reproducible, quant-focused RL agent to allocate fixed capital across multiple simulated IPO opportunities. It features a realistic IPO + market simulator and a Gym environment compatible with Stable-Baselines3 (PPO), supporting both immediate expected and delayed realized reward modes. The codebase is fully self-contained, requires no external data, and enables vectorized training/backtesting for robust financial RL experimentation.

Features

  • Simulated quant IPO + market environment (no external data needed)
  • Reproducible randomness with explicit seeding (numpy.random.RandomState)
  • Two reward modes: immediate expected vs. delayed realized
  • Gym v0.26.2, Stable-Baselines3, VecNormalize-compatible
  • Utilities: Sharpe ratio, annualized return, max drawdown calculation
  • Training and backtesting scripts with CLI and plotting
  • Unit tests for critical functionality (simulator & env)

Reward Modes Explained

  • Immediate Expected Reward (default): Agent gets expected profit reward instantly based on allocation, expected listing gain, and risk penalty. Allows smoother gradients and fast credit assignment.
  • Delayed Realized Reward (--delayed_reward True): Rewards are only granted after IPOs actually list. Allocated capital is reserved, and realized profit/loss is returned when IPOs list. Rewards per step are mostly zero until listing occurs (except time-penalty), better modeling real-world delay and risk.
  • Toggling: Use the --delayed_reward CLI flag in training/backtesting scripts.

Project Structure

  • README.md: This file. Project details, install, usage, reward modes.
  • requirements.txt: Python dependencies (Python 3.9+).
  • data_simulator_quant.py: Quantitative IPO data simulator. Provides IPODataSimulator class.
  • env_quant.py: Gym environment for RL allocation (QuantIPOEnv).
  • train_ppo_quant.py: Train RL agent via PPO (Stable-Baselines3) with config options.
  • backtest_quant.py: Backtests saved models, computes and plots metrics.
  • utils_metrics.py: Helpers for Sharpe ratio, drawdown, returns.
  • tests/test_simulator_env.py: Unit tests (pytest compatible).
  • run.sh: Convenience script for full repo demo.
  • .gitignore: Ignore venv, logs, pyc, and typical build files.

Installation & Quick Start

# Clone repo and enter project directory
cd "RL agent"
# (Recommended) create venv
python -m venv venv && source venv/bin/activate
# Install Python dependencies
pip install -r requirements.txt

Training Example

python train_ppo_quant.py --timesteps 200000 --M 8 --capital 1000000 --seed 0 --save-dir logs/ppo_quant

Backtest Example

python backtest_quant.py --model logs/ppo_quant/ppo_quant.zip --episodes 50

Run All: install, train short model, backtest

bash run.sh

File List & Brief Descriptions

File Description
README.md Overview, usage, reward mode tradeoff
requirements.txt Package dependencies
data_simulator_quant.py Quant IPO/market simulator (IPODataSimulator)
env_quant.py Gym RL environment (QuantIPOEnv)
train_ppo_quant.py Script to train RL agent
backtest_quant.py Analyze trained agent, metrics & plot
utils_metrics.py Financial metrics utilities
tests/test_simulator_env.py Unit tests for critical components
run.sh Single-command install, train, backtest demo
.gitignore Ignore venv, logs, cache, build artifacts, pyc

License

MIT License (see LICENSE file if present)