Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions docs/70_WALKTHROUGH.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ mkdir -p ~/.qlib/qlib_data/cn_data
# 使用 wget 下载最新版本数据
wget https://github.com/chenditc/investment_data/releases/latest/download/qlib_bin.tar.gz
# 解压到数据目录
tar -xzf qlib_bin.tar.gz -C ~/.qlib/qlib_data/cn_data
tar -zxvf qlib_bin.tar.gz -C ~/.qlib/qlib_data/cn_data --strip-components=1
```

> 详见 https://github.com/chenditc/investment_data 了解数据更新频率和内容说明。
Expand All @@ -106,7 +106,12 @@ tar -xzf qlib_bin.tar.gz -C ~/.qlib/qlib_data/cn_data
export QLIB_DATA_DIR="/your/custom/path/cn_data"
export QLIB_REGION="cn"
```
如果使用Windows系统的PowerShell,可在工作区的`run_env.ps1`中配置:

```bash
$env:QLIB_DATA_DIR = "X:/your/custom/path/cn_data"
$env:QLIB_REGION = "cn"
```
---

## 3. 初始化工作区
Expand All @@ -123,10 +128,18 @@ QuantPits 采用 **Workspace(工作区)** 机制实现配置与数据的完
# cd QuantPits

source workspaces/Demo_Workspace/run_env.sh

# For Windows
. ./workspaces/Demo_Workspace/run_env.ps1
```

输出 `Workspace activated: .../Demo_Workspace` 表示激活成功。

**特别提醒:**在 Windows 上,如果你要让 PowerShell 执行 .ps1 脚本,如果这是你第一次运行脚本,PowerShell 可能会报“禁止执行脚本”的错误。此时你需要以**管理员身份**运行 PowerShell,并输入:
```bash
Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
```

### 3.2 创建新工作区

```bash
Expand All @@ -144,6 +157,9 @@ python quantpits/scripts/init_workspace.py \

```bash
source workspaces/MyWorkspace/run_env.sh

# For Windows
. ./workspaces/Demo_Workspace/run_env.ps1
```

### 3.4 工作区目录结构
Expand All @@ -162,7 +178,8 @@ workspaces/MyWorkspace/
├── output/ # 输出文件
├── archive/ # 历史归档
├── mlruns/ # MLflow 追踪
└── run_env.sh # 环境激活脚本
├── run_env.ps1 # 环境激活脚本(Win)
└── run_env.sh # 环境激活脚本(Linux)
```

---
Expand Down
34 changes: 29 additions & 5 deletions docs/en/70_WALKTHROUGH.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ mkdir -p ~/.qlib/qlib_data/cn_data
# Download the latest version using wget
wget https://github.com/chenditc/investment_data/releases/latest/download/qlib_bin.tar.gz
# Extract to the data directory
tar -xzf qlib_bin.tar.gz -C ~/.qlib/qlib_data/cn_data
tar -zxvf qlib_bin.tar.gz -C ~/.qlib/qlib_data/cn_data --strip-components=1
```

> See https://github.com/chenditc/investment_data for update frequency and data details.
Expand All @@ -106,6 +106,12 @@ If your data is stored elsewhere, configure it in your workspace's `run_env.sh`:
export QLIB_DATA_DIR="/your/custom/path/cn_data"
export QLIB_REGION="cn"
```
If using Windows PowerShell, configure it in your workspace's `run_env.ps1`:

```powershell
$env:QLIB_DATA_DIR = "X:/your/custom/path/cn_data"
$env:QLIB_REGION = "cn"
```

---

Expand All @@ -123,10 +129,18 @@ QuantPits uses a **Workspace** mechanism to fully isolate configurations and dat
# cd QuantPits

source workspaces/Demo_Workspace/run_env.sh

# For Windows
. ./workspaces/Demo_Workspace/run_env.ps1
```

Output: `Workspace activated: .../Demo_Workspace` confirms successful activation.

**Note for Windows:** If this is your first time running a `.ps1` script, PowerShell may block execution. You need to run PowerShell as **Administrator** and execute:
```powershell
Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
```

### 3.2 Create a New Workspace

```bash
Expand All @@ -138,12 +152,15 @@ python quantpits/scripts/init_workspace.py \
This will:
- Clone all configuration files from `Demo_Workspace/config/`
- Create empty `data/`, `output/`, `archive/`, and `mlruns/` directories
- Generate a `run_env.sh` environment activation script
- Generate a `run_env.sh` (Linux) and `run_env.ps1` (Windows) environment activation script

### 3.3 Activate the New Workspace

```bash
source workspaces/MyWorkspace/run_env.sh

# For Windows
. ./workspaces/MyWorkspace/run_env.ps1
```

### 3.4 Workspace Directory Structure
Expand All @@ -162,7 +179,8 @@ workspaces/MyWorkspace/
├── output/ # Output files
├── archive/ # Historical archives
├── mlruns/ # MLflow tracking
└── run_env.sh # Environment activation script
├── run_env.ps1 # Environment activation script (Win)
└── run_env.sh # Environment activation script (Linux)
```

---
Expand Down Expand Up @@ -403,7 +421,7 @@ The Post-Trade script processes settlement files exported from your broker, upda

1. Export settlement records (`.xlsx`) from your broker and place them in `data/`
2. File naming convention: `YYYY-MM-DD-table.xlsx` (e.g., `2026-02-24-table.xlsx`)
3. For non-trading days, use the empty template `emp-table.xlsx` (all empty cells)
3. For non-trading days, use the empty template `emp-table.xlsx` (an empty Excel file)

### 9.2 Configure Cashflow (If Applicable)

Expand Down Expand Up @@ -520,6 +538,9 @@ python quantpits/scripts/run_rolling_health_report.py

source workspaces/Demo_Workspace/run_env.sh

# For Windows
. ./workspaces/Demo_Workspace/run_env.ps1

# ① Full training
python quantpits/scripts/prod_train_predict.py

Expand All @@ -536,10 +557,13 @@ python quantpits/scripts/ensemble_fusion.py --from-config-all
cd QuantPits
source workspaces/MyWorkspace/run_env.sh

# For Windows
. ./workspaces/MyWorkspace/run_env.ps1

# ① Predict
python quantpits/scripts/prod_predict_only.py --all-enabled
```

# ② Fuse
python quantpits/scripts/ensemble_fusion.py --from-config-all

# ③ Post-Trade (if live trading)
Expand Down
62 changes: 48 additions & 14 deletions quantpits/scripts/init_workspace.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"""
import argparse
import os
import platform
import shutil
import stat

Expand Down Expand Up @@ -63,24 +64,57 @@ def init_workspace(source, target):
dir_path = os.path.join(target, d)
print(f"Creating empty directory: {dir_path}")
os.makedirs(dir_path)

# 3. Create activation script
is_windows = platform.system() == "Windows"
script_ext = ".ps1" if is_windows else ".sh"
script_name = "run_env" + script_ext
run_env_path = os.path.join(target, script_name)

with open(run_env_path, "w", encoding="utf-8") as f:
if is_windows:
f.write("# run this file to activate the workspace\n\n")
f.write('$env:QLIB_WORKSPACE_DIR = Split-Path -Parent $MyInvocation.MyCommand.Path\n')
f.write("# Uncomment and modify to use a custom Qlib data directory:\n")
f.write('# $env:QLIB_DATA_DIR = "D:\\data\\cn_data"\n')
f.write('# $env:QLIB_REGION = "cn"\n\n')
f.write('Write-Host "Workspace activated: $env:QLIB_WORKSPACE_DIR"\n')
else:
f.write("#!/bin/bash\n")
f.write("# Source this file to activate the workspace\n")
f.write('export QLIB_WORKSPACE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"\n')
f.write("# Uncomment and modify to use a custom Qlib data directory:\n")
f.write('# export QLIB_DATA_DIR="~/.qlib/qlib_data/cn_data"\n')
f.write('# export QLIB_REGION="cn"\n')
f.write('echo "Workspace activated: $QLIB_WORKSPACE_DIR"\n')

# Make it executable for Linux/macOS
if not is_windows:
os.chmod(run_env_path, os.stat(run_env_path).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)

print("\nWorkspace initialization complete!")
if is_windows:
print(f"To use this workspace, run: .\\{script_name}")
print(f"Or in PowerShell: $env:QLIB_WORKSPACE_DIR='{target}'; python ...")
else:
print(f"To use this workspace, run: source {run_env_path}")
print(f"Or prepend commands with: QLIB_WORKSPACE_DIR={target} python ...")
# 3. Create run_env.sh
run_env_path = os.path.join(target, "run_env.sh")
with open(run_env_path, "w") as f:
f.write("#!/bin/bash\n")
f.write("# Source this file to activate the workspace\n")
f.write('export QLIB_WORKSPACE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"\n')
f.write("# Uncomment and modify to use a custom Qlib data directory:\n")
f.write('# export QLIB_DATA_DIR="~/.qlib/qlib_data/cn_data"\n')
f.write('# export QLIB_REGION="cn"\n')
f.write("echo \"Workspace activated: $QLIB_WORKSPACE_DIR\"\n")
# run_env_path = os.path.join(target, "run_env.sh")
# with open(run_env_path, "w") as f:
# f.write("#!/bin/bash\n")
# f.write("# Source this file to activate the workspace\n")
# f.write('export QLIB_WORKSPACE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"\n')
# f.write("# Uncomment and modify to use a custom Qlib data directory:\n")
# f.write('# export QLIB_DATA_DIR="~/.qlib/qlib_data/cn_data"\n')
# f.write('# export QLIB_REGION="cn"\n')
# f.write("echo \"Workspace activated: $QLIB_WORKSPACE_DIR\"\n")

# Make it executable just in case, though it should be sourced
os.chmod(run_env_path, os.stat(run_env_path).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
# os.chmod(run_env_path, os.stat(run_env_path).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)

print("\\nWorkspace initialization complete!")
print(f"To use this workspace, run: source {run_env_path}")
print(f"Or prepend commands with: QLIB_WORKSPACE_DIR={target} python ...")
# print("\\nWorkspace initialization complete!")
# print(f"To use this workspace, run: source {run_env_path}")
# print(f"Or prepend commands with: QLIB_WORKSPACE_DIR={target} python ...")

def main():
parser = argparse.ArgumentParser(description="Initialize a new Qlib Workspace")
Expand Down
52 changes: 48 additions & 4 deletions tests/quantpits/scripts/test_init_workspace.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ def test_init_workspace_success(tmp_path):

target = tmp_path / "NewWorkspace"

init_workspace(str(source), str(target))
# Mock Linux behavior to ensure run_env.sh is tested here
with patch("platform.system", return_value="Linux"):
init_workspace(str(source), str(target))

# Verify directory structure
assert target.exists()
Expand All @@ -37,8 +39,9 @@ def test_init_workspace_success(tmp_path):
assert run_env.exists()
content = run_env.read_text()
assert "QLIB_WORKSPACE_DIR" in content
# Should have executable permission
assert os.stat(run_env).st_mode & stat.S_IXUSR
# Should have executable permission (only check on non-windows)
if os.name != 'nt':
assert os.stat(run_env).st_mode & stat.S_IXUSR


def test_init_workspace_strategy_yaml_generated(tmp_path):
Expand Down Expand Up @@ -97,8 +100,49 @@ def test_init_workspace_no_source_config(tmp_path, capsys):
assert "Warning" in captured.out
assert (target / "config").exists()


def test_init_workspace_windows_script(tmp_path):
"""Should generate run_env.ps1 when running on Windows."""
source = tmp_path / "Source"
source.mkdir()
(source / "config").mkdir()
target = tmp_path / "NewWorkspacePS"

with patch("platform.system", return_value="Windows"):
init_workspace(str(source), str(target))

run_env = target / "run_env.ps1"
assert run_env.exists()
content = run_env.read_text(encoding="utf-8")
assert "$env:QLIB_WORKSPACE_DIR" in content
assert "run_env.sh" not in [f.name for f in target.iterdir() if f.is_file() and f.suffix == ".sh"]


def test_init_workspace_linux_script(tmp_path):
"""Should generate run_env.sh when running on Linux."""
source = tmp_path / "Source"
source.mkdir()
(source / "config").mkdir()
target = tmp_path / "NewWorkspaceSH"

with patch("platform.system", return_value="Linux"):
init_workspace(str(source), str(target))

run_env = target / "run_env.sh"
assert run_env.exists()
content = run_env.read_text()
assert "export QLIB_WORKSPACE_DIR" in content
# Should have executable permission (only check on non-windows)
if os.name != 'nt':
assert os.stat(run_env).st_mode & stat.S_IXUSR


def test_main(tmp_path, monkeypatch):
from quantpits.scripts import init_workspace as iw
import platform
is_windows = platform.system() == "Windows"
script_name = "run_env.ps1" if is_windows else "run_env.sh"

source = tmp_path / "Source"
source.mkdir()
(source / "config").mkdir()
Expand All @@ -109,4 +153,4 @@ def test_main(tmp_path, monkeypatch):
iw.main()

assert target.exists()
assert (target / "run_env.sh").exists()
assert (target / script_name).exists()
8 changes: 8 additions & 0 deletions workspaces/Demo_Workspace/run_env.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# run this file to activate the workspace

$env:QLIB_WORKSPACE_DIR = Split-Path -Parent $MyInvocation.MyCommand.Path
# Uncomment and modify to use a custom Qlib data directory:
$env:QLIB_DATA_DIR = "D:\data\cn_data"
$env:QLIB_REGION = "cn"

Write-Host "Workspace activated: $env:QLIB_WORKSPACE_DIR"