Refactor configuration management with unified loader and update all related scripts, tests, and documentation.

Author: Tonny@Home

docs/00_SYSTEM_OVERVIEW.md

@@ -345,9 +345,9 @@ python quantpits/scripts/ensemble_fusion.py \
 | 文件 | 用途 | 相关模块 |
 |------|------|----------|
 | `model_registry.yaml` | 模型注册表：定义模型名、算法、数据集、标签 | 训练、预测 |
-| `model_config.json` | 日期参数：训练窗口、滑动/固定模式 | 训练、预测 |
+| `model_config.json` |环境与日期参数：训练窗口、滑动/固定模式、市场(market)、基准(benchmark) | 训练、预测、整体环境 |
 | `strategy_config.yaml` | 策略参数：订单生成规则 (TopK等)，回测环境配置 | 订单生成、回测组合 |
-| `prod_config.json` | 实盘状态：持仓、现金、市场(market)、基准(benchmark) | Post-Trade、订单生成 |
+| `prod_config.json` | 实盘与状态层：持仓、现金、处理时间 | Post-Trade、订单生成 |
 | `cashflow.json` | 出入金记录：按日期的出入金 | Post-Trade |
 | `ensemble_config.json` | 多组合融合配置：combo 定义、权重、default | 融合预测、订单生成、信号排名 |
 | `workflow_config_*.yaml` | Qlib 工作流：各模型的训练配置 | 训练 |

docs/06_ORDER_GEN_GUIDE.md

@@ -36,12 +36,13 @@ python quantpits/scripts/order_gen.py --verbose
 
 ## 配置要求
 
-脚本运行需要 `config/prod_config.json` 中的以下关键字段：
+脚本运行会通过 `config_loader` 统一合并工作区配置（包含 `prod_config.json`, `model_config.json`, `strategy_config.yaml`）。以下是影响脚本的核心逻辑字段：
 
-- **market**: (推荐) 指定市场（如 `csi300`, `csi1000`）。若缺失，脚本会尝试从预测标的中自动推断，但建议显式指定。
-- **benchmark**: (推荐) 指定基准指数代码（如 `SH000300`, `SH000852`）。
-- **current_cash**: 当前可用现金余额。
-- **current_holding**: 当前持仓列表 `[{"instrument": "...", "amount": "...", "value": "..."}, ...]`。
+- **market**: (推荐) 交易市场界定（如 `csi300`, `csi1000`）。通常在 `model_config.json` 定义。若缺失，脚本会从预测文件中自动推断，但建议显式指定。
+- **benchmark**: (推荐) 交易基准（如 `SH000300`, `SH000852`）。通常在 `model_config.json` 定义。
+- **current_cash**: 当前可用现金余额。由 Post-Trade 脚本自动更新维护在 `prod_config.json` 中。
+- **current_holding**: 存量持仓状态 `[{"instrument": "...", "amount": "...", "value": "..."}, ...]`。由 Post-Trade 脚本自动更新维护在 `prod_config.json` 中。
+- **topk** / **n_drop** / **buy_suggestion_factor**: 订单生成策略参数。定义于 `strategy_config.yaml` 或兼容旧版由根配置管理。
 
 > [!NOTE]
 > 脚本具备**自动容错**能力：即使 `market` 配置错误，也会根据预测文件中的标的列表自动获取对应的价格数据。

docs/en/00_SYSTEM_OVERVIEW.md

@@ -343,9 +343,9 @@ latest_train_records.json   prod_config.json (Update Pos/Cash)
 | File | Purpose | Related Module |
 |------|------|----------|
 | `model_registry.yaml` | Model Registry: Defines model names, algorithms, datasets, labels | Train, Predict |
-| `model_config.json` | Date Params: Training windows, sliding/fixed mode | Train, Predict |
+| `model_config.json` | Environment Params: Training windows, slicing mode, market, benchmark | Train, Predict, Environment |
 | `strategy_config.yaml` | Strategy Params: Order generation boundaries (TopK, etc.), Backtest environment config | Order Gen, Backtesting |
-| `prod_config.json` | Live State: Holdings, Cash, Market, Benchmark | Post-Trade, Order Gen |
+| `prod_config.json` | Live State: Holdings, Cash, Processing state | Post-Trade, Order Gen |
 | `cashflow.json` | Cashflow Records: Deposits/Withdrawals by date | Post-Trade |
 | `ensemble_config.json` | Multi-combo Config: Combo definitions, weights, defaults | Fusion, Order Gen, Ranking |
 | `workflow_config_*.yaml` | Qlib Workflows: Training configurations for each model | Train |

docs/en/06_ORDER_GEN_GUIDE.md

@@ -36,12 +36,13 @@ python quantpits/scripts/order_gen.py --verbose
 
 ## Configuration Requirements
 
-The script relies on the following key fields in `config/prod_config.json`:
+The script loads unified workspace parameters implicitly leveraging `config_loader` (which aggregates `prod_config.json`, `model_config.json`, and `strategy_config.yaml`). Below are the vital operative boundaries steering execution logic:
 
-- **market**: (Recommended) Specifies the target universe (e.g., `csi300`, `csi1000`). If omitted, the script infers it from prediction data, but explicit definition is preferred.
-- **benchmark**: (Recommended) Reference index code (e.g., `SH000300`, `SH000852`).
-- **current_cash**: Current liquid balance for trade allocation.
-- **current_holding**: Array of active positions `[{"instrument": "...", "amount": "...", "value": "..."}, ...]`.
+- **market**: (Recommended) Target operational universe (e.g., `csi300`, `csi1000`). Customarily bounded centrally inside `model_config.json`. If omitted, the architecture auto-infers data scopes utilizing target prediction limits.
+- **benchmark**: (Recommended) Index metric boundaries (e.g., `SH000300`, `SH000852`). Built into `model_config.json`.
+- **current_cash**: Current liquid balance limits defining order sizes. Systematically maintained via Post-Trade executing upon `prod_config.json`.
+- **current_holding**: Active held component quantities arrays. Systematically maintained via Post-Trade executing upon `prod_config.json`.
+- **topk** / **n_drop** / **buy_suggestion_factor**: Operation positioning strategy boundaries natively registered executing inside `strategy_config.yaml`.
 
 > [!NOTE]
 > The engine features **automatic resilience**: even if the `market` configuration is mismatched, it will dynamically fetch pricing for all assets identified in the prediction source.

quantpits/scripts/analysis/utils.py

@@ -25,20 +25,18 @@
 
 def load_market_config():
     """
-    从 model_config.json 读取 market 和 benchmark 配置。
-    返回 (market, benchmark) 元组。
+    使用 config_loader 加载统一配置并返回 market 和 benchmark。
     """
-    market = DEFAULT_MARKET
-    benchmark = DEFAULT_BENCHMARK
-    if os.path.exists(MODEL_CONFIG_FILE):
-        try:
-            with open(MODEL_CONFIG_FILE, 'r') as f:
-                cfg = json.load(f)
-            market = cfg.get("market", DEFAULT_MARKET)
-            benchmark = cfg.get("benchmark", DEFAULT_BENCHMARK)
-        except (json.JSONDecodeError, IOError):
-            pass
-    return market, benchmark
+    from config_loader import load_workspace_config
+    try:
+        config = load_workspace_config(ROOT_DIR)
+        
+        market = config.get("market", DEFAULT_MARKET)
+        benchmark = config.get("benchmark", DEFAULT_BENCHMARK)
+        return market, benchmark
+    except Exception as e:
+        print(f"Warning: Failed to load market config: {e}")
+        return DEFAULT_MARKET, DEFAULT_BENCHMARK
 
 
 def init_qlib():

quantpits/scripts/brute_force_ensemble.py

@@ -113,12 +113,15 @@ def init_qlib():
 
 
 def load_config(record_file="latest_train_records.json"):
-    """加载训练记录和模型配置"""
-    with open(record_file, "r") as f:
-        train_records = json.load(f)
+    """使用 config_loader 加载统一配置"""
+    from config_loader import load_workspace_config
+    if os.path.exists(record_file):
+        with open(record_file, "r") as f:
+            train_records = json.load(f)
+    else:
+        train_records = {"models": {}, "experiment_name": "unknown"}
 
-    with open("config/model_config.json", "r") as f:
-        model_config = json.load(f)
+    model_config = load_workspace_config(ROOT_DIR)
 
     return train_records, model_config
 

quantpits/scripts/brute_force_fast.py

@@ -132,12 +132,15 @@ def init_qlib():
 
 
 def load_config(record_file="latest_train_records.json"):
-    """加载训练记录和模型配置"""
-    with open(record_file, "r") as f:
-        train_records = json.load(f)
+    """使用 config_loader 加载统一配置"""
+    from config_loader import load_workspace_config
+    if os.path.exists(record_file):
+        with open(record_file, "r") as f:
+            train_records = json.load(f)
+    else:
+        train_records = {"models": {}, "experiment_name": "unknown"}
 
-    with open("config/model_config.json", "r") as f:
-        model_config = json.load(f)
+    model_config = load_workspace_config(ROOT_DIR)
 
     return train_records, model_config
 

quantpits/scripts/config_loader.py

@@ -0,0 +1,71 @@
+import os
+import json
+import yaml
+from pathlib import Path
+
+def load_workspace_config(workspace_path):
+    """
+    Unified configuration loader for QuantPits workspaces.
+    Merges model_config.json, prod_config.json, and strategy_config.yaml.
+    
+    Returns a unified dict with all necessary parameters.
+    """
+    workspace_path = Path(workspace_path)
+    config_dir = workspace_path / "config"
+    
+    # Files
+    model_cfg_path = config_dir / "model_config.json"
+    prod_cfg_path = config_dir / "prod_config.json"
+    strat_cfg_path = config_dir / "strategy_config.yaml"
+    
+    config = {}
+    
+    # 1. Load Model Config (Base environment properties)
+    if model_cfg_path.exists():
+        with open(model_cfg_path, 'r') as f:
+            config.update(json.load(f))
+            
+    # 2. Load Strategy Config (Single Source of Truth for strategy params)
+    if strat_cfg_path.exists():
+        with open(strat_cfg_path, 'r') as f:
+            strat_data = yaml.safe_load(f)
+            if strat_data:
+                config['strategy'] = strat_data.get('strategy', {})
+                config['backtest'] = strat_data.get('backtest', {})
+                
+                # Promote core strategy params to top-level for convenience/compatibility
+                strat_params = config['strategy'].get('params', {})
+                config['topk'] = strat_params.get('topk', config.get('TopK'))
+                config['n_drop'] = strat_params.get('n_drop', config.get('DropN'))
+                config['buy_suggestion_factor'] = strat_params.get('buy_suggestion_factor', config.get('buy_suggestion_factor'))
+                
+                # Compatibility mapping (Upper case versions if they don't exist)
+                if 'TopK' not in config: config['TopK'] = config['topk']
+                if 'DropN' not in config: config['DropN'] = config['n_drop']
+                
+    # 3. Load Prod Config (Current state - handles cash/holding)
+    if prod_cfg_path.exists():
+        with open(prod_cfg_path, 'r') as f:
+            prod_data = json.load(f)
+            # We only want State fields from prod_config, others should come from model/strategy
+            state_fields = [
+                'current_date', 'last_processed_date', 'initial_cash', 
+                'current_full_cash', 'initial_holding', 'current_cash', 
+                'current_holding', 'model', 'experiment_name',
+                'current_train_record_id', 'current_pred_record_id'
+            ]
+            for field in state_fields:
+                if field in prod_data:
+                    config[field] = prod_data[field]
+                    
+    # Sanity checks / Cross-file consistency (Optional but recommended)
+    # If market/benchmark exist in both, we prefer model_config but can log warnings if they mismatch
+    
+    return config
+
+if __name__ == "__main__":
+    # Test loading
+    import sys
+    if len(sys.argv) > 1:
+        c = load_workspace_config(sys.argv[1])
+        print(json.dumps(c, indent=2))

quantpits/scripts/ensemble_fusion.py

@@ -84,12 +84,16 @@ def init_qlib():
 
 
 def load_config(record_file="latest_train_records.json"):
-    """加载训练记录、模型配置和 ensemble 配置"""
-    with open(record_file, "r") as f:
-        train_records = json.load(f)
+    """使用 config_loader 加载统一配置"""
+    from config_loader import load_workspace_config
+    if os.path.exists(record_file):
+        with open(record_file, "r") as f:
+            train_records = json.load(f)
+    else:
+        train_records = {"models": {}, "experiment_name": "unknown"}
 
-    with open("config/model_config.json", "r") as f:
-        model_config = json.load(f)
+    # 使用 config_loader 加载工作区配置 (取代直接读取 model_config.json)
+    model_config = load_workspace_config(ROOT_DIR)
 
     ensemble_config = {}
     if os.path.exists("config/ensemble_config.json"):
@@ -331,7 +335,7 @@ def calculate_weights(norm_df, model_metrics, method, model_config,
     """
     model_names = list(norm_df.columns)
 
-    top_k = model_config.get('TopK', 22)
+    top_k = model_config.get('topk', model_config.get('TopK', 20))
     min_ic = ensemble_config.get('min_model_ic', 0.01)
 
     print(f"\n{'='*60}")

quantpits/scripts/order_gen.py

@@ -70,9 +70,9 @@ def get_anchor_date():
 
 
 def load_configs():
-    """加载 prod_config.json 和 cashflow.json"""
-    with open(CONFIG_FILE, 'r') as f:
-        config = json.load(f)
+    """使用 config_loader 加载统一配置并读取 cashflow.json"""
+    from config_loader import load_workspace_config
+    config = load_workspace_config(ROOT_DIR)
 
     cashflow_config = {}
     if os.path.exists(CASHFLOW_FILE):
@@ -626,18 +626,14 @@ def main():
     strategy_config = strategy.load_strategy_config()
     order_gen = strategy.create_order_generator(strategy_config)
 
-    # 策略参数（由 Strategy Provider 统一管理）
+    # 策略参数（由 Strategy Provider 统一管理，底层已切到 config_loader）
     st_params = strategy.get_strategy_params(strategy_config)
     top_k = st_params.get('topk', 20)
     drop_n = st_params.get('n_drop', 3)
     buy_suggestion_factor = st_params.get('buy_suggestion_factor', 2)
 
-    # 运行时状态变量（来自 prod_config）
-    market = config.get('market')
-    if not market:
-        market = 'csi300'
-        print(f"⚠️  Warning: 'market' not found in prod_config.json. Defaulting to '{market}'.")
-
+    # 运行时状态变量（来自 config_loader 整合后的 config）
+    market = config.get('market', 'csi300')
     current_cash = float(config.get('current_cash', 0))
     current_holding = config.get('current_holding', [])
     cash_flow_today = get_cashflow_today(cashflow_config, anchor_date)