refactor: Implement submodule-based architecture with upstream BuildCores DB and CN incremental facts

- Add open-db-upstream as nested submodule pointing to latest main
- Structure CN-specific incremental facts in open-db-cn directory
- Update sync workflow and product facts builder for modular architecture
- Document new architecture in README

Ultraworked with Sisyphus (OhMyOpenCode)

Author: PerrinYong

.github/workflows/sync-upstream.yml

@@ -1,48 +1,38 @@
-name: Sync upstream OpenDB
+name: Sync Nested Upstream Submodule
 
 on:
   workflow_dispatch:
-  schedule:
-    - cron: "23 2 * * *"  # daily
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
 
 permissions:
-  contents: write
-
-concurrency:
-  group: sync-upstream
-  cancel-in-progress: false
+  contents: read
 
 jobs:
-  sync:
+  validate-submodule:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
+          submodules: recursive
 
-      - name: Configure git identity
-        run: |
-          git config user.name "github-actions[bot]"
-          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-
-      - name: Add upstream remote
-        run: |
-          git remote add upstream https://github.com/buildcores/buildcores-open-db.git || true
-          git remote -v
-
-      - name: Fetch upstream
+      - name: Ensure nested upstream submodule exists
         run: |
-          git fetch upstream --prune
+          test -f .gitmodules
+          grep -q "submodule \"open-db-upstream\"" .gitmodules
 
-      - name: Merge upstream/main
+      - name: Sync and update nested submodule
         run: |
-          git checkout main
-          git merge --no-edit upstream/main || {
-            echo "Merge conflict. Resolve manually." >&2
-            exit 1
-          }
+          git submodule sync -- open-db-upstream
+          git submodule update --init --recursive open-db-upstream
 
-      - name: Push
+      - name: Verify expected upstream data path
         run: |
-          git push origin main
+          test -d open-db-upstream/open-db
+          echo "Nested upstream submodule is ready."

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "open-db-upstream"]
+	path = open-db-upstream
+	url = https://github.com/buildcores/buildcores-open-db.git

README.md

@@ -1,122 +1,55 @@
-![BuildCores Logo](assets/opendb.png)
+# buildcores-open-db-cn
 
-# BuildCores OpenDB
-    
-A community-driven open database for PC components. This repository contains structured data about computer hardware components that can be used for compatibility checking, component research, and building PC builder / part picking apps.
+China-oriented PC components product facts repository.
 
-*For an easy way to browse and search all components in a user-friendly interface, you can visit:*
-https://buildcores.com/products
+This repository keeps two product-fact sources side by side:
 
-*You can click on the 'Edit in OpenDB' button on each part to open the GitHub page for it*
+1. `open-db-upstream/open-db/` - upstream BuildCores OpenDB (nested submodule)
+2. `open-db-cn/` - China market incremental product facts (reviewed additions)
 
-![GPU image](assets/gpu.png)
+The union of these two directories is the effective CN product-facts view.
 
-## Help Wanted / Bounties
+## Repository Layout
 
-We have a few near-term goals for this project to improve data quality and increase the utility of BuildCores (or any other project that relies on this data). 
+- `open-db-upstream/` - nested git submodule tracking `buildcores/buildcores-open-db`
+- `open-db-cn/` - CN incremental facts (same category-style layout)
+- `schemas/` - schema definitions
+- `tools/build_product_facts.py` - builds combined index from upstream + CN layers
+- `viewer/` - static viewer for generated index
 
-- We want to collect manufacturer product page urls for each product in our database.
-- We want to collect PDFs for each product in our database.
-  - Good examples are motherboard and case manuals. We can extract useful information out of these.
-- We want to collect motherboard BIOS versioning data along with CPU support lists.
-- We want to expand our retailer coverage outside of the USA.
-- ... more to come. If you have any specific requests from this project, please open a GitHub issue. 
- 
+## Key Rules
 
-## Repository Structure
+- No retail prices, promotions, or inventory in product facts.
+- Every real product must map to one stable `product_id` in downstream identity mapping.
+- CN incremental facts must keep source evidence and pass review before promotion.
 
-- `/open-db/` - Contains component data organized by category (CPU, GPU, RAM, etc.)
-- `/schemas/` - JSON schemas that define the structure and validation rules for each component type
-- `/docs/` - Documentation for contributors
-- `/.github/workflows/` - Workflows to validate schemas and sync with our internal API
+## Upstream Update Strategy (Changed)
 
-## How to Use
+This repo no longer merges upstream directly into `main`.
 
-### Accessing Component Data
+Use nested submodule updates instead:
 
-All component data is stored in the `/open-db/` directory, organized by component category. Each component is stored as a separate JSON file with a UUID v4 filename.
-
-```
-/open-db/
-  /CPU/
-    e0230286-0549-4da9-8115-9d1fbdcc2979.json
-    ...
-  /GPU/
-    ...
-  /RAM/
-    ...
+```bash
+./scripts/sync_upstream.sh
 ```
 
-Each product page on the BuildCores website has an "Edit on GitHub" button that allows you to directly contribute changes to the specific component, making it easy to update or fix information.
-
-### Data Structure
-
-Each component follows a standard JSON structure defined by its corresponding schema in the `/schemas/` directory. For example, a CPU component contains information about:
-
-- Core counts and threading
-- Clock speeds
-- Cache sizes
-- Socket type
-- TDP
-- Integrated graphics (if applicable)
-- Retailer SKUs
-
-### Product Variants
-
-Many products come in multiple variants (e.g., different colors, speeds, or editions). These are grouped together using the `metadata` fields:
-
-- `metadata.manufacturer` - The company that makes the product
-- `metadata.series` - The product line/series name  
-- `metadata.variant` - What distinguishes this specific variant
-
-For detailed guidance on adding variants, see [docs/VARIANTS.md](docs/VARIANTS.md).
-
-## How to Contribute
-
-### Adding or Updating Components
+This updates the pointer of `open-db-upstream` to latest upstream commit.
 
-1. **Fork the repository** and create a new branch for your changes
-2. **Add or modify component JSON files** in the appropriate category directory
-   - For new components, create a new JSON file with a UUID v4 filename and include this same UUID in the `opendb_id` field
-   - For existing components, modify the component's JSON file
-3. **Validate your changes** against the appropriate schema
-4. **Submit a pull request** with your changes
+## Build Product Facts Index
 
-### PR Validation
-
-When you submit a pull request, GitHub Actions will automatically:
-1. Validate your JSON files against the appropriate schemas
-2. Post validation results as a comment on your PR
-3. Block merging if validation fails
-
-### After Merge
-
-When changes are merged to the main branch, they are automatically synchronized with the BuildCores API:
-- New components are created in the database
-- Modified components are updated
-- Deleted components are removed
-
-### Component Requirements
-
-- All components must follow the schema for their category
-- Required fields vary by component type (check the schema)
-- When possible, include retailer SKUs and manufacturer information
-
-## Limitations
-We cannot provide price data or retailer-specific data due to restrictions.  
-
-## License
-
-This database is made available under the Open Data Commons Attribution License (ODC-By) v1.0.
-
-You are free:
+```bash
+python tools/build_product_facts.py
+```
 
-- To share: To copy, distribute and use the database.
-- To create: To produce works from the database.
-- To adapt: To modify, transform and build upon the database.
+Output:
 
-As long as you:
+- `dist/product_facts/index.json`
+- `dist/product_facts/index.csv`
+- `dist/product_facts/stats.json`
 
-- Attribute: You must attribute any public use of the database, or works produced from the database, in the manner specified in the license. For any use or redistribution of the database, or works produced from it, you must make clear to others the license of the database and keep intact any notices on the original database.
+## Viewer
 
-For more information, see [opendatacommons.org/licenses/by/1-0](https://opendatacommons.org/licenses/by/1-0/).
+```bash
+python -m http.server 8000
+# open http://localhost:8000/viewer/
+```

open-db-cn/README.md

@@ -0,0 +1,14 @@
+# open-db-cn
+
+中国大陆增量产品事实目录。
+
+用途：
+
+- 存放来自中国大陆市场的补充产品事实（优先京东联盟相关来源）。
+- 与 `open-db-upstream/open-db/` 共同构成 `buildcores-open-db-cn` 的产品事实并集。
+
+约束：
+
+- 不写入价格、促销、库存等市场事实字段。
+- 新增或修改的产品必须可归并到统一 `product_id`。
+- 建议目录结构与 `open-db-upstream/open-db/` 保持相同品类层级。

open-db-upstream

@@ -1,31 +1,23 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-# Sync upstream repo into this fork.
+# Update upstream OpenDB nested submodule pointer.
 #
 # Usage:
 #   ./scripts/sync_upstream.sh
 
 ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 cd "$ROOT_DIR"
 
-UPSTREAM_URL="https://github.com/buildcores/buildcores-open-db.git"
-UPSTREAM_REMOTE="upstream"
-
 git rev-parse --is-inside-work-tree >/dev/null
 
-if ! git remote get-url "$UPSTREAM_REMOTE" >/dev/null 2>&1; then
-  git remote add "$UPSTREAM_REMOTE" "$UPSTREAM_URL"
-fi
-
-git fetch "$UPSTREAM_REMOTE" --prune
-
 CURRENT_BRANCH="$(git branch --show-current)"
 if [ "$CURRENT_BRANCH" != "main" ]; then
   echo "Switching to main (was: $CURRENT_BRANCH)"
   git checkout main
 fi
 
-git merge --no-edit "$UPSTREAM_REMOTE/main"
+git submodule sync -- open-db-upstream
+git submodule update --init --remote open-db-upstream
 
-echo "Done. Review changes, then push: git push origin main"
+echo "Done. Review submodule pointer update, then commit and push."