update ddl and results

Author: wangmj17

hologres/README.md

@@ -2,22 +2,95 @@ Hologres is an all-in-one real-time data warehouse engine that is compatible wit
 
 To evaluate the performance of Hologres, follow these guidelines to set up and execute the benchmark tests.
 
-1. **Instance Purchase**:  
-   Refer to the [Alibaba Cloud Hologres TPC-H Testing Documentation](https://www.alibabacloud.com/help/en/hologres/user-guide/test-plan?spm=a2c63.p38356.help-menu-113622.d_2_14_0_0.54e14f70oTAEXO) for details on purchasing Hologres and ECS instances. Both instances must be purchased within the same region and same zone.
+### 1. Create an Alibaba Cloud Account and Provide Your UID  
+Please first create an Alibaba Cloud account. After registration, kindly provide us with your **UID** (Account ID), which you can find by:  
+- Clicking on your profile icon in the top-right corner → **Account Center** 
+We will issue you an **Alibaba Cloud coupon** to support your testing, so please share your UID with us.
 
-2. **Benchmark Execution**:  
-   Once the instances are set up, you need to prepare the following parameters:
-   - `user`: user name for hologres, you can create users on Hologres web console
-   - `password`: password for hologres, you can set this when create users
-   - `host_name`: hostname of the Hologres instance, you can find this on Alibaba Cloud Console, you should select VPC network to achieve best performance
-   - `port`: Port of the Hologres instance (usally '80')
+---
 
-   And then setup environments variables:
-   ```
-   export PG_USER={user};export PG_PASSWORD={password};export PG_HOSTNAME={host_name};export PG_PORT={port}
-   ```
+### 2. Purchase an Alibaba Cloud Hologres and ECS Instance  
+Refer to the [Alibaba Cloud Hologres TPC-H Testing Documentation](https://www.alibabacloud.com/help/en/hologres/user-guide/test-plan?spm=a2c63.p38356.help-menu-113622.d_2_14_0_0.54e14f70oTAEXO) for details on purchasing Hologres and ECS instances. Both instances must be purchased within the same region and same zone.
+
+#### 2.1 When creating the Hologres instance, please use the following configuration:
+
+- **Region**: `China (Beijing)`  
+  *(The new version is in gray-scale release in China (Beijing). Choosing this region ensures you can access the latest features)*
+- **Specifications**: ✅ **Compute Group Type**  
+- **Zone**: `Zone L`  
+- **Gateway Nodes**: `2 Pieces`  
+- **Reserved Computing Resources of Virtual Warehouse**: `32 CU`  
+  *(This is the actual compute unit (CU) value used in the JSON result files.)*
+- **Allocate to Initial Virtual Warehouse**: `Yes`
+- **Enable Serverless Computing**: ✅ **True (Enabled)**  
+- **Storage Redundancy Type**: `LRS `
+- **VPC & vSwitch**:  
+  - You need to **create a new VPC**.  
+    - Region: `China (Beijing)`  
+    - Name: Any name you prefer  
+    - IPv4 CIDR Block: Select "Manually enter" and use one of the recommended values  
+    - IPv6 CIDR Block: `Do Not Assign`  
+  - During VPC creation, you’ll also create a **vSwitch**:  
+    - Name: Any name  
+    - Zone: `Beijing Zone L`  
+    - IPv4 CIDR: Automatically filled based on VPC CIDR  
+  > 💡 A **VPC (Virtual Private Cloud)** is a private network in the cloud. The **vSwitch** is a subnet within the VPC. We need both Hologres and ECS instances in the same VPC for fast internal communication.
+- **Instance Name**: Choose any name
+- **Service-linked Role**: Click **Create**
+
+Once everything is configured and you’ve received the coupon, click **Buy Now** to proceed.
+
+#### 2.2 When creating the ECS instance, please use the following configuration:
+- **Billing Method**: `Pay-as-you-go` (you can release it after testing)
+- **Region**: `China (Beijing)`
+- **Network & Security Group**:  
+  - VPC: Select the one you just created  
+  - vSwitch: Automatically populated
+- **Instance Type**:  
+  - Series: `Compute Optimized c9i`  
+  - Instance: `ecs.c9i.4xlarge` (16 vCPUs, 32 GiB RAM)  
+  *(This is not performance-critical — it only runs the client script.)*
+- **Image**:  
+  - `Alibaba Cloud Linux` → `Alibaba Cloud Linux 3.2104 LTS 64-bit`  
+- **System Disk**:  
+  - Size: `2048 GiB`  
+  - Performance: `PL3`  
+  *(Larger and faster disk improves import speed since we’re loading ~70GB of TSV data. IO on the ECS can be a bottleneck.)*
+- **Public IP Address**: ✅ Assign Public IPv4 Address
+- **Management Settings**:  
+  - Logon Credential: `Custom Password`  
+  - Username: `root`  
+  - Set a secure password
+
+Click **Create Order** to launch the instance.
+
+---
 
-3. **Sample Execution**:
+### 3. Connect to the ECS and Run the Benchmark  
+
+After the ECS instance is ready:
+
+1. SSH into the ECS instance.
+2. Install Git and clone the repo:
    ```bash
-   ./main.sh 5 /root/bluesky
+   yum -y install git
+   git clone https://github.com/ClickHouse/JSONBench.git
+   cd JSONBench/hologres
+   ```
+3. Run the benchmark script:
+   ```
+    export PG_USER={AccessKeyID};export PG_PASSWORD={AccessKeySecret};export PG_HOSTNAME={Host};export PG_PORT={Port}
+   ./main.sh 5 {your_bluesky_data_dir}
    ```
+
+   - **AccessKeyID & AccessKeySecret**:  
+     Go to the Alibaba Cloud Console → Profile Icon → **AccessKey** → Create one if needed.
+
+     You can also create a hologres user (Click your instance to enter instance detail page -> click "Account Management"  -> "Create Custom User" -> Choose "Superuser") and use the username and password for PG_USER and PG_PASSWORD.
+   - **Host & Port**:  
+     In the Hologres console, click your instance ID → Copy the **VPC Endpoint** (e.g., `hgxxx-cn-beijing-vpc.hologres.aliyuncs.com:xxxx`).  
+     - `Host` = domain without port (e.g., `hgxxx-cn-beijing-vpc.hologres.aliyuncs.com`)  
+     - `Port` = the number after `:`
+
+---
+

hologres/ddl.sql

@@ -1,6 +1,32 @@
+set hg_experimental_enable_nullable_clustering_key = true;
 CREATE TABLE bluesky (
-    data JSONB NOT NULL
-);
+    data JSONB NOT NULL,
+    sort_key TEXT GENERATED ALWAYS AS (
+        -- col1: kind
+        CASE 
+            WHEN data ->> 'kind' IS NULL THEN '[NULL]'
+            ELSE '[VAL]' || (data ->> 'kind')
+        END || '|__COL1__|' ||
+
+        -- col2: operation
+        CASE 
+            WHEN data -> 'commit' ->> 'operation' IS NULL THEN '[NULL]'
+            ELSE '[VAL]' || (data -> 'commit' ->> 'operation')
+        END || '|__COL2__|' ||
+
+        -- col3: collection
+        CASE 
+            WHEN data -> 'commit' ->> 'collection' IS NULL THEN '[NULL]'
+            ELSE '[VAL]' || (data -> 'commit' ->> 'collection')
+        END || '|__COL3__|' ||
+
+        -- col4: did
+        CASE 
+            WHEN data ->> 'did' IS NULL THEN '[NULL]'
+            ELSE '[VAL]' || (data ->> 'did')
+        END
+    ) STORED
+) WITH (clustering_key='sort_key');
 
 ALTER TABLE bluesky ALTER COLUMN data SET (enable_columnar_type = ON);
 CALL set_table_property('bluesky', 'dictionary_encoding_columns', 'data:auto');

hologres/results/hologres_bluesky_1000m.json

@@ -1,19 +1,19 @@
 {
   "system": "Alibaba Cloud Hologres",
   "version": "r4.0",
-  "date": "2025-09-12",
+  "date": "2025-10-09",
   "machine": "Hologres Instance 32CU",
   "retains_structure": "yes",
   "tags": [
   ],
   "dataset_size": 1000000000,
-  "num_loaded_documents": 993999604,
-  "total_size":  113935090070,
+  "num_loaded_documents": 996999604,
+  "total_size":  127917465027,
   "result": [
-	  [1.28344,0.387996,0.394197],
-[14.0496,12.3343,13.6941],
-[1.63458,0.75316,0.754978],
-[3.82664,2.58757,2.50737],
-[3.60571,2.67633,2.66443]
+[0.697501,0.374251,0.365646],
+[12.2569,13.2993,12.0589],
+[0.939368,0.928345,0.903049],
+[1.83206,1.7379,1.73663],
+[1.84332,1.83347,1.83504]
   ]
 }

hologres/results/hologres_bluesky_100m.json

@@ -1,19 +1,19 @@
 {
   "system": "Alibaba Cloud Hologres",
   "version": "r4.0",
-  "date": "2025-09-12",
+  "date": "2025-10-09",
   "machine": "Hologres Instance 32CU",
   "retains_structure": "yes",
   "tags": [
   ],
   "dataset_size": 100000000,
-  "num_loaded_documents":  97999984,
-  "total_size":  11157250421,
+  "num_loaded_documents":  99999984,
+  "total_size":  12601313624,
   "result": [
-	  [0.164415,0.060582,0.061967],
-[2.1726,1.81505,1.83729],
-[0.202728,0.096507,0.097319],
-[0.397231,0.249259,0.246972],
-[0.407714,0.265204,0.265106]
+[0.101054,0.057932,0.055567],
+[2.03141,1.84107,1.81131],
+[0.120029,0.082595,0.08217],
+[0.207844,0.200552,0.20372],
+[0.224962,0.220324,0.223028]
   ]
 }

hologres/results/hologres_bluesky_10m.json

@@ -1,19 +1,19 @@
 {
   "system": "Alibaba Cloud Hologres",
   "version": "r4.0",
-  "date": "2025-09-12",
+  "date": "2025-10-09",
   "machine": "Hologres Instance 32CU",
   "retains_structure": "yes",
   "tags": [
   ],
   "dataset_size": 10000000,
   "num_loaded_documents": 9999997,
-  "total_size":  1185315603,
+  "total_size":  1301202397,
   "result": [
-	  [0.041749,0.026402,0.025736],
-[0.349737,0.282813,0.301415],
-[0.053863,0.033616,0.033054],
-[0.0812,0.055007,0.056155],
-[0.089086,0.061353,0.059887]
+[0.036402,0.025785,0.025481],
+[0.305669,0.268813,0.293167],
+[0.038443,0.032315,0.032234],
+[0.049201,0.051427,0.051265],
+[0.054289,0.055561,0.055911]
   ]
 }

hologres/results/hologres_bluesky_1m.json

@@ -1,19 +1,19 @@
 {
   "system": "Alibaba Cloud Hologres",
   "version": "r4.0",
-  "date": "2025-09-12",
+  "date": "2025-10-09",
   "machine": "Hologres Instance 32CU",
   "retains_structure": "yes",
   "tags": [
   ],
   "dataset_size": 1000000,
   "num_loaded_documents": 1000000,
-  "total_size":  121403438,
+  "total_size":  134518668,
   "result": [
-	  [0.05821,0.024254,0.025168],
-[0.096473,0.05756,0.051771],
-[0.074038,0.025263,0.031265],
-[0.094472,0.035952,0.035384],
-[0.08237,0.047902,0.03876]
+[0.023704,0.021303,0.021568],
+[0.053402,0.047572,0.046956],
+[0.025029,0.023787,0.022639],
+[0.026078,0.025655,0.02596],
+[0.027063,0.028376,0.039474]
   ]
 }

hologres/run_queries.sh

@@ -15,9 +15,12 @@ echo "[$(date '+%Y-%m-%d %H:%M:%S')] $(basename "$0") START"
 
 cat queries.sql | while read -r query; do
 
-    echo "Clearing cache..."
-    $HOLOGRES_PSQL -d "$DB_NAME" -c "select hg_admin_command('freecache');"
-    echo "Cache cleared."
+    # Hologres stores data in Pangu (distributed FS), which has no caching - therefore no need to clear OS or Pangu cache.
+
+    # echo "Clearing file system cache..."
+    # sync
+    # echo 3 | sudo tee /proc/sys/vm/drop_caches >/dev/null
+    # echo "File system cache cleared."
 
     # Print the query
     echo "(TRIES: $TRIES) Running query: $query"