[opt] update iceberg and maxcompute doc (#3406)

Author: morningman

docs/lakehouse/best-practices/doris-dlf-paimon.md

@@ -1,44 +1,44 @@
 ---
 {
-    "title": "Integration with Aliyun DLF Rest Catalog",
+    "title": "Integrating Alibaba Cloud DLF Rest Catalog",
     "language": "en",
-    "description": "Aliyun Data Lake Formation (DLF) serves as a core component of cloud-native data lake architecture,"
+    "description": "This article explains how to integrate Apache Doris with Alibaba Cloud DLF (Data Lake Formation) Rest Catalog for seamless access and analysis of Paimon table data, including guides on creating Catalog, querying data, and incremental reading."
 }
 ---
 
-Aliyun [Data Lake Formation (DLF)](https://www.alibabacloud.com/en/product/datalake-formation) serves as a core component of cloud-native data lake architecture, helping users quickly build cloud-native data lake architectures. Data Lake Formation provides unified metadata management on the lake, enterprise-level permission control, and seamlessly integrates with multiple computing engines to break data silos and uncover business value.
+Alibaba Cloud [Data Lake Formation (DLF)](https://cn.aliyun.com/product/bigdata/dlf), as a core component of the cloud-native data lake architecture, helps users quickly build cloud-native data lake solutions. DLF provides unified metadata management on the data lake, enterprise-level permission control, and seamless integration with multiple compute engines, breaking down data silos and enabling business insights.
 
 - Unified Metadata and Storage
 
-    Computing engines share a unified set of lake metadata and storage, enabling data flow between lake ecosystem products.
+    Big data compute engines share a single set of lake metadata and storage, with data flowing seamlessly between lake products.
 
 - Unified Permission Management
 
-    Computing engines share a unified set of lake table permission configurations, achieving one-time configuration with multi-location effectiveness.
+    Big data compute engines share a single set of lake table permission configurations, enabling one-time setup with universal effect.
 
 - Storage Optimization
 
-    Provides optimization strategies including small file merging, expired snapshot cleanup, partition organization, and obsolete file cleanup to improve storage efficiency.
+    Provides optimization strategies including small file compaction, expired snapshot cleanup, partition reorganization, and obsolete file cleanup to improve storage efficiency.
 
 - Comprehensive Cloud Ecosystem Support
 
-    Deep integration with Alibaba Cloud products, including streaming and batch computing engines, enabling out-of-the-box functionality and enhancing user experience and operational convenience.
+    Deep integration with Alibaba Cloud products, including streaming and batch compute engines, delivering out-of-the-box functionality and enhanced user experience.
 
-Starting from DLF version 2.5, Paimon Rest Catalog is supported. Doris, beginning from version 3.1.0, supports integration with DLF 2.5+ Paimon Rest Catalog, enabling seamless connection to DLF for accessing and analyzing Paimon table data. This document demonstrates how to use Apache Doris to connect to DLF 2.5+ and access Paimon table data.
+DLF supports Paimon Rest Catalog starting from version 2.5. Doris supports integration with DLF 2.5+ Paimon Rest Catalog starting from version 3.0.3/3.1.0, enabling seamless connection to DLF for accessing and analyzing Paimon table data. This article demonstrates how to connect Apache Doris with DLF 2.5+ and access Paimon table data.
 
 :::tip
-This feature is supported since Doris 3.1
+This feature is supported starting from Doris version 3.0.3/3.1.0.
 :::
 
 ## Usage Guide
 
 ### 01 Enable DLF Service
 
-Please refer to the DLF official documentation to enable the DLF service and create corresponding Catalog, Database, and Table.
+Please refer to the DLF official documentation to enable the DLF service and create the corresponding Catalog, Database, and Table.
 
 ### 02 Access DLF Using EMR Spark SQL
 
-- Connection
+- Connect
 
     ```sql
     spark-sql --master yarn \
@@ -53,7 +53,7 @@ Please refer to the DLF official documentation to enable the DLF service and cre
         --conf spark.sql.catalog.paimon.dlf.token-loader=ecs
     ```
 
-    > Replace the corresponding `warehouse` and `uri` address.
+    > Replace the corresponding `warehouse` and `uri` addresses.
 
 - Write Data
 
@@ -81,15 +81,15 @@ Please refer to the DLF official documentation to enable the DLF service and cre
     (6, '18-24', 'F', false);
     ```
 
-    If you encounter the following error, please try removing `paimon-jindo-x.y.z.jar` from `/opt/apps/PAIMON/paimon-dlf-2.5/lib/spark3` and restart the Spark service before retrying.
+    If you encounter the following error, try removing `paimon-jindo-x.y.z.jar` from `/opt/apps/PAIMON/paimon-dlf-2.5/lib/spark3`, then restart the Spark service and retry.
 
     ```
     Ambiguous FileIO classes are:
     org.apache.paimon.jindo.JindoLoader
     org.apache.paimon.oss.OSSLoader
     ```
 
-### 03 Connect Doris to DLF
+### 03 Connect to DLF Using Doris
 
 - Create Paimon Catalog
 
@@ -105,8 +105,8 @@ Please refer to the DLF official documentation to enable the DLF service and cre
     );
     ```
 
-    - Doris will use temporary credentials returned by DLF to access OSS object storage, without requiring additional OSS credential information.
-    - Only supports accessing DLF within the same VPC, ensure you provide the correct uri address.
+    - Doris uses the temporary credentials returned by DLF to access OSS object storage, so no additional OSS credentials are required.
+    - DLF can only be accessed within the same VPC. Ensure you provide the correct URI address.
 
 - Query Data
 
@@ -137,7 +137,7 @@ Please refer to the DLF official documentation to enable the DLF service and cre
     +-------------+-------------------------+--------------------+
     ```
 
-- Batch Incremental Reading
+- Incremental Reading
 
     ```sql
     SELECT * FROM users_samples@incr('startSnapshotId'=1, 'endSnapshotId'=2) ORDER BY user_id;
@@ -148,3 +148,4 @@ Please refer to the DLF official documentation to enable the DLF service and cre
     |       4 | 18-24     | F                 |    0 |
     +---------+-----------+-------------------+------+
     ```
+    

docs/lakehouse/best-practices/doris-maxcompute.md

@@ -1,76 +1,85 @@
 ---
 {
-    "title": "From MaxCompute to Doris",
+    "title": "Doris and MaxCompute Data Integration",
     "language": "en",
-    "description": "This document explains how to quickly import data from Alibaba Cloud MaxCompute into Apache Doris using the MaxCompute Catalog."
+    "description": "Achieve bidirectional data integration between Apache Doris and Alibaba Cloud MaxCompute through MaxCompute Catalog, supporting data import, write-back, and database/table management to help enterprises build an efficient lakehouse architecture."
 }
 ---
 
-This document explains how to quickly import data from Alibaba Cloud MaxCompute into Apache Doris using the [MaxCompute Catalog](../catalogs/maxcompute-catalog.md).
+This document describes how to achieve data integration between Apache Doris and Alibaba Cloud MaxCompute through [MaxCompute Catalog](../catalogs/maxcompute-catalog.md):
 
-This document is based on Apache Doris version 2.1.9.
+- **Data Import**: Quickly import data from MaxCompute into Doris for analysis.
+- **Data Write-back** (4.1.0+): Write analysis results or data from other sources in Doris back to MaxCompute.
+- **Database/Table Management** (4.1.0+): Create and manage MaxCompute databases and tables directly in Doris.
+
+This document is based on Apache Doris version 2.1.9. Some features require version 4.1.0 or later.
 
 ## Environment Preparation
 
 ### 01 Enable MaxCompute Open Storage API
 
-In the left navigation bar of the [MaxCompute Console](https://maxcompute.console.aliyun.com/) -> `Tenant Management` -> `Tenant Properties` -> Turn on the `Open Storage (Storage API) switch`.
+In the [MaxCompute Console](https://maxcompute.console.aliyun.com/), navigate to the left sidebar -> `Tenant Management` -> `Tenant Properties` -> Enable the `Open Storage (Storage API) Switch`
 
-### 02 Enable MaxCompute Permissions
+### 02 Grant MaxCompute Permissions
 
-Doris uses AK/SK to access MaxCompute services. Please ensure that the IAM user corresponding to the AK/SK has the following roles or permissions for the corresponding MaxCompute services:
+Doris uses AK/SK to access MaxCompute services. Ensure that the IAM user corresponding to the AK/SK has the following roles or permissions for the MaxCompute service:
 
 ```json
 {
-    "Statement": [{
-            "Action": ["odps:List",
-                "odps:Usage"],
+    "Statement": [
+        {
+            "Action": [
+                "odps:List",
+                "odps:Usage"
+            ],
             "Effect": "Allow",
-            "Resource": ["acs:odps:*:regions/*/quotas/pay-as-you-go"]}],
+            "Resource": ["acs:odps:*:regions/*/quotas/pay-as-you-go"]
+        }
+    ],
     "Version": "1"
 }
 ```
 
-### 03 Confirm Doris and MaxCompute Network Environment
+### 03 Verify Doris and MaxCompute Network Environment
 
-It is strongly recommended that the Doris cluster and MaxCompute service are in the same VPC and ensure that the correct security group is set.
+It is strongly recommended that the Doris cluster and MaxCompute service are in the same VPC, with proper security groups configured.
 
-The examples in this document are tested in the same VPC network environment.
+The examples in this document are tested under the same VPC network conditions.
 
-## Import MaxCompute Data
+## Importing MaxCompute Data
 
 ### 01 Create Catalog
 
 ```sql
 CREATE CATALOG mc PROPERTIES (
-  "type" = "max_compute",
-  "mc.default.project" = "xxx",
-  "mc.access_key" = "AKxxxxx",
-  "mc.secret_key" = "SKxxxxx",
-  "mc.endpoint" = "xxxxx"
+    "type" = "max_compute",
+    "mc.default.project" = "xxx",
+    "mc.access_key" = "AKxxxxx",
+    "mc.secret_key" = "SKxxxxx",
+    "mc.endpoint" = "xxxxx"
 );
 ```
 
-Support Schema Level (3.1.3+):
+To support Schema hierarchy (3.1.3+):
 
 ```sql
 CREATE CATALOG mc PROPERTIES (
-  "type" = "max_compute",
-  "mc.default.project" = "xxx",
-  "mc.access_key" = "AKxxxxx",
-  "mc.secret_key" = "SKxxxxx",
-  "mc.endpoint" = "xxxxx",
-  'mc.enable.namespace.schema' = 'true'
+    "type" = "max_compute",
+    "mc.default.project" = "xxx",
+    "mc.access_key" = "AKxxxxx",
+    "mc.secret_key" = "SKxxxxx",
+    "mc.endpoint" = "xxxxx",
+    "mc.enable.namespace.schema" = "true"
 );
 ```
 
-Please refer to the [MaxCompute Catalog](../catalogs/maxcompute-catalog.md) documentation for details.
+For more details, please refer to the [MaxCompute Catalog](../catalogs/maxcompute-catalog.md) documentation.
 
 ### 02 Import TPCH Dataset
 
-We use the TPCH 100 dataset from the public datasets in MaxCompute as an example (data has already been imported into MaxCompute), and use the `CREATE TABLE AS SELECT` statement to import MaxCompute data into Doris.
+We use the TPCH 100 dataset from MaxCompute public datasets as an example (data has already been imported into MaxCompute), and use the `CREATE TABLE AS SELECT` statement to import MaxCompute data into Doris.
 
-This dataset contains 7 tables. The largest table, `lineitem`, has 16 columns and 600,037,902 rows. It occupies about 30GB of disk space.
+This dataset contains 7 tables. The largest table `lineitem` has 16 columns, 600,037,902 rows, and occupies approximately 30GB of disk space.
 
 ```sql
 -- switch catalog
@@ -87,13 +96,13 @@ CREATE TABLE tpch_100g.region AS SELECT * FROM mc.selectdb_test.region;
 CREATE TABLE tpch_100g.supplier AS SELECT * FROM mc.selectdb_test.supplier;
 ```
 
-In a Doris cluster with a single BE of 16C 64G specification, the above operations take about 6-7 minutes to execute serially.
+On a Doris cluster with a single BE (16C 64G), the above operations executed serially take approximately 6-7 minutes.
 
-### 03 Import Github Event Dataset
+### 03 Import GitHub Event Dataset
 
-We use the Github Event dataset from the public datasets in MaxCompute as an example (data has already been imported into MaxCompute), and use the `CREATE TABLE AS SELECT` statement to import MaxCompute data into Doris.
+We use the GitHub Event dataset from MaxCompute public datasets as an example (data has already been imported into MaxCompute), and use the `CREATE TABLE AS SELECT` statement to import MaxCompute data into Doris.
 
-Here we select data from the `dwd_github_events_odps` table for the 365 partitions from '2015-01-01' to '2016-01-01'. The data has 32 columns and 212,786,803 rows. It occupies about 10GB of disk space.
+Here we select data from 365 partitions of the `dwd_github_events_odps` table, from `2015-01-01` to `2016-01-01`. The data contains 32 columns, 212,786,803 rows, and occupies approximately 10GB of disk space.
 
 ```sql
 -- switch catalog
@@ -106,4 +115,123 @@ AS SELECT * FROM mc.github_events.dwd_github_events_odps
 WHERE ds BETWEEN '2015-01-01' AND '2016-01-01';
 ```
 
-In a Doris cluster with a single BE of 16C 64G specification, the above operation takes about 2 minutes.
+On a Doris cluster with a single BE (16C 64G), the above operation takes approximately 2 minutes.
+
+## Writing Data Back to MaxCompute (4.1.0+)
+
+Starting from version 4.1.0, Doris supports writing data back to MaxCompute. This feature is applicable to the following scenarios:
+
+- **Analysis Result Write-back**: After completing data analysis in Doris, write the results back to MaxCompute for use by other systems.
+- **Data Processing**: Leverage Doris's powerful computing capabilities to perform ETL processing on data, and store the processed data in MaxCompute.
+- **Cross-source Data Integration**: Consolidate data from multiple sources in Doris and write it to MaxCompute for unified management.
+
+:::note
+- This is an experimental feature, supported starting from version 4.1.0.
+- Supports writing to partitioned and non-partitioned tables.
+- Does not support writing to clustered tables, transactional tables, Delta Tables, and external tables.
+:::
+
+### 01 INSERT INTO Append Write
+
+The INSERT operation appends data to the MaxCompute target table.
+
+```sql
+-- Switch to MaxCompute Catalog
+SWITCH mc;
+
+-- Insert a single row of data
+INSERT INTO mc_db.mc_tbl VALUES (val1, val2, val3, val4);
+
+-- Import data from Doris internal table to MaxCompute
+INSERT INTO mc_db.mc_tbl SELECT col1, col2 FROM internal.db1.tbl1;
+
+-- Write to specific columns
+INSERT INTO mc_db.mc_tbl(col1, col2) VALUES (val1, val2);
+
+-- Write to specific partition (you can specify only some partition columns, with the rest written dynamically)
+INSERT INTO mc_db.mc_tbl PARTITION(ds='20250201') SELECT id, name FROM internal.db1.source_tbl;
+```
+
+### 02 INSERT OVERWRITE Overwrite Write
+
+INSERT OVERWRITE completely replaces the existing data in the table with new data.
+
+```sql
+-- Full table overwrite
+INSERT OVERWRITE TABLE mc_db.mc_tbl VALUES (val1, val2, val3, val4);
+
+-- Overwrite from another table
+INSERT OVERWRITE TABLE mc_db.mc_tbl(col1, col2) SELECT col1, col2 FROM internal.db1.tbl1;
+
+-- Overwrite specific partition
+INSERT OVERWRITE TABLE mc_db.mc_tbl PARTITION(ds='20250101') VALUES (10, 'new1');
+```
+
+### 03 CTAS Create Table and Write
+
+You can use the `CREATE TABLE AS SELECT` statement to create a new table in MaxCompute and write data to it.
+
+```sql
+-- Create table in MaxCompute and import data
+CREATE TABLE mc_db.mc_new_tbl AS SELECT * FROM internal.db1.source_tbl;
+```
+
+## Database/Table Management (4.1.0+)
+
+Starting from version 4.1.0, Doris supports creating and deleting databases and tables directly in MaxCompute. This feature is applicable to the following scenarios:
+
+- **Unified Data Management**: Manage metadata from multiple data sources centrally in Doris, without switching to the MaxCompute console.
+- **Automated Data Pipelines**: Dynamically create target tables in ETL workflows to achieve end-to-end automation.
+
+:::note
+- This is an experimental feature, supported starting from version 4.1.0.
+- This feature is only available when the `mc.enable.namespace.schema` property is set to `true`.
+- Supports creating and deleting partitioned and non-partitioned tables.
+- Does not support creating clustered tables, transactional tables, Delta Tables, and external tables.
+:::
+
+### 01 Create and Drop Database
+
+```sql
+-- Switch to MaxCompute Catalog
+SWITCH mc;
+
+-- Create Schema
+CREATE DATABASE IF NOT EXISTS mc_schema;
+
+-- Create using fully qualified name
+CREATE DATABASE IF NOT EXISTS mc.mc_schema;
+
+-- Drop Schema (will also delete all tables within it)
+DROP DATABASE IF EXISTS mc.mc_schema;
+```
+
+:::caution
+For MaxCompute Database, dropping it will also delete all tables within it. Please proceed with caution.
+:::
+
+### 02 Create and Drop Table
+
+```sql
+-- Create non-partitioned table
+CREATE TABLE mc_schema.mc_tbl1 (
+    id INT,
+    name STRING,
+    amount DECIMAL(18, 6),
+    create_time DATETIME
+);
+
+-- Create partitioned table
+CREATE TABLE mc_schema.mc_tbl2 (
+    id INT,
+    val STRING,
+    ds STRING,
+    region STRING
+)
+PARTITION BY (ds, region)();
+
+-- Drop table (will also delete data, including partition data)
+DROP TABLE IF EXISTS mc_schema.mc_tbl1;
+```
+
+For more details, please refer to the [MaxCompute Catalog](../catalogs/maxcompute-catalog.md) documentation.

docs/lakehouse/catalogs/iceberg-catalog.mdx

@@ -1648,6 +1648,18 @@ INSERT OVERWRITE TABLE iceberg_tbl@branch(b1) values (val1, val2, val3, val4);
 INSERT OVERWRITE TABLE iceberg_tbl@branch(b1) (col3, col4) values (val3, val4);
 ```
 
+Since version 4.1.0, support for writing data to static partition(or hybrid):
+
+```sql
+-- Full Static Partition
+INSERT OVERWRITE TABLE iceberg_tbl PARTITION (dt='2025-01-25', region='bj')
+SELECT id, name FROM source_table;
+
+-- Hybrid Partition Mode: "dt" is static, "region" comes from SELECT dynamically
+INSERT OVERWRITE TABLE iceberg_tbl PARTITION (dt='2025-01-25')
+SELECT id, name, region FROM source_table;
+```
+
 ### CTAS
 
 You can create an Iceberg table and write data using the `CTAS` (Create Table As Select) statement: