You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrations/engines/doris.md
+6-67Lines changed: 6 additions & 67 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@
6
6
7
7
SQLMesh supports Doris through its MySQL-compatible protocol, while providing Doris-specific optimizations for table models, indexing, partitioning, and other features. The adapter is designed to leverage Doris's strengths for analytical workloads, with sensible defaults and support for advanced configuration.
8
8
9
-
## Connection Configuration
9
+
## Connection Configuration Example
10
10
11
11
```yaml
12
12
doris:
@@ -21,13 +21,8 @@ doris:
21
21
charset: utf8mb4
22
22
connect_timeout: 60
23
23
state_connection:
24
-
# Use postgres as state connection
25
-
type: postgres
26
-
host: 127.0.0.1
27
-
port: 5432
28
-
user: your_user
29
-
password: your_password
30
-
database: your_database
24
+
# Use duckdb as state connection
25
+
type: duckdb
31
26
```
32
27
33
28
## Table Models
@@ -36,13 +31,6 @@ Doris supports three table models: DUPLICATE, UNIQUE, and AGGREGATE. SQLMesh sup
36
31
37
32
### DUPLICATE Model (Default)
38
33
39
-
The DUPLICATE model allows duplicate data and is optimized for high-throughput scenarios like log data and streaming ingestion.
40
-
41
-
**Features:**
42
-
- **High Write Performance**: Optimized for append-only workloads
43
-
- **No Deduplication**: Allows duplicate records
44
-
- **Streaming Friendly**: Ideal for real-time data ingestion
45
-
46
34
**Example Configuration:**
47
35
```sql
48
36
MODEL (
@@ -61,26 +49,18 @@ MODEL (
61
49
62
50
### UNIQUE Model
63
51
64
-
The UNIQUE model is ideal for dimension tables and scenarios requiring data updates. It ensures key uniqueness and supports efficient UPSERT operations.
65
-
66
-
**Features:**
67
-
- **Primary Key Updates**: New data overwrites existing records with matching keys
68
-
- **Merge-on-Write**: Can be enabled for better query performance
69
-
- **Automatic Deduplication**: Ensures data uniqueness based on specified key columns
70
-
71
52
**Example Configuration:**
72
53
```sql
73
54
MODEL (
74
-
name dim_users,
55
+
name user_events,
75
56
kind FULL,
76
57
physical_properties (
77
58
unique_key 'user_id',
78
59
distributed_by (
79
60
kind = 'HASH',
80
61
expressions = 'user_id',
81
62
buckets = 16
82
-
),
83
-
enable_unique_key_merge_on_write = 'true'
63
+
)
84
64
)
85
65
);
86
66
```
@@ -139,7 +119,7 @@ MODEL (
139
119
140
120
### Partitioning
141
121
142
-
Doris supports range partitioning and list partitioning to improve query performance.
122
+
Doris table supports range partitioning and list partitioning to improve query performance.
143
123
144
124
**Custom Partition Expression:**
145
125
```sql
@@ -327,47 +307,6 @@ FROM user_events
327
307
GROUP BY user_id;
328
308
```
329
309
330
-
## Schema Management
331
-
332
-
### Creating Schemas
333
-
334
-
```sql
335
-
-- Schemas in Doris are databases
336
-
CREATEDATABASEIF NOT EXISTS my_schema;
337
-
```
338
-
339
-
### Dropping Schemas
340
-
341
-
```sql
342
-
-- Doris doesn't support CASCADE clause
343
-
DROPDATABASE my_schema;
344
-
```
345
-
346
-
## Data Operations
347
-
348
-
### Table Operations
349
-
350
-
**Create Table Like:**
351
-
```sql
352
-
-- Create a new table with the same structure as an existing table
353
-
CREATETABLEnew_tableLIKE existing_table;
354
-
```
355
-
356
-
**Rename Table:**
357
-
```sql
358
-
-- Rename a table
359
-
ALTERTABLE old_table_name RENAME new_table_name;
360
-
```
361
-
362
-
**Delete Operations:**
363
-
```sql
364
-
-- Delete specific records
365
-
DELETEFROM table_name WHERE condition;
366
-
367
-
-- Full table deletion uses TRUNCATE for better performance
368
-
DELETEFROM table_name WHERE TRUE; -- Executes as TRUNCATE TABLE
369
-
```
370
-
371
310
## Dependencies
372
311
373
312
To use Doris with SQLMesh, install the required MySQL driver:
0 commit comments