Skip to content

Commit 9248408

Browse files
guillesdguillesd
authored
Merge pull request #299 from guillesd/release_swap
swap preview with stable, add 0.3 to archive
2 parents 60a42ab + ec5b390 commit 9248408

File tree

134 files changed

+5057
-69
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

134 files changed

+5057
-69
lines changed

‎_data/menu_docs_03.json‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
{
2+
"docsmenu": [
3+
{
4+
"page": "Documentation",
5+
"open": true,
6+
"slug": "",
7+
"mainfolderitems": [
8+
{
9+
"page": "Specification",
10+
"slug": "specification",
11+
"subfolderitems": [
12+
{
13+
"page": "Introduction",
14+
"url": "introduction"
15+
},
16+
{
17+
"page": "Data Types",
18+
"url": "data_types"
19+
},
20+
{
21+
"page": "Queries",
22+
"url": "queries"
23+
},
24+
{
25+
"page": "Tables",
26+
"slug": "tables",
27+
"subsubfolderitems": [
28+
{"page": "Overview", "url": "overview"},
29+
{"page": "column", "url": "ducklake_column"},
30+
{"page": "column_mapping", "url": "ducklake_column_mapping"},
31+
{"page": "name_mapping", "url": "ducklake_name_mapping"},
32+
{"page": "column_tag", "url": "ducklake_column_tag"},
33+
{"page": "data_file", "url": "ducklake_data_file"},
34+
{"page": "delete_file", "url": "ducklake_delete_file"},
35+
{"page": "file_column_stats", "url": "ducklake_file_column_stats"},
36+
{"page": "file_partition_value", "url": "ducklake_file_partition_value"},
37+
{"page": "files_scheduled_for_deletion", "url": "ducklake_files_scheduled_for_deletion"},
38+
{"page": "inlined_data_tables", "url": "ducklake_inlined_data_tables"},
39+
{"page": "metadata", "url": "ducklake_metadata"},
40+
{"page": "partition_column", "url": "ducklake_partition_column"},
41+
{"page": "partition_info", "url": "ducklake_partition_info"},
42+
{"page": "schema", "url": "ducklake_schema"},
43+
{"page": "schema_versions", "url": "ducklake_schema_versions"},
44+
{"page": "snapshot", "url": "ducklake_snapshot"},
45+
{"page": "snapshot_changes", "url": "ducklake_snapshot_changes"},
46+
{"page": "table", "url": "ducklake_table"},
47+
{"page": "table_column_stats", "url": "ducklake_table_column_stats"},
48+
{"page": "table_stats", "url": "ducklake_table_stats"},
49+
{"page": "tag", "url": "ducklake_tag"},
50+
{"page": "view", "url": "ducklake_view"}
51+
]
52+
}
53+
]
54+
},
55+
{
56+
"page": "DuckDB Extension",
57+
"open": true,
58+
"slug": "duckdb",
59+
"subfolderitems": [
60+
{
61+
"page": "Introduction",
62+
"url": "introduction"
63+
},
64+
{
65+
"page": "Usage",
66+
"slug": "usage",
67+
"subsubfolderitems": [
68+
{
69+
"page": "Connecting",
70+
"url": "connecting"
71+
},
72+
{
73+
"page": "Choosing a Catalog Database",
74+
"url": "choosing_a_catalog_database"
75+
},
76+
{
77+
"page": "Choosing Storage",
78+
"url": "choosing_storage"
79+
},
80+
{
81+
"page": "Snapshots",
82+
"url": "snapshots"
83+
},
84+
{
85+
"page": "Schema Evolution",
86+
"url": "schema_evolution"
87+
},
88+
{
89+
"page": "Time Travel",
90+
"url": "time_travel"
91+
},
92+
{
93+
"page": "Upserting",
94+
"url": "upserting"
95+
},
96+
{
97+
"page": "Configuration",
98+
"url": "configuration"
99+
},
100+
{
101+
"page": "Paths",
102+
"url": "paths"
103+
}
104+
]
105+
},
106+
{
107+
"page": "Maintenance",
108+
"slug": "maintenance",
109+
"subsubfolderitems": [
110+
{
111+
"page": "Recommended Maintenance",
112+
"url": "recommended_maintenance"
113+
},
114+
{
115+
"page": "Merge Files",
116+
"url": "merge_adjacent_files"
117+
},
118+
{
119+
"page": "Expire Snapshots",
120+
"url": "expire_snapshots"
121+
},
122+
{
123+
"page": "Cleanup of Files",
124+
"url": "cleanup_of_files"
125+
},
126+
{
127+
"page": "Rewrite Files with Deletes",
128+
"url": "rewrite_data_files"
129+
},
130+
{
131+
"page": "Checkpoint",
132+
"url": "checkpoint"
133+
}
134+
]
135+
},
136+
{
137+
"page": "Advanced Features",
138+
"slug": "advanced_features",
139+
"subsubfolderitems": [
140+
{
141+
"page": "Constraints",
142+
"url": "constraints"
143+
},
144+
{
145+
"page": "Conflict Resolution",
146+
"url": "conflict_resolution"
147+
},
148+
{
149+
"page": "Data Change Feed",
150+
"url": "data_change_feed"
151+
},
152+
{
153+
"page": "Data Inlining",
154+
"url": "data_inlining"
155+
},
156+
{
157+
"page": "Encryption",
158+
"url": "encryption"
159+
},
160+
{
161+
"page": "Partitioning",
162+
"url": "partitioning"
163+
},
164+
{
165+
"page": "Transactions",
166+
"url": "transactions"
167+
},
168+
{
169+
"page": "Row Lineage",
170+
"url": "row_lineage"
171+
},
172+
{
173+
"page": "Views",
174+
"url": "views"
175+
},
176+
{
177+
"page": "Comments",
178+
"url": "comments"
179+
}
180+
]
181+
},
182+
{
183+
"page": "Metadata",
184+
"slug": "metadata",
185+
"subsubfolderitems": [
186+
{
187+
"page": "List Files",
188+
"url": "list_files"
189+
},
190+
{
191+
"page": "Adding Files",
192+
"url": "adding_files"
193+
}
194+
]
195+
},
196+
{
197+
"page": "Migrations",
198+
"slug": "migrations",
199+
"subsubfolderitems": [
200+
{
201+
"page": "DuckDB to DuckLake",
202+
"url": "duckdb_to_ducklake"
203+
}
204+
]
205+
},
206+
{
207+
"page": "Guides",
208+
"slug": "guides",
209+
"subsubfolderitems": [
210+
{
211+
"page": "Overview",
212+
"url": "overview"
213+
},
214+
{
215+
"page": "Access Control",
216+
"url": "access_control"
217+
},
218+
{
219+
"page": "Backups and Recovery",
220+
"url": "backups_and_recovery"
221+
},
222+
{
223+
"page": "Using a Remote Data Path",
224+
"url": "using_a_remote_data_path"
225+
},
226+
{
227+
"page": "Public DuckLake on Object Storage",
228+
"url": "public_ducklake_on_object_storage"
229+
}
230+
]
231+
},
232+
{
233+
"page": "Unsupported Features",
234+
"url": "unsupported_features"
235+
}
236+
]
237+
}
238+
]
239+
}
240+
]
241+
}

‎_data/menu_docs_stable.json‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
{"page": "data_file", "url": "ducklake_data_file"},
3434
{"page": "delete_file", "url": "ducklake_delete_file"},
3535
{"page": "file_column_stats", "url": "ducklake_file_column_stats"},
36+
{"page": "file_variant_stats", "url": "ducklake_file_variant_stats"},
3637
{"page": "file_partition_value", "url": "ducklake_file_partition_value"},
3738
{"page": "files_scheduled_for_deletion", "url": "ducklake_files_scheduled_for_deletion"},
3839
{"page": "inlined_data_tables", "url": "ducklake_inlined_data_tables"},
@@ -47,7 +48,12 @@
4748
{"page": "table_column_stats", "url": "ducklake_table_column_stats"},
4849
{"page": "table_stats", "url": "ducklake_table_stats"},
4950
{"page": "tag", "url": "ducklake_tag"},
50-
{"page": "view", "url": "ducklake_view"}
51+
{"page": "view", "url": "ducklake_view"},
52+
{"page": "macro", "url": "ducklake_macro"},
53+
{"page": "macro_impl", "url": "ducklake_macro_impl"},
54+
{"page": "macro_parameters", "url": "ducklake_macro_parameters"},
55+
{"page": "sort_info", "url": "ducklake_sort_info"},
56+
{"page": "sort_expression", "url": "ducklake_sort_expression"}
5157
]
5258
}
5359
]
@@ -169,13 +175,21 @@
169175
"page": "Row Lineage",
170176
"url": "row_lineage"
171177
},
178+
{
179+
"page": "Macros",
180+
"url": "macros"
181+
},
172182
{
173183
"page": "Views",
174184
"url": "views"
175185
},
176186
{
177187
"page": "Comments",
178188
"url": "comments"
189+
},
190+
{
191+
"page": "Sorted Tables",
192+
"url": "sorted_tables"
179193
}
180194
]
181195
},
@@ -219,13 +233,17 @@
219233
"page": "Backups and Recovery",
220234
"url": "backups_and_recovery"
221235
},
236+
{
237+
"page": "Public DuckLake on Object Storage",
238+
"url": "public_ducklake_on_object_storage"
239+
},
222240
{
223241
"page": "Using a Remote Data Path",
224242
"url": "using_a_remote_data_path"
225243
},
226244
{
227-
"page": "Public DuckLake on Object Storage",
228-
"url": "public_ducklake_on_object_storage"
245+
"page": "Troubleshooting",
246+
"url": "troubleshooting"
229247
}
230248
]
231249
},

‎_data/versions.csv‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
version
22
preview
33
stable
4+
0.3
45
0.2
56
0.1

‎docs/0.3/duckdb/advanced_features/comments.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
layout: docu
3+
title: Comments
4+
---
5+
6+
Comments can be added to tables, views and columns using the [`COMMENT ON`](https://duckdb.org/docs/stable/sql/statements/comment_on) syntax.
7+
The comments are stored in the metadata, and can be modified in a transactional manner:
8+
- For tables and views, comments are in the [`ducklake_tag`]({% link docs/0.3/specification/tables/ducklake_tag.md %}) table.
9+
- For columns, comments are in the [`ducklake_column_tag`]({% link docs/0.3/specification/tables/ducklake_column_tag.md %}) table.
10+
11+
## Examples
12+
13+
Create a comment on a `TABLE`:
14+
15+
```sql
16+
COMMENT ON TABLE test_table IS 'very nice table';
17+
```
18+
19+
Create a comment on a `COLUMN`:
20+
21+
```sql
22+
COMMENT ON COLUMN test_table.test_table_column IS 'very nice column';
23+
```

‎docs/0.3/duckdb/advanced_features/conflict_resolution.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
layout: docu
3+
title: Conflict Resolution
4+
---
5+
6+
In DuckLake, snapshot identifiers are written in a sequential order.
7+
The first snapshot has `snapshot_id` 0, and subsequent snapshot ids are monotonically increasing such that the second snapshot has id 1, etc.
8+
The sequential nature of snapshot identifiers is used to **detect conflicts** between snapshots. The [`ducklake_snapshot` table]({% link docs/0.3/specification/tables/ducklake_snapshot.md %}) has a `PRIMARY KEY` constraint defined over the `snapshot_id` column.
9+
10+
When two connections try to write to a `ducklake` table, they will try to write a snapshot with the same identifier and one of the transactions will trigger a `PRIMARY KEY` constraint violation and fail to commit.
11+
When such a conflict occurs – we try to resolve the conflict. In many cases, such as when both transactions are inserting data into a table, we can retry the commit without having to rewrite any actual files.
12+
During conflict resolution, we query the [`ducklake_snapshot_changes` table]({% link docs/0.3/specification/tables/ducklake_snapshot_changes.md %}) to figure out the high-level set of changes that other snapshots have made in the meantime.
13+
14+
* If there are no logical conflicts between the changes that the snapshots have made – we automatically retry the transaction in the metadata catalog without rewriting any data files.
15+
* If there are logical conflicts, we abort the transaction and instruct the user that conflicting changes have been made.
16+
17+
## Logical Conflicts
18+
19+
Below is a list of logical conflicts based on the snapshot's changeset. Snapshots conflict when any of the following conflicts occur:
20+
21+
### Schemas
22+
23+
* Both snapshots create a schema with the same name
24+
* Both snapshots drop the same schema
25+
* A snapshot tries to drop a schema in which another transaction created an entry
26+
27+
### Tables / Views
28+
29+
* Both snapshots create a table or view with the same name in the same schema
30+
* A snapshot tries to create a table or view in a schema that was dropped by another snapshot
31+
* Both snapshots drop the same table or view
32+
* A snapshot tries to alter a table or view that was dropped or altered by another snapshot
33+
34+
### Data
35+
36+
* A snapshot tries to insert data into a table that was dropped or altered by another snapshot
37+
* A snapshot tries to delete data from a table that was dropped, altered, deleted from or compacted by another snapshot
38+
39+
### Compaction
40+
41+
* A snapshot tries to compact a table that was deleted from by another snapshot
42+
* A snapshot tries to compact a table that was dropped by another snapshot

0 commit comments

Comments
 (0)