Skip to content

Commit 406b57c

Browse files
authored
docs: add new table storage version configuration guide (#177)
Document new_table_data_storage_version and related table creation flags with a decision flowchart, Python/TypeScript examples, and a deprecation notice for the data_storage_version parameter on create_table().
1 parent f5f7720 commit 406b57c

File tree

1 file changed

+52
-0
lines changed

1 file changed

+52
-0
lines changed

docs/storage/configuration.mdx

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,58 @@ If `fetch_storage_options()` returns `expires_at_millis`, LanceDB refreshes cred
120120
These are commonly used options. Cloud-specific keys (for example `region`, `endpoint`, `service_account`, and Azure credential keys) are backend-dependent and can be provided in `storage_options` as needed.
121121
</Info>
122122

123+
#### New table configuration
124+
125+
These options control the Lance file format and features used when creating new tables. Pass them via `storage_options` at connection or table level.
126+
127+
| Key | Values | Default | Description |
128+
| :-- | :-- | :-- | :-- |
129+
| `new_table_data_storage_version` | `legacy`, `stable` | `stable` | Lance file format version for new tables. Use `legacy` for backward compatibility with older clients, or `stable` for the current format with better performance. |
130+
| `new_table_enable_v2_manifest_paths` | `true`, `false` | `false` | Use v2 manifest path naming. Requires LanceDB >= 0.10.0 to read. |
131+
| `new_table_enable_stable_row_ids` | `true`, `false` | `false` | Keep row IDs stable across compaction, delete, and merge operations. |
132+
133+
```mermaid
134+
flowchart TD
135+
A[Creating a new table] --> B{Need backward compatibility\nwith older LanceDB clients?}
136+
B -->|Yes| C[new_table_data_storage_version: legacy]
137+
B -->|No| D[new_table_data_storage_version: stable\nDefault — recommended]
138+
D --> E{Need stable row IDs\nacross compaction and deletes?}
139+
E -->|Yes| F[new_table_enable_stable_row_ids: true]
140+
E -->|No| G[Default: false]
141+
D --> H{All clients on LanceDB >= 0.10.0?}
142+
H -->|Yes| I[new_table_enable_v2_manifest_paths: true]
143+
H -->|No| J[Default: false]
144+
```
145+
146+
<CodeGroup>
147+
```python Python icon="python"
148+
import lancedb
149+
150+
# Set the Lance file format version at connection level
151+
db = lancedb.connect(
152+
"s3://bucket/path",
153+
storage_options={
154+
"new_table_data_storage_version": "stable",
155+
},
156+
)
157+
```
158+
159+
```typescript TypeScript icon="square-js"
160+
import * as lancedb from "@lancedb/lancedb";
161+
162+
// Set the Lance file format version at connection level
163+
const db = await lancedb.connect("s3://bucket/path", {
164+
storageOptions: {
165+
newTableDataStorageVersion: "stable",
166+
},
167+
});
168+
```
169+
</CodeGroup>
170+
171+
<Warning title="Deprecated parameter">
172+
The `data_storage_version` parameter on `create_table()` is deprecated. Use `new_table_data_storage_version` in `storage_options` instead.
173+
</Warning>
174+
123175
## AWS S3
124176

125177
![](/static/assets/images/storage/aws.jpg)

0 commit comments

Comments
 (0)