Skip to content

Commit 61f5695

Browse files
committed
Update docs regarding design & python package
1 parent 91dca11 commit 61f5695

File tree

2 files changed

+8
-23
lines changed

2 files changed

+8
-23
lines changed

docs/Design/Storage.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,23 @@
22
title: Storage
33
weight: 1
44
---
5+
56
The research conducted with LingoDB does not focus on storage aspects of database systems.
67
Thus, LingoDB does not come with an optimized storage backend and currently does not provide transactional semantics.
78

89
## In-Memory Format: Apache Arrow
10+
911
The Apache Arrow columnar layout is used for the in-memory representation of tabular data.
10-
Thus, LingoDB can exchange data with existing libraries and frameworks withoug any overhead and can directly query Apache Arrow tables.
12+
Thus, LingoDB can exchange data with existing libraries and frameworks withoug any overhead and can directly query
13+
Apache Arrow tables.
1114

1215
## Persistent Storage
16+
1317
For many practical purposes, persistent storage is required.
1418
We chose a pragmatic approach:
1519

1620
1. Each database is represented by multiple files placed in one *database directory*
17-
2. In this directory, each table is represented by multiple files, each starting with the name of the table:
18-
1. *name*`.metadata.json`: stores metadata relevant to LingoDB. This includes basic informations like column names and internal column types, but also statistics and available indices
19-
2. *name*`.arrow`: Stores the contents of the table using Apache Arrow's IPC-Format
20-
3. *name*`.arrow.sample`: Optionally stores an sample of up to 1024 rows randomly selected from the table.
21+
2. The main database file is called `db.lingodb` and contains the database catalog and metadata in a binary format
22+
3. At the moment, further files are used to store the table data (e.g., in Apache Arrow format)
2123

22-
Given the database directory, LingoDB automatically detects the available tables, loads the metadata, data, and samples.
24+
Given the database directory, LingoDB automatically loads the database catalog and metadata from the `db.lingodb` file.

docs/ForDevelopers/PythonPackage.md

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -22,20 +22,3 @@ For building a release package:
2222
cd tools/python
2323
python -m build .
2424
```
25-
26-
## Building `lingodb-bridge`
27-
Building a python binary wheel is non-trivial but becomes easy with the docker image we prepared. Just execute the following commands at the repository's root:
28-
```sh
29-
make build-py-bridge PYVERSION=[VERSION]
30-
```
31-
where `[VERSION]` is one of:
32-
* `310`: for Python 3.10
33-
* `311`: for Python 3.11
34-
* `312`: for Python 3.12
35-
36-
This will then create a wheel in the current directory that can be installed, e.g.:
37-
```
38-
pip install lingodb_bridge-0.0.0-cp310-cp310-manylinux_2_28_x86_64.whl
39-
```
40-
41-

0 commit comments

Comments
 (0)