Skip to content

Commit 028b372

Browse files
committed
update doc regarding dependencies
1 parent 1eb5ca4 commit 028b372

File tree

2 files changed

+76
-13
lines changed

2 files changed

+76
-13
lines changed

docs/ForDevelopers/Dependencies.md

Lines changed: 66 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,68 @@
1-
* All "non-standard" dependencies are packaged as python programs
2-
* Also MLIR/LLVM is packaged as a python program.
3-
* ***This will be subject to change in the near future!*** We are working on using system-wide installed MLIR/LLVM packages and reduce the number of dependencies in general.
1+
LingoDB relies on three main external dependencies:
2+
* [LLVM/MLIR 20](https://github.com/llvm/llvm-project)
3+
* [Apache Arrow 19](https://arrow.apache.org/release/19.0.0.html)
4+
* [Boost Context 1.83](https://www.boost.org/doc/libs/1_83_0/libs/context/doc/html/index.html)
45

6+
**Additional tools and libraries required:**
7+
* C++ compiler supporting C++ 20
8+
* CMake 3.13.4 or newer
9+
* Ninja
10+
* lit (optional, for testing), can be e.g., installed via `pip install lit`
511

6-
### Building the custom MLIR/LLVM package
7-
* in `tools/mlir-package`:
8-
* `docker build -t mlir-package .`
9-
* `docker run -v ".:/built-packages" -v ".:/repo" --rm -it mlir-package /usr/bin/create_package.sh cp312-cp312`
12+
We also provide a [Dockerfile](https://github.com/lingo-db/lingo-db/pkgs/container/lingodb-dev) that contains all dependencies and tools required to build LingoDB.
13+
14+
When building dependencies from source, make sure that either the cmake config files are installed in a system-wide locations, or for example, the `CMAKE_PREFIX_PATH` is set accordingly.
15+
16+
## LLVM/MLIR
17+
### Ubuntu/Linux
18+
Follow the instructions on [https://apt.llvm.org/](https://apt.llvm.org/) to install the repository on your system.
19+
Then install the following packages: `clang-20 llvm-20 libclang-20-dev llvm-20-dev libmlir-20-dev mlir-20-tools clang-tidy-20`
20+
21+
### Binaries
22+
For other recent Linux distributions, you can also rely on the pre-built binaries provided by the LLVM project on the Github release pages.
23+
24+
### Building from Source
25+
26+
```shell
27+
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-20.1.0-rc1/llvm-project-20.1.0-rc1.src.tar.xz
28+
tar -xf llvm-project-20.1.0-rc1.src.tar.xz
29+
mkdir llvm-project-20.1.0-rc1.src/build
30+
cd llvm-project-20.1.0-rc1.src
31+
env VIRTUAL_ENV=/venv cmake -B build -DLLVM_ENABLE_PROJECTS="llvm;mlir;clang;clang-tools-extra" -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_BUILD_EXAMPLES=OFF -DCMAKE_BUILD_TYPE=Release -G Ninja -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_BUILD_TESTS=OFF -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=OFF -DLLVM_ENABLE_DUMP=ON -DLLVM_ENABLE_FFI=ON -DCMAKE_CXX_FLAGS="-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" -DLLVM_PARALLEL_LINK_JOBS=1 -DLLVM_PARALLEL_TABLEGEN_JOBS=10 -DBUILD_SHARED_LIBS=OFF -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_ZLIB=OFF -DCMAKE_INSTALL_PREFIX=[output-dir] llvm/
32+
RUN cmake --build build --target install -j$(nproc)
33+
```
34+
35+
36+
## Apache Arrow
37+
### Ubuntu/Linux
38+
```shell
39+
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
40+
apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
41+
apt-get update
42+
apt-get install libarrow-dev=19.*
43+
```
44+
### Binaries
45+
For other recent Linux distributions, you can also rely on the pre-built binaries provided by the Apache Arrow project.
46+
47+
### Building from Source
48+
49+
```shell
50+
wget https://dlcdn.apache.org/arrow/arrow-19.0.1/apache-arrow-19.0.1.tar.gz
51+
tar -xf apache-arrow-19.0.1.tar.gz
52+
RUN cd apache-arrow-19.0.1/cpp
53+
cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=[output-dir] -DARROW_DEPENDENCY_SOURCE=BUNDLED -DARROW_BUILD_STATIC=ON -DARROW_CSV=ON -DARROW_COMPUTE=ON
54+
cmake --build build --target install -j$(nproc)
55+
```
56+
## Boost Context
57+
### Ubuntu/Linux
58+
```shell
59+
apt-get install libboost-context1.83-dev
60+
```
61+
### Build from Source
62+
```shell
63+
wget https://archives.boost.io/release/1.83.0/source/boost_1_83_0.tar.gz
64+
tar -xf boost_1_83_0.tar.gz
65+
cd boost_1_83_0
66+
./bootstrap.sh --prefix=/usr # or any other directory in the PATH/LD_LIBRARY_PATH
67+
./b2 install --with-context
68+
```

docs/GettingStarted/Install.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,23 +3,27 @@ title: Installation
33
type: docs
44
weight: 1
55
---
6+
## Binaries
7+
On the [Release Page](https://github.com/lingo-db/lingo-db/releases/tag/v0.0.3), you can find pre-built binaries for Linux/AMD64.
8+
These binaries only depend on standard libraries (`libpthread`,`librt`,`libstdc++`,`libm`,`libgcc`,`libc`) and can be run on most Linux systems.
69

10+
11+
12+
13+
14+
[//]: <> (Think about where to distribute the python packages)
715
## Python Package
816
Install via pip, then use as [documented here](./Python.md)
917
```
1018
pip install lingodb
1119
```
1220

13-
## Docker Image
14-
You can build the docker image yourself using `make build-docker`
1521

1622
## Building from source
1723
1. Ensure you have a machine with sufficient compute power and space
18-
1. Make sure that you have the following build dependencies installed:
19-
1. Python3.10 or higher
20-
1. standard build tools, including `cmake` and `Ninja`
24+
1. Make sure that you have the all [dependencies](../ForDevelopers/Dependencies.md) installed
2125
1. Build LingoDB
2226
* Debug Version : `make build-debug` (will create binaries under `build/lingodb-debug`)
2327
* Release Version : `make build-release` (will create binaries under `build/lingodb-release`)
24-
1. Run test: `make run-test`
28+
2529

0 commit comments

Comments
 (0)