Skip to content

Commit 2e2ad70

Browse files
author
Evert
committed
Fixed lint errors
1 parent 06a248d commit 2e2ad70

File tree

1 file changed

+40
-40
lines changed

1 file changed

+40
-40
lines changed

docs/stable/dev/building/python.md

Lines changed: 40 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,14 @@ title: Python
77

88
The DuckDB Python package lives in the main [DuckDB source on Github](https://github.com/duckdb/duckdb/) under the `/tools/pythonpkg/` folder. It uses [pybind11](https://pybind11.readthedocs.io/en/stable/) to create Python bindings with DuckDB.
99

10-
# Prerequisites
10+
## Prerequisites
1111

1212
For everything described on this page we make the following assumptions:
1313

1414
1. You have a working copy of the duckdb source (including the git tags) and you run commands from the root of the source
1515
2. You have a suitable Python installation available in a dedicated virtual env
1616

17-
## 1. DuckDB code
17+
### 1. DuckDB code
1818

1919
Make sure you have checked out the [DuckDB source](https://github.com/duckdb/duckdb/) and that you are in its root. E.g.:
2020

@@ -38,7 +38,7 @@ git fetch --tags upstream
3838
git push --tags
3939
```
4040

41-
## 2. Python Virtual Env
41+
### 2. Python Virtual Env
4242

4343
For everything described here you will need a suitable Python installation. While you technically might be able to use your system Python, we **strongly** recommend you use a Python virtual environment. A virtual environment isolates dependencies and, depending on the tooling you use, gives you control over which Python interpreter you use. This way you don't pollute your system-wide Python with the different packages you need for your projects.
4444

@@ -68,27 +68,27 @@ If that fails with `No module named pip` and you use `uv`, then run:
6868
$ uv pip install pip
6969
```
7070

71-
# Building From Source
71+
## Building From Source
7272

7373
Below are a number of options to build the python library from source, with or without debug symbols, and with a default or custom set of [extensions]({% link docs/stable/extensions/overview.md %}). Make sure to check out the [DuckDB build documentation]({% link docs/stable/dev/building/overview.md %}) if you run into trouble building the DuckDB main library.
7474

75-
## Default release, debug build or cloud storage
75+
### Default release, debug build or cloud storage
7676

7777
The following will build the package with the default set of extensions (json, parquet, icu and core_function).
7878

79-
### Release build
79+
#### Release build
8080

8181
```bash
8282
GEN=ninja BUILD_PYTHON=1 make release
8383
```
8484

85-
### Debug build
85+
#### Debug build
8686

8787
```bash
8888
GEN=ninja BUILD_PYTHON=1 make debug
8989
```
9090

91-
### Cloud Storage
91+
#### Cloud Storage
9292

9393
You may need the package files to reside under the same prefix where the library is installed; e.g., when installing to cloud storage from a notebook.
9494

@@ -110,13 +110,13 @@ tar --directory=$DUCKDB_PREFIX/src/duckdb-pythonpkg -xzpf tools/pythonpkg/dist/d
110110
pip install --prefix $DUCKDB_PREFIX -e $DUCKDB_PREFIX/src/duckdb-pythonpkg/duckdb-${SETUPTOOLS_SCM_PRETEND_VERSION}
111111
```
112112

113-
### Verify
113+
#### Verify
114114

115115
```bash
116116
python3 -c "import duckdb; print(duckdb.sql('SELECT 42').fetchall())"
117117
```
118118

119-
## Adding extensions
119+
### Adding extensions
120120

121121
Before thinking about statically linking extensions you should know that the Python package currently doesn't handle linked in extensions very well. If you don't really need to have an extension baked in than the advice is to just stick to [installing them at runtime]({% link docs/stable/extensions/installing_extensions.md %}). See `tools/pythonpkg/duckdb_extension_config.cmake` for the default list of extensions that are built with the python package. Any other extension should be considered problematic.
122122

@@ -131,35 +131,35 @@ The DuckDB build process follows the following logic for building extensions:
131131

132132
The following mechanisms add to the set of **_included_ extensions**:
133133

134-
| Mechanism | Syntax / Example |
135-
| --- | --- |
136-
| **Built-in extensions enabled by default** | `extension/extension_config.cmake` (≈30 built-ins) |
137-
| **Python package extensions enabled by default** | `tools/pythonpkg/duckdb_extension_config.cmake` (`json;parquet;icu`) |
138-
| **Semicolon-separated include list** | `DUCKDB_EXTENSIONS=fts;tpch;json` |
139-
| **Flags** | `BUILD_TPCH=1`, `BUILD_JEMALLOC=1`, `BUILD_FTS=1`, … |
140-
| **Presets** | `BUILD_ALL_EXT=1` - Build all in-tree extensions<br/>`BUILD_ALL_IT_EXT=1` - _Only_ build in-tree extensions<br/>`BUILD_ALL_OOT_EXT=1` - Build all out-of-tree extensions |
141-
| **Custom config file(s)** | `DUCKDB_EXTENSION_CONFIGS=path/to/my.cmake` |
142-
| **Core-only overrides** <br/>_only relevant with `DISABLE_BUILTIN_EXTENSIONS=1`_ | `CORE_EXTENSIONS=httpfs;fts` |
134+
| Mechanism | Syntax / Example |
135+
| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
136+
| **Built-in extensions enabled by default** | `extension/extension_config.cmake` (≈30 built-ins) |
137+
| **Python package extensions enabled by default** | `tools/pythonpkg/duckdb_extension_config.cmake` (`json;parquet;icu`) |
138+
| **Semicolon-separated include list** | `DUCKDB_EXTENSIONS=fts;tpch;json` |
139+
| **Flags** | `BUILD_TPCH=1`, `BUILD_JEMALLOC=1`, `BUILD_FTS=1`, … |
140+
| **Presets** | `BUILD_ALL_EXT=1` - Build all in-tree extensions<br/>`BUILD_ALL_IT_EXT=1` - _Only_ build in-tree extensions<br/>`BUILD_ALL_OOT_EXT=1` - Build all out-of-tree extensions |
141+
| **Custom config file(s)** | `DUCKDB_EXTENSION_CONFIGS=path/to/my.cmake` |
142+
| **Core-only overrides** <br/>_only relevant with `DISABLE_BUILTIN_EXTENSIONS=1`_ | `CORE_EXTENSIONS=httpfs;fts` |
143143

144144
---
145145

146146
The following mechanisms add to the set of **_excluded_ extensions**:
147147

148-
| Mechanism | Syntax / Example |
149-
| --- | --- |
150-
| **Semicolon-separated skip list** | `SKIP_EXTENSIONS=parquet;jemalloc` |
151-
| **Flags** | `DISABLE_PARQUET=1`, `DISABLE_CORE_FUNCTIONS=1`, … |
152-
| **“No built-ins” switch** <br/>_Throws out *every* statically linked extension **except** `core_functions`. Use `CORE_EXTENSIONS=…` to whitelist a subset back in._ | `DISABLE_BUILTIN_EXTENSIONS=1` |
148+
| Mechanism | Syntax / Example |
149+
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
150+
| **Semicolon-separated skip list** | `SKIP_EXTENSIONS=parquet;jemalloc` |
151+
| **Flags** | `DISABLE_PARQUET=1`, `DISABLE_CORE_FUNCTIONS=1`, … |
152+
| **“No built-ins” switch** <br/>_Throws out *every* statically linked extension **except** `core_functions`. Use `CORE_EXTENSIONS=…` to whitelist a subset back in._ | `DISABLE_BUILTIN_EXTENSIONS=1` |
153153

154154
---
155155

156-
## Show all installed extensions
156+
### Show all installed extensions
157157

158158
```bash
159159
python3 -c "import duckdb; print(duckdb.sql('SELECT extension_name, installed, description FROM duckdb_extensions();'))"
160160
```
161161

162-
# Development Environment
162+
## Development Environment
163163

164164
To set up the codebase for development you should run build duckdb as follows:
165165

@@ -178,7 +178,7 @@ Once the build completes, do a sanity check to make sure everything works:
178178
python3 -c "import duckdb; print(duckdb.sql('SELECT 42').fetchall())"
179179
```
180180

181-
## Debugging
181+
### Debugging
182182

183183
The basic recipe is to start `lldb` with your virtual env's Python interpreter and your script, then set a breakpoint and run your script.
184184

@@ -214,11 +214,11 @@ WARNING: Unable to resolve breakpoint to any actual locations.
214214
Target 0: (python3) stopped.
215215
```
216216
217-
## Debugging in an IDE / CLion
217+
### Debugging in an IDE / CLion
218218
219219
After creating a debug build with `PYTHON_DEV` enabled, you should be able to get debugging going in an IDE that support `lldb`. Below are the instructions for CLion, but you should be able to get this going in e.g. VSCode as well.
220220
221-
### Configure the CMake Debug Profile
221+
#### Configure the CMake Debug Profile
222222
223223
This is a prerequisite for debugging, and will enable Intellisense and clang-tidy by generating a `compile-commands.json` file so your IDE knows how to inspect the source code. It also makes sure your Python virtual env can be found by your IDE's cmake.
224224
@@ -229,7 +229,7 @@ Under `Settings | Build, Execution, Deployment | CMake` add the following CMake
229229
-DBUILD_PYTHON=1 -DPYTHON_DEV=1
230230
```
231231
232-
### Create a run config for debugging
232+
#### Create a run config for debugging
233233
234234
Under Run -> Edit Configurations... create a new CMake Application. Use the following values:
235235
* Name: Python Debug
@@ -241,11 +241,11 @@ That should be enough: Save and close.
241241
242242
Now you can set a breakpoint in a C++ file. You then open your Python script in your editor and use this config to start a debug session.
243243
244-
## Development and Stubs
244+
### Development and Stubs
245245
246246
`*.pyi` stubs in `duckdb-stubs` are manually maintained. The connection-related stubs are generated using dedicated scripts in `tools/pythonpkg/scripts/`:
247-
- `generate_connection_stubs.py`
248-
- `generate_connection_wrapper_stubs.py`
247+
* `generate_connection_stubs.py`
248+
* `generate_connection_wrapper_stubs.py`
249249
250250
These stubs are important for autocomplete in many IDEs, as static-analysis based language servers can't introspect `duckdb`'s binary module.
251251
@@ -256,7 +256,7 @@ python3 -m pytest tests/stubs
256256
257257
If you add new methods to the DuckDB Python API, you'll need to manually add corresponding type hints to the stub files.
258258
259-
## What are py::objects and a py::handles??
259+
### What are py::objects and a py::handles??
260260
261261
These are classes provided by pybind11, the library we use to manage our interaction with the python environment.
262262
py::handle is a direct wrapper around a raw PyObject* and does not manage any references.
@@ -268,9 +268,9 @@ I say *can* because it doesn't have to, using `py::reinterpret_borrow<py::object
268268
269269
When directly interacting with python functions that return a `PyObject*`, such as `PyDateTime_DATE_GET_TZINFO`, you should generally wrap the call in `py::reinterpret_steal` to take ownership of the returned object.
270270
271-
# Troubleshooting
271+
## Troubleshooting
272272
273-
## Pip fails with `No names found, cannot describe anything`
273+
### Pip fails with `No names found, cannot describe anything`
274274
275275
If you've forked DuckDB you may run into trouble when building the Python package when you haven't pulled in the tags.
276276
@@ -286,7 +286,7 @@ git fetch --tags upstream
286286
git push --tags
287287
```
288288
289-
## Building with the httpfs extension Fails
289+
### Building with the httpfs extension Fails
290290
291291
The build fails on OSX when both the [`httpfs` extension]({% link docs/stable/extensions/httpfs/overview.md %}) and the Python package are included:
292292
@@ -300,7 +300,7 @@ make: *** [release] Error 1
300300
301301
Linking in the httpfs extension is problematic. Please install it at runtime, if you can.
302302
303-
## Importing duckdb fails with `symbol not found in flat namespace`
303+
### Importing duckdb fails with `symbol not found in flat namespace`
304304
305305
If you seen an error that looks like this:
306306
@@ -310,7 +310,7 @@ ImportError: dlopen(/usr/bin/python3/site-packages/duckdb/duckdb.cpython-311-dar
310310
311311
... then you've probably tried to link in a problematic extension. As mentioned above: `tools/pythonpkg/duckdb_extension_config.cmake` contains the default list of extensions that are built with the python package. Any other extension might cause problems.
312312
313-
## Python fails with `No module named 'duckdb.duckdb'`
313+
### Python fails with `No module named 'duckdb.duckdb'`
314314
315315
If you're in `tools/pythonpkg` and try to `import duckdb` you might see:
316316
@@ -324,4 +324,4 @@ Traceback (most recent call last):
324324
ModuleNotFoundError: No module named 'duckdb.duckdb'
325325
```
326326
327-
This is because Python imported from the `duckdb` directory (i.e. `tools/pythonpkg/duckdb/`), rather than from the installed package. You should start your interpreter from a different directory instead.
327+
This is because Python imported from the `duckdb` directory (i.e. `tools/pythonpkg/duckdb/`), rather than from the installed package. You should start your interpreter from a different directory instead.

0 commit comments

Comments
 (0)