Skip to content

Commit 9650a82

Browse files
committed
Add developer instructions to speed up build processes
1 parent 31fee39 commit 9650a82

File tree

1 file changed

+53
-0
lines changed

1 file changed

+53
-0
lines changed

docs/source/contributor-guide/introduction.rst

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,3 +95,56 @@ To update dependencies, run
9595
.. code-block:: shell
9696
9797
uv sync --dev --no-install-package datafusion
98+
99+
Improving Build Speed
100+
---------------------
101+
102+
The `pyo3 <https://github.com/PyO3/pyo3>`_ dependency of this project contains a ``build.rs`` file which
103+
can cause it to rebuild frequently. You can prevent this from happening by defining a ``PYO3_CONFIG_FILE``
104+
environment variable that points to a file with your build configuration. Whenever your build configuration
105+
changes, such as during some major version updates, you will need to regenerate this file. This variable
106+
should point to a fully resolved path on your build machine.
107+
108+
To generate this file, use the following command:
109+
110+
.. code-block:: shell
111+
112+
PYO3_PRINT_CONFIG=1 cargo build
113+
114+
This will generate some output that looks like the following. You will want to copy these contents intro
115+
a file. If you place this file in your project directory with filename ``.pyo3_build_config`` it will
116+
be ignored by ``git``.
117+
118+
.. code-block::
119+
120+
implementation=CPython
121+
version=3.8
122+
shared=true
123+
abi3=true
124+
lib_name=python3.12
125+
lib_dir=/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.12/lib
126+
executable=/Users/myusername/src/datafusion-python/.venv/bin/python
127+
pointer_width=64
128+
build_flags=
129+
suppress_build_script_link_lines=false
130+
131+
Add the environment variable to your system.
132+
133+
.. code-block:: shell
134+
135+
export PYO3_CONFIG_FILE="/Users//myusername/src/datafusion-python/.pyo3_build_config"
136+
137+
If you are on a Mac and you use VS Code for your IDE, you will want to add these variables
138+
to your settings. You can find the appropriate rust flags by looking in the
139+
``.cargo/config.toml`` file.
140+
141+
.. code-block::
142+
143+
"rust-analyzer.cargo.extraEnv": {
144+
"RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup",
145+
"PYO3_CONFIG_FILE": "/Users/myusername/src/datafusion-python/.pyo3_build_config"
146+
},
147+
"rust-analyzer.runnables.extraEnv": {
148+
"RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup",
149+
"PYO3_CONFIG_FILE": "/Users/myusername/src/personal/datafusion-python/.pyo3_build_config"
150+
}

0 commit comments

Comments
 (0)