Skip to content

Commit 4c28503

Browse files
committed
Restore support for external platform Python package dependencies
Although Arduino boards platforms traditionally have bundled all dependencies, there are some cases where the user will need to use a preceding workflow step to install external dependencies in the GitHub Actions runner environment. Since it is a "composite action", this is feasible and was previously supported. However, support for Python package dependencies is more complicated due to the action script running in a Python virtual environment which isolates it from packages installed globally. The workaround for this was to use the `--system-site-packages` flag in the venv invocation that created the action's Python virtual environment. This approach introduced a requirement that the project Python version match the minor version series of the runner system Python. That requirement was violated when the workflow's runner was updated to `ubuntu-22.04` which has system Python 3.10.x, whereas the action used 3.8. The breakage went unnoticed because the sole Python package platform dependency in real world usage, pyserial, had been preinstalled in the runner's Linux package manager "site-packages" folder, which caused it to be installed in the action's virtual environment even though it would not have if depending on the user installation of the package as was previously the case. The solution is to use a path configuration file to add the path of the system Python's "user site-packages" folder (where package dependencies installed via the workflow are located) to the module search paths used by the action's Python virtual environment. This is actually superior to the previous approach of using the `--system-site-packages` flag in the venv invocation (this flag is also supported by pipx and Poetry) because: - It avoids the chance of interference with the virtual environment through the unnecessary introduction of the global "site-packages" path into the module search paths along with the intended introduction of the "user site-packages" path. - It allows any version of Python to be used to run the action, rather than being forced to use the same minor version series as the runner's system Python (which can be updated at any time and also depends on the workflow configuration)
1 parent 042dd4a commit 4c28503

File tree

3 files changed

+46
-1
lines changed

3 files changed

+46
-1
lines changed

.github/workflows/test-integration.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,11 @@ jobs:
212212
uses: actions/checkout@v3
213213

214214
- name: Install Python package dependency
215-
run: pip install cowsay
215+
run: |
216+
pip install \
217+
--ignore-installed \
218+
--user \
219+
cowsay
216220
217221
- name: Run action with board that has external Python package dependency
218222
# Use action from local path

action.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@ inputs:
4949
runs:
5050
using: composite
5151
steps:
52+
# User installations of external Python package platform dependencies will be located here.
53+
- name: Get system Python "user site-packages" path
54+
id: system-user-site-packages
55+
shell: bash
56+
run: |
57+
echo "path=$(python -m site --user-site)" >> $GITHUB_OUTPUT
58+
5259
- name: Install Python
5360
uses: actions/[email protected]
5461
with:
@@ -67,6 +74,23 @@ runs:
6774
poetry install \
6875
--only main
6976
77+
- name: Make user-installed Python packages available to platforms
78+
shell: bash
79+
working-directory: ${{ github.action_path }}
80+
run: |
81+
readonly PYTHON_ENVIRONMENT_PATH="$(
82+
poetry env info \
83+
--path
84+
)"
85+
readonly VENV_SITE_PACKAGES_PATH="$(
86+
poetry run \
87+
python -c \
88+
'import site; print(site.getsitepackages()[0])'
89+
)"
90+
echo \
91+
"${{ steps.system-user-site-packages.outputs.path }}" > \
92+
"${VENV_SITE_PACKAGES_PATH}/system-user-site-packages.pth"
93+
7094
- name: Run script
7195
shell: bash
7296
env:

docs/FAQ.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,20 @@ The **arduino/compile-sketches** action runs in the same environment as the rest
4141
```
4242
4343
---
44+
45+
#### Python Packages
46+
47+
The **arduino/compile-sketches** action uses a Python [virtual environment](https://docs.python.org/glossary.html#term-virtual-environment). In order to enable user installation of Python [package](https://docs.python.org/glossary.html#term-package) dependencies of boards platforms, the packages installed in the "[user site-packages](https://peps.python.org/pep-0370/)" folder are included in this virtual environment.
48+
49+
In order to be certain your installation of a package dependency will be available to the platform, add the [`--ignore-installed`](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-ignore-installed) and [`--user`](https://pip.pypa.io/en/stable/cli/pip_install/#install-user) flags to the [**pip**](https://pip.pypa.io/) command used to install the package.
50+
51+
---
52+
53+
**Example:**
54+
55+
```yaml
56+
- run: pip install --ignore-installed --user pyserial
57+
- uses: arduino/compile-sketches@v1
58+
```
59+
60+
---

0 commit comments

Comments
 (0)