-
Notifications
You must be signed in to change notification settings - Fork 223
fix(docker): bundle Python runtime for portable /agent-server #2676
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
b32e1c7
957aadd
cb7fecd
c4227cd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,8 +20,14 @@ ARG PORT=8000 | |
| # Docker-in-Docker) reject dlopen() on such libraries with: | ||
| # "cannot enable executable stack as shared object requires: Invalid argument" | ||
| # Debian's CPython packages do not have this issue. | ||
| # | ||
| # PORTABILITY: After the venv is built we bundle the interpreter, stdlib, | ||
| # and libpython into /agent-server/.python/ and repoint the venv at it. | ||
| # This makes /agent-server fully self-contained — downstream consumers | ||
| # (e.g. eval images) can COPY it onto any base image without needing a | ||
| # compatible system Python. | ||
| #################################################################################### | ||
| FROM python:3.13-bookworm AS builder | ||
| FROM --platform=$TARGETPLATFORM python:3.13-bookworm AS builder | ||
| ARG USERNAME UID GID | ||
| ENV UV_PROJECT_ENVIRONMENT=/agent-server/.venv | ||
|
|
||
|
|
@@ -40,6 +46,51 @@ COPY --chown=${USERNAME}:${USERNAME} openhands-agent-server ./openhands-agent-se | |
| RUN --mount=type=cache,target=/home/${USERNAME}/.cache,uid=${UID},gid=${GID} \ | ||
| uv venv --python-preference only-system .venv && uv sync --frozen --no-editable --extra boto3 | ||
|
|
||
| # Bundle the Python runtime inside /agent-server so that the entire directory | ||
| # is self-contained and portable. Eval images (and any other consumer) can | ||
| # COPY /agent-server onto *any* base image without requiring that base image | ||
| # to ship a compatible system Python. | ||
| # | ||
| # What we copy: | ||
| # .python/bin/python3.13 – the interpreter binary | ||
| # .python/lib/python3.13/ – the standard library (minus tests) | ||
| # .python/lib/libpython*.so* – shared libraries (Debian builds --enable-shared) | ||
| # | ||
| # We then repoint the venv's symlinks and pyvenv.cfg at the bundled copy. | ||
| RUN set -eux; \ | ||
| REAL_PYTHON=$(readlink -f .venv/bin/python3); \ | ||
| PY_VER=$("${REAL_PYTHON}" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')"); \ | ||
| PYTHON_PREFIX=$("${REAL_PYTHON}" -c "import sys; print(sys.base_prefix)"); \ | ||
| # --- copy interpreter binary ------------------------------------------------- \ | ||
| mkdir -p .python/bin; \ | ||
| cp "${REAL_PYTHON}" ".python/bin/python${PY_VER}"; \ | ||
| ln -s "python${PY_VER}" .python/bin/python3; \ | ||
| ln -s "python${PY_VER}" .python/bin/python; \ | ||
| # --- copy standard library (skip test suite to save ~30 MB) ------------------ \ | ||
| mkdir -p .python/lib; \ | ||
| cp -a "${PYTHON_PREFIX}/lib/python${PY_VER}" ".python/lib/python${PY_VER}"; \ | ||
| rm -rf ".python/lib/python${PY_VER}/test" \ | ||
| ".python/lib/python${PY_VER}/tests" \ | ||
| ".python/lib/python${PY_VER}/idle_test" \ | ||
| ".python/lib/python${PY_VER}/idlelib"; \ | ||
| # --- copy shared libraries (libpython) --------------------------------------- \ | ||
| for lib in "${PYTHON_PREFIX}"/lib/libpython*.so*; do \ | ||
| [ -e "$lib" ] && cp -a "$lib" .python/lib/; \ | ||
| done; \ | ||
| # --- repoint venv at the bundled Python -------------------------------------- \ | ||
| for f in .venv/bin/python*; do \ | ||
| [ -L "$f" ] || continue; \ | ||
| name=$(basename "$f"); \ | ||
| rm "$f"; \ | ||
| ln -s "../../.python/bin/${name}" "$f"; \ | ||
| done; \ | ||
| # Ensure canonical names resolve (some venvs only create python3 + python) \ | ||
| [ -L .venv/bin/python ] || ln -s "../../.python/bin/python" .venv/bin/python; \ | ||
| [ -L .venv/bin/python3 ] || ln -s "../../.python/bin/python3" .venv/bin/python3; \ | ||
| sed -i "s|^home = .*|home = /agent-server/.python/bin|" .venv/pyvenv.cfg; \ | ||
| # --- quick smoke-test inside the builder ------------------------------------- \ | ||
| .venv/bin/python -c "import sys; print('bundled python:', sys.executable, sys.version)" | ||
|
|
||
| #################################################################################### | ||
| # Binary Builder (binary mode) | ||
| # We run pyinstaller here to produce openhands-agent-server | ||
|
|
@@ -272,13 +323,32 @@ EXPOSE ${PORT} ${NOVNC_PORT} | |
| FROM base-image AS source | ||
| ARG USERNAME | ||
| COPY --chown=${USERNAME}:${USERNAME} --from=builder /agent-server /agent-server | ||
| # Bundled Python's libpython*.so lives under /agent-server/.python/lib | ||
| ENV LD_LIBRARY_PATH=/agent-server/.python/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟢 Acceptable: The Potential consideration: If a runtime image has conflicting Python libraries in its LD_LIBRARY_PATH, prepending should resolve it. If issues arise, may need |
||
| ENTRYPOINT ["/agent-server/.venv/bin/python", "-m", "openhands.agent_server"] | ||
|
|
||
| FROM base-image-minimal AS source-minimal | ||
| ARG USERNAME | ||
| COPY --chown=${USERNAME}:${USERNAME} --from=builder /agent-server /agent-server | ||
| ENV LD_LIBRARY_PATH=/agent-server/.python/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} | ||
| ENTRYPOINT ["/agent-server/.venv/bin/python", "-m", "openhands.agent_server"] | ||
|
|
||
| ############################ | ||
| # Target A.1: portability-test | ||
| # Verifies that /agent-server stays runnable when copied onto a base image | ||
| # with no Python installed, matching the benchmarks repo assembly pattern. | ||
| ############################ | ||
| FROM debian:bookworm-slim AS portability-test | ||
| RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
| COPY --from=builder /agent-server /agent-server | ||
| ENV LD_LIBRARY_PATH=/agent-server/.python/lib | ||
| RUN set -eux; \ | ||
| test -x /agent-server/.python/bin/python3; \ | ||
| test -f /agent-server/.venv/pyvenv.cfg; \ | ||
| /agent-server/.venv/bin/python -c "import sys; print('OK:', sys.executable, sys.version)"; \ | ||
| /agent-server/.venv/bin/python -c "import openhands.agent_server; print('agent_server importable')" | ||
|
|
||
| ############################ | ||
| # Target B: binary-runtime | ||
| # Production mode: build the binary inside Docker and copy it in. | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.