Skip to content

RHOAIENG-10351: Split JupyterLab installation out of the user's venv, so that there's fewer dependencies that need to be aligned #728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

RHOAIENG-10351: Split JupyterLab installation out of the user's venv, so that there's fewer dependencies that need to be aligned #728

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
54223e7
create poetry venv for jupyter
jiridanek Oct 4, 2024
255fae7
move jupyter packages into the new virtualenv and regenerate both loc…
jiridanek Oct 4, 2024
e07bba4
add jupyter kernel to our working venv
jiridanek Oct 4, 2024
d38aac8
ehm, everything else that needed doing
jiridanek Oct 4, 2024
133279c
forgot elyra binary
jiridanek Oct 4, 2024
b62056f
fixup, don't run jupyter as root
jiridanek Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions jupyter/datascience/ubi9-python-3.11/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,19 @@ LABEL name="odh-notebook-jupyter-datascience-ubi9-python-3.11" \
io.openshift.build.source-location="https://github.com/opendatahub-io/notebooks/tree/main/jupyter/datascience/ubi9-python-3.11" \
io.openshift.build.image="quay.io/opendatahub/workbench-images:jupyter-datascience-ubi9-python-3.11"

WORKDIR /opt/app-root/opt
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is the largest question right now; where to locate the other venv

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/opt/app-root/opt is poor choice

COPY requirements-elyra.txt ./
RUN echo "Installing elyra" && \
export VIRTUAL_ENV=/opt/app-root/opt && \
(source $VIRTUAL_ENV/bin/activate && pip install --require-hashes -r requirements-elyra.txt) && \
rm -f ./requirements-elyra.txt && \
# Fix permissions to support pip in Openshift environments \
chmod -R g+w /opt/app-root/opt/lib/python${PYTHON_VERSION}/site-packages && \
fix-permissions /opt/app-root -P
USER root
RUN ln -s /opt/app-root/opt/bin/jupyter-elyra /usr/local/bin/jupyter-elyra
USER default

WORKDIR /opt/app-root/bin

# Install Python packages and Jupyterlab extensions from Pipfile.lock
Expand All @@ -34,9 +47,6 @@ USER root
# Install usefull OS packages
RUN dnf install -y jq unixODBC postgresql git-lfs libsndfile && dnf clean all && rm -rf /var/cache/yum

# Disable announcement plugin of jupyterlab
RUN jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
Comment on lines -37 to -38
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're running this as root on main, that's a mistake


# Install MongoDB Client, We need a special repo for MongoDB as they do their own distribution
COPY mongodb-org-6.0.repo-x86_64 /etc/yum.repos.d/mongodb-org-6.0.repo

Expand All @@ -52,17 +62,18 @@ ENV PATH="$PATH:/opt/mssql-tools18/bin"
# Other apps and tools installed as default user
USER 1001

# Disable announcement plugin of jupyterlab
RUN jupyter labextension disable "@jupyterlab/apputils-extension:announcements"

# setup path for runtime configuration
RUN mkdir /opt/app-root/runtimes && \
# Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y \
sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json && \
# Remove default Elyra runtime-images \
rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json && \
rm /opt/app-root/opt/share/jupyter/metadata/runtime-images/*.json && \
# Fix permissions to support pip in Openshift environments \
Comment on lines 70 to 72
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

rm *.json fails the build when the glob is empty

rm /opt/app-root/opt/share/jupyter/metadata/runtime-images/*.json exits with status 1 if no files match, aborting the RUN layer. Make the command tolerant:

-rm /opt/app-root/opt/share/jupyter/metadata/runtime-images/*.json && \
+rm -f /opt/app-root/opt/share/jupyter/metadata/runtime-images/*.json || true && \
🤖 Prompt for AI Agents 
In jupyter/datascience/ubi9-python-3.11/Dockerfile around lines 70 to 72, the rm
command fails if no JSON files exist, causing the build to abort. Modify the rm
command to avoid failure when no files match by adding the -f flag or using a
shell pattern that does not error on empty matches, ensuring the build continues
smoothly even if no files are present.

chmod -R g+w /opt/app-root/lib/python3.11/site-packages && \
fix-permissions /opt/app-root -P

# Copy Elyra runtime-images definitions and set the version
COPY runtime-images/ /opt/app-root/share/jupyter/metadata/runtime-images/
COPY runtime-images/ /opt/app-root/opt/share/jupyter/metadata/runtime-images/

WORKDIR /opt/app-root/src
18 changes: 3 additions & 15 deletions jupyter/datascience/ubi9-python-3.11/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,9 @@ psycopg = "~=3.2.1"
pyodbc = "~=5.1.0"
mysql-connector-python = "~=9.0.0"

# JupyterLab packages

odh-elyra = "==4.0.0"

jupyterlab = "~=4.2.4"
jupyter-bokeh = "~=4.0.5"
jupyter-server = "~=2.14.2"
jupyter-server-proxy = "~=4.3.0"
jupyter-server-terminals = "~=0.5.3"
jupyterlab-git = "~=0.50.1"
jupyterlab-lsp = "~=5.1.0"
jupyterlab-widgets = "~=3.0.13"
jupyter-resource-usage = "~=1.1.0"
nbdime = "~=4.0.1"
nbgitpuller = "~=1.2.1"
# Jupyter kernel
# c.f. https://docs.jupyter.org/en/latest/install/kernels.html#how-do-i-install-python-2-and-python-3
ipykernel = "~=6.29.5"

Comment on lines +29 to 32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Same jupyter-client issue as minimal image

Add an 8.x upper-bound to avoid unvetted protocol changes:

 ipykernel = "~=6.29.5"
+jupyter-client = ">=8,<9"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Jupyter kernel
# c.f. https://docs.jupyter.org/en/latest/install/kernels.html#how-do-i-install-python-2-and-python-3
ipykernel = "~=6.29.5"
# Jupyter kernel
# c.f. https://docs.jupyter.org/en/latest/install/kernels.html#how-do-i-install-python-2-and-python-3
ipykernel = "~=6.29.5"
jupyter-client = ">=8,<9"
🤖 Prompt for AI Agents 
In jupyter/datascience/ubi9-python-3.11/Pipfile around lines 29 to 32, the
ipykernel dependency lacks an upper bound on jupyter-client, which can lead to
issues due to unvetted protocol changes. Modify the ipykernel dependency to
include an upper bound of 8.x for jupyter-client by specifying a compatible
version range that prevents upgrading beyond 8.x.

# Base packages
wheel = "~=0.44.0"
Expand Down
Loading
Loading