[CTM-142] New terra base image with sudo

Author: lucymcnatt

README.md

@@ -5,7 +5,7 @@ This repo provides docker images for running jupyter notebook in [Terra](https:/
 Make sure to go through the [contributing guide](https://github.com/DataBiosphere/terra-docker/blob/master/CONTRIBUTING.md#contributing) as you make changes to this repo.
 
 # Terra Base Images
-[terra-jupyter-base](terra-jupyter-base/README.md)
+[terra-base](terra-base/README.md)
 
 [terra-jupyter-python](terra-jupyter-python/README.md)
 
@@ -20,15 +20,18 @@ Make sure to go through the [contributing guide](https://github.com/DataBiospher
 # How to create your own Custom image to use with notebooks on Terra
 Custom docker images need to use a Terra base image (see above) in order to work with the service that runs notebooks on Terra.
 * You can use any of the base images above
+  * `terra-base` is the smallest image, but doesn't include any scientific packages on top of Jupyter and R
 * Here is an example of how to build off of a base image: Add `FROM us.gcr.io/broad-dsp-gcr-public/terra-jupyter-base:0.0.1` to your dockerfile (`terra-jupyter-base` is the smallest image you can extend from)
 * Customize your image (see the [terra-jupyter-python](terra-jupyter-python/Dockerfile) dockerfile for an example of how to extend from one of our base images
-* Publish the image to either GCR or Dockerhub; the image must be public to be used
+* Publish the image to either GAR or Dockerhub;
+  * If using Dockerhub, the image **must be public** to be used
 * Use the published container image location when creating notebook runtime
 * Dockerhub image example: [image name]:[tag]
-* GCR image example: us.gcr.io/repository/[image name]:[tag]
-* Since 6/28/2021, we introduced a few changes that might impact building custom images
-    - Home directory of new images will be `/home/jupyter`. This means if your dockerfile is referencing `/home/jupyter-user` directory, you need to update it to $HOME (recommended) or `/home/jupyter`.
-    - Creating VMs with custom images will take much longer than terra supported images because `docker pull` will take a few min. If the custom image ends up being too large, VM creation may time out. New base images are much larger in size than previous versions.
+* GAR image example: us.gcr.io/repository/[image name]:[tag]
+* Some things to keep in mind when creating custom images:
+  - The home directory of new images will be `/home/jupyter`. This means if your dockerfile is referencing the `/home/jupyter-user` directory, you need to update it to $HOME (recommended) or `/home/jupyter`.
+  - Creating VMs with custom images may take longer than terra supported images because `docker pull` will take a few min. If the custom image ends up being too large, VM creation may time out.
+-
 
 # Development
 ## Using git secrets
@@ -56,25 +59,25 @@ Once you have the container running, you should be able to access jupyter at htt
 Detailed documentation on how to integrate the terra-docker image with Leonardo can be found [here](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo)
 
 ### If you are adding a new image:
-- Create a new directory with the Dockerfile and a CHANGELOG.md. 
+- Create a new directory with the Dockerfile and a CHANGELOG.md.
 - Add the directory name (also referred to as the image name) as an entry to the image_data array in the file in config/conf.json. For more info on what is needed for a new image, see the section on the config
 - If you wish the image to be baked into our custom image, which makes the runtime load significantly faster (recommended), make a PR into the leonardo [repo](https://github.com/DataBiosphere/leonardo) doing the following within the `jenkins` folder:
-    - Add the image to the parameter list in the Jenkinsfile
-    - Update the relevant `prepare` script in each subdirectory. Currently there is a prepare script for gce and dataproc.
-    - It is recommended to add a test in the `automation` directory (`automation/src/test/resources/reference.conf`)
-    - Add your image to the `reference.conf` in the automation directory. This will be the only place any future version updates to your image happen. This ensures, along with the test in the previous step, that any changes to the image are tested.
-    - Run the GHA to generate the image, and add it to `reference.conf` in the http directory (`http/src/main/resources/reference.conf`)
+  - Add the image to the parameter list in the Jenkinsfile
+  - Update the relevant `prepare` script in each subdirectory. Currently there is a prepare script for gce and dataproc.
+  - It is recommended to add a test in the `automation` directory (`automation/src/test/resources/reference.conf`)
+  - Add your image to the `reference.conf` in the automation directory. This will be the only place any future version updates to your image happen. This ensures, along with the test in the previous step, that any changes to the image are tested.
+  - Run the GHA to generate the image, and add it to `reference.conf` in the http directory (`http/src/main/resources/reference.conf`)
 
 ### If you are updating an existing image:
 - [Create your terra-docker PR](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo#1.-Create-a-terra-docker-PR)
-    - Update the version in config/conf.json
-    - Update CHANGELOG.md and VERSION file
-    - Ensure that no `From` statements need to be updated based on the image you updated (i.e., if you update the base image, you will need to update several other images)
-    - Run updateVersions.sc to bump all images dependent on the base
+  - Update the version in config/conf.json
+  - Update CHANGELOG.md and VERSION file
+  - Ensure that no `From` statements need to be updated based on the image you updated (i.e., if you update the base image, you will need to update several other images)
+  - Run updateVersions.sc to bump all images dependent on the base
 - [Merge your terra-docker PR and check if the image(s) and version json files are created](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo#2.-Merge-your-terra-docker-PR-and-check-images-are-created)
 - [Open a PR in leonardo](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo#3.-Create-a-new-leo-PR-that-integrates-the-new-images)
-    - Update the relevant `prepare` script within the `jenkins` folder
-    - Update the automation `reference.conf` file
+  - Update the relevant `prepare` script within the `jenkins` folder
+  - Update the automation `reference.conf` file
 - [Run the GHA on your branch to generate the new image](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo#4.-Run-the-Github-Action-in-leo-to-generate-a-new-custom-COS-image)
 - [Update the leonardo PR to use the newly generated image](https://broadworkbench.atlassian.net/wiki/spaces/IA/pages/2519564289/Integrating+new+Terra+docker+images+with+Leonardo#5.-Update-the-Leo-PR-to-use-the-generated-OS-images)
 - Ensure that the `terra-docker-versions-candidate.json` file (which is what the UI sources the dropdown from) in the `terra-docjker-image-documentation-[env]` bucket correclty references your new docker image
@@ -120,9 +123,9 @@ To launch an image through Terra, navigate to https://app.terra.bio or your BEE'
 
 ## Config
 
-There is a config file located at `config/conf.json` that contains the configuration used by all automated jobs and build scripts that interface with this repo. 
+There is a config file located at `config/conf.json` that contains the configuration used by all automated jobs and build scripts that interface with this repo.
 
-There is a field for "spark_version" top-level which must be updated if we update the debian version used in the custom image. 
+There is a field for "spark_version" top-level which must be updated if we update the debian version used in the custom image.
 Currently it assumes 1.4x https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-1.4
 
 There are some constants included, such as the tools supported by this repo. Of particular interest is the image_data array.
@@ -163,7 +166,7 @@ Each time you update or add an image, you will need to update the appropriate en
 
 The scripts folder has scripts used for building.
 - `generate_package_docs.py` This script is run once by build.sh each time an image is built. It is used to generate a .json with the versions for the packages in the image.
-- `generate_version_docs.py` This script is run each time an image is built. It builds a new file master version file for the UI to look up the current versions to reference. 
+- `generate_version_docs.py` This script is run each time an image is built. It builds a new file master version file for the UI to look up the current versions to reference.
 
 ## Image dependencies
 

terra-base/Dockerfile

@@ -0,0 +1,248 @@
+# Latest gpu-enabled base image on Ubuntu 22, 132 MB compressed
+FROM --platform=linux/amd64 nvidia/cuda:13.0.1-base-ubuntu22.04
+
+LABEL maintainer="DSP Analysis Team <dsp-analysis@broadinstitute.org>"
+
+# TODO:
+# ! try moving conda setup before UV setup
+# ! specify that UV uses the conda python
+# add more comments/clearer naming around the python versions being used
+# double check that the correct CUDA/GPU drivers are available, otherwise copy from old base image via a multi-stage build
+# look into copying over apt-get packages from the old terra base image in next level of images
+# have examples of custom images extending this base image for different use cases (e.g. R, GATK, etc)
+
+# want the command to fail due to an error at any stage in the pipe: https://github.com/hadolint/hadolint/wiki/DL4006
+SHELL ["/usr/bin/bash", "-o", "pipefail", "-c"]
+
+#######################
+# General Environment Variables
+#######################
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LC_ALL=en_US.UTF-8
+
+# Version of python to be installed and used
+ENV PYTHON_VERSION=3.10
+# Paired conda installer
+ENV CONDA_INSTALLER=https://repo.anaconda.com/miniconda/Miniconda3-py310_25.9.1-1-Linux-x86_64.sh
+ENV JUPYTER_VERSION=5.7.2
+ENV NODE_MAJOR=20
+
+#################
+# Install Prerequisites
+#################
+RUN apt-get update && apt-get install -yq --no-install-recommends \
+    # basic necessities
+    sudo \
+    ca-certificates \
+    curl \
+    jq \
+    # gnupg requirement
+    gnupg \
+    dirmngr \
+    # useful utilities for debugging within docker itself \
+    nano \
+    less \
+    procps \
+    lsb-release \
+    # gcc compiler
+    build-essential \
+    locales \
+    # for ssh-agent and ssh-add
+    keychain \
+    # extras \
+    wget \
+    bzip2 \
+    git \
+    # Uncomment en_US.UTF-8 for inclusion in generation
+    && sed -i 's/^# *\(en_US.UTF-8\)/\1/' /etc/locale.gen \
+    # Generate locale
+    && locale-gen \
+    # cleanup
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+##############################
+# Set up Node for Jupyterlab
+##############################
+RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
+RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
+
+# Install Node >18
+RUN apt-get update && apt-get install -yq --no-install-recommends
+RUN mkdir -p /etc/apt/keyrings
+RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
+
+RUN echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
+RUN dpkg --remove --force-remove-reinstreq libnode-dev
+RUN apt-get update && apt-get install -f -yq nodejs
+
+##########################
+# Create the User's Jupyter User
+##########################
+# Create the jupyter user and give sudo permission
+ENV USER=jupyter
+# This UID must stay static to be one greater than the welder user (1001)
+ENV USER_UID=1002
+
+# Create the user home and add to the users group
+ENV USER_HOME=/home/$USER
+RUN useradd -m -s /bin/bash -d $USER_HOME -N -u $USER_UID  $USER
+RUN usermod -g users $USER
+
+# We want to grant the user sudo permissions without password
+ # so they can install the necessary packages that they want to use on the docker container
+RUN echo "$USER ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/$USER \
+    && chmod 0440 /etc/sudoers.d/$USER
+
+############
+# Install R
+############
+# add R repo for later R package installation \
+RUN wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc \
+    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 \
+    && apt-get update && apt-get install -yq --no-install-recommends software-properties-common \
+    && add-apt-repository --no-update "deb https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/" \
+    # Install R base
+    && apt-get update && apt-get install -y r-base \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+############################################
+# Install Miniconda and setup base conda environment
+############################################
+## CONDA should not be used by devs to manage package dependencies,
+# but is a widely used tool to manage python environments in a runtime
+ # and we should provide it to users
+
+# download and install conda
+ENV CONDA_HOME=/opt/conda
+
+RUN curl -so $HOME/miniconda.sh $CONDA_INSTALLER \
+    && chmod +x $HOME/miniconda.sh \
+    && $HOME/miniconda.sh -b -p $CONDA_HOME \
+    && rm $HOME/miniconda.sh
+# ENV PATH="${PATH}:${CONDA_ENV_HOME}/bin:${CONDA_HOME}/bin"
+ENV PATH="${PATH}:${CONDA_HOME}/bin"
+
+# In order to override the default kernel, the conda environment must be named 'python3'
+ENV CONDA_ENV_NAME=python3
+ENV CONDA_ENV_HOME=$USER_HOME/.envs/$CONDA_ENV_NAME
+ENV CONDA_FILES=$CONDA_HOME/custom
+
+# Copy over the conda environment files
+COPY conda/ $CONDA_FILES
+ENV CONDA_FILES=$CONDA_HOME/custom
+
+# Set up the path to the user python from conda
+ENV BASE_PYTHON_PATH=$CONDA_HOME/bin/python$PYTHON_VERSION
+# Tell conda to NOT write bite code (aka these.pyc files)
+ENV PYTHONDONTWRITEBYTECODE=true
+
+# Have to accept the conda terms of service to be able to install packages
+RUN conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main \
+ && conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
+
+#RUN conda init bash \
+#    && . ~/.bashrc \
+RUN conda env create --prefix $CONDA_ENV_HOME --file $CONDA_FILES/conda-environment.yaml \
+    # Remove packages tarballs and python bytecode files from the image
+    && conda clean -afy \
+    && rm $CONDA_FILES/conda-environment.yaml \
+    # Make sure the USER is the owner of the folder where the base conda is installed
+    && chown -R $USER:users $USER_HOME
+
+# Create the base conda environment that will be used by the jupyter user
+RUN conda run -p ${CONDA_ENV_HOME} python -m ipykernel install --name=$CONDA_ENV_NAME
+
+
+##############################
+# Setup UV Environment for Jupyter
+##############################
+# Using UV (Universal Virtualenv) to create a virtual environment
+# UV is used in place of poetry for speed and simplicity.
+
+# NOTE: this is for the environment the jupyter server will be running in,
+# separate from the jupyter user's conda environment.
+COPY uv.lock .
+COPY pyproject.toml .
+
+# Setup UV environment variables
+# - tells uv to copy the Python files into the container from the cache mount,
+# - tell uv to byte-compile packages for faster application startups,
+# - don't seed venv, we need to install separately
+# - don't cache to keep the image size small
+ENV UV_LINK_MODE=copy \
+    UV_COMPILE_BYTECODE=1 \
+    UV_VENV_SEED=false \
+    UV_NO_CACHE=true
+
+# Download the latest installer
+ADD https://astral.sh/uv/install.sh /uv-installer.sh
+RUN sh /uv-installer.sh && rm /uv-installer.sh
+
+# Add local bin to PATH for uv
+ENV PATH="/root/.local/bin/:$PATH"
+
+ENV JUPYTER_HOME=/etc/jupyter
+
+# Create a virtual environment and install jupyter packages
+# setuptools and wheel are required for some of the jupyter extensions
+RUN uv venv $JUPYTER_HOME --python $BASE_PYTHON_PATH \
+    && source $JUPYTER_HOME/bin/activate \
+    && uv pip install wheel \
+    && uv pip install setuptools \
+     && uv pip install -r pyproject.toml  --no-cache --no-build-isolation \
+    # Cleanup
+     && rm uv.lock && rm pyproject.toml
+
+# add jupyter to path
+ENV PATH="${PATH}:${JUPYTER_HOME}/bin"
+
+# Remove default jupyter kernel (to force use of the conda python kernel)
+RUN $JUPYTER_HOME/bin/jupyter kernelspec remove python3 -y
+
+# #######################
+# # Terra-specific Utilities
+# #######################
+# copy over jupyter extensions and kernel config files
+COPY jupyter/ $JUPYTER_HOME
+
+# give user ownership of jupyter home and conda env home
+# and make extension files executable
+RUN chown -R $USER:users $JUPYTER_HOME $CONDA_HOME  \
+&& find $JUPYTER_HOME/extensions/scripts -name '*.sh' -type f | xargs chmod +x
+
+# make the run-jupyter script executable
+RUN chmod +x -R $JUPYTER_HOME/run-jupyter.sh
+
+# Setup jupyter and r kernels
+ENV JUPYTER_KERNELSPEC_DIR=/usr/local/share/jupyter/
+RUN chown -R $USER:users $JUPYTER_KERNELSPEC_DIR \
+    && find $JUPYTER_HOME/kernel -name '*.sh' -type f | xargs chmod +x \
+    # You can get kernel directory by running `jupyter kernelspec list`
+    && $JUPYTER_HOME/kernel/kernelspec.sh $JUPYTER_HOME/kernel $JUPYTER_KERNELSPEC_DIR/kernels
+
+# Create the welder user
+# The welder uid is consistent with the Welder docker definition here:
+#  https://github.com/DataBiosphere/welder/blob/master/project/Settings.scala
+# Adding welder-user to the Jupyter container isn't strictly required, but it makes welder-added
+# files display nicer when viewed in a terminal.
+ENV WELDER_USER welder-user
+# This UID must stay consistent with the UID defined in Welder
+ENV WELDER_UID 1001
+RUN useradd -m -s /bin/bash -N -u $WELDER_UID $WELDER_USER
+
+# Make sure that the jupyter user will have access to the jupyter path in the working directory
+EXPOSE $JUPYTER_PORT
+WORKDIR $USER_HOME
+
+# make pip install to a user directory, instead of a system directory which requires root.
+# this is useful so `pip install` commands can be run in the context of a notebook.
+ENV PIP_USER=true
+USER $USER
+
+# Note: this entrypoint is provided for running Jupyter independently of Leonardo.
+# When Leonardo deploys this image onto a cluster, the entrypoint is overwritten to enable
+# additional setup inside the container before execution.  Jupyter execution occurs when the
+# init-actions.sh script uses 'docker exec' to call run-jupyter.sh.
+ENTRYPOINT ["/etc/jupyter/bin/jupyter", "notebook"]