Merge pull request #278 from nmfs-opensci/test

create script for geospatial r install; fix vscode extensions

Author: eeholmes

Dockerfile

@@ -5,7 +5,7 @@ LABEL org.opencontainers.image.author="eli.holmes@noaa.gov"
 LABEL org.opencontainers.image.source=https://github.com/nmfs-opensci/py-rocket-base
 LABEL org.opencontainers.image.description="Python (3.12), R (4.4.3), Desktop and Publishing tools"
 LABEL org.opencontainers.image.licenses=Apache2.0
-LABEL org.opencontainers.image.version=2025.04.17
+LABEL org.opencontainers.image.version=2025.04.26
 
 USER root
 
@@ -60,7 +60,7 @@ RUN wget -qO- https://raw.githubusercontent.com/retorquere/zotero-deb/master/ins
 RUN /pyrocket_scripts/install-apt-packages.sh ${REPO_DIR}/apt.txt
 
 # Install some basic VS Code extensions
-RUN /pyrocket_scripts/install-vscode-extensions.sh ${REPO_DIR}/vscode-extensions.txt
+RUN /pyrocket_scripts/install-vscode-extensions.sh ${REPO_DIR}/vscode-extensions.txt || (echo "install-vscode-extensions.sh failed for some extensions" >&2 && exit 1)
 
 # Re-enable man pages disabled in Ubuntu 18 minimal image
 # https://wiki.ubuntu.com/Minimal

README.md

@@ -1,6 +1,6 @@
 # py-rocket-base: JupyterHub base image
 
-[![Build and push container image](https://github.com/nmfs-opensci/py-rocket-base/actions/workflows/build.yaml/badge.svg)](https://github.com/nmfs-opensci/py-rocket-base/actions/workflows/build.yaml)[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.13942617.svg)](https://doi.org/10.5281/zenodo.13942617)[![GitHub Release](https://img.shields.io/github/v/release/nmfs-opensci/py-rocket-base)](https://github.com/nmfs-opensci/py-rocket-base/releases)
+[![Build and push container image](https://github.com/nmfs-opensci/py-rocket-base/actions/workflows/build.yaml/badge.svg)](https://github.com/nmfs-opensci/py-rocket-base/actions/workflows/build.yaml)   [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.13942617.svg)](https://doi.org/10.5281/zenodo.13942617)   [![GitHub Release](https://img.shields.io/github/v/release/nmfs-opensci/py-rocket-base)](https://github.com/nmfs-opensci/py-rocket-base/releases)
 
 The py-rocket-base image is a base image for the JupyterHubs with Python and RStudio. The py-rocket-base image uses the [Pangeo base-image](https://github.com/pangeo-data/pangeo-docker-images/tree/master/base-image) (minus the `ONBUILD` commands) as the base image (stored in `py-rocket-base/base-image`) and the [pangeo-notebook metapackage](https://github.com/conda-forge/pangeo-notebook-feedstock/blob/main/recipe/meta.yaml) to setup the core JupyterHub environment (following [Pangeo base-notebook](https://github.com/pangeo-data/pangeo-docker-images/blob/master/base-notebook/environment.yml). [Additional Python packages](https://github.com/nmfs-opensci/py-rocket-base/blob/main/environment.yml) are installed to provide a fuller JupyterLab, RStudio, Desktop, and VSCode environment.  The R environment is installed with [Rocker](https://rocker-project.org/) installation scripts. You can scroll through the Rocker [installation scripts](https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_rstudio.sh) to see how that environment is set up.
 
@@ -14,9 +14,12 @@ The base image has the following support:
 * Python + JupyterLab with mamba handling package installation
 * R + RStudio with the rocker scripts and other functions available
 * Desktop ready for installing applications, including Java-based applications
-* VSCode
+* VSCode + R and Python extensions pre-installed
+* Publishing infrastructure installed (TexLive, Quarto, JupyterBook, MyST, pandoc).
 
-*There are many ways to install R and RStudio into an image designed for JupyterHubs.* The objective of py-rocket-base is to create a JupyterHub (or binder) image such when you click the RStudio button in the JupyterLab UI to enter the RStudio UI (`/rstudio`), you **enter an environment that is the same as if you had used a Rocker image** but if you are in the JupyterLab UI (`/lab`), the **environment is the same as if you had used repo2docker** to create the environment. There are many other ways to install R and RStudio in a JupyterHub image. See History below for other approaches we have used. 
+py-rocket-base is the base image for the NMFS OpenSci specialized images, specifically [py-rocket-geospatial](https://nmfs-opensci.github.io/container-images/).
+
+*There are many ways to install R and RStudio into an image designed for JupyterHubs.* The objective of py-rocket-base is to create a JupyterHub (or binder) image such when you click the RStudio button in the JupyterLab UI to enter the RStudio UI (`/rstudio`), you **enter an environment that is the same as if you had used a Rocker image** but if you are in the JupyterLab UI (`/lab`), the **environment is the same as if you had used the Pangeo base image** to create the environment. See History below for other approaches we have used to create py-rocket over the years (and why this current approach is used). 
 
 ## Using this in a JupyterHub
 
@@ -34,9 +37,9 @@ py-rocket-base has pyrocket and rocket scripts that you can use to help customiz
 
 ## History and motivation
 
-The original [py-rocket 1.0](https://github.com/NASA-Openscapes/py-rocket) was developed by Luis Lopez and was built off a Rocker base image. Carl Boettiger and Eli Holmes later altered the image (py-rocket 2.0) so that the Python environment matched the Pangeo image structure but the image was still built off a Rocker image. Subsequently, Carl Boettiger developed [repo2docker-r](https://github.com/boettiger-lab/repo2docker-r) that creates a JupyterHub-compatible image that uses a [Jupyter docker stack image](https://jupyter-docker-stacks.readthedocs.io/en/latest/) as base. For py-rocker 3.0, Eli Holmes used Carl's ideas but used [repo2docker](https://repo2docker.readthedocs.io/en/latest/) and [repo2docker-action](https://github.com/jupyterhub/repo2docker-action) to build the base image. To do this, the [CryoCloud hub image](https://github.com/CryoInTheCloud/hub-image) repo was used for the basic structure and approach. Eli added the `rocker.sh` script and `appendix` modifications to install R and RStudio via the Rocker scripts (rather than using a Rocker image as base). Yuvi Panda (repo2docker) gave input throughout the process as snags were hit.
+The original [py-rocket 1.0](https://github.com/NASA-Openscapes/py-rocket) was developed by Luis Lopez and was built off a Rocker base image. Carl Boettiger and Eli Holmes later altered the image (py-rocket 2.0) so that the Python environment matched the Pangeo image structure but the image was still built off a Rocker image. Subsequently, Carl Boettiger developed [repo2docker-r](https://github.com/boettiger-lab/repo2docker-r) that creates a JupyterHub-compatible image that uses a [Jupyter docker stack image](https://jupyter-docker-stacks.readthedocs.io/en/latest/) as base. For py-rocker 3.0, Eli Holmes used Carl's ideas but used [repo2docker](https://repo2docker.readthedocs.io/en/latest/) and [repo2docker-action](https://github.com/jupyterhub/repo2docker-action) to build the base image. To do this, the [CryoCloud hub image](https://github.com/CryoInTheCloud/hub-image) repo was used for the basic structure and approach. Eli added the `rocker.sh` script and `appendix` modifications to install R and RStudio via the Rocker scripts (rather than using a Rocker image as base). Yuvi Panda (repo2docker) gave input throughout the process as snags were hit. For py-rocker 4.0, current approach, repo2docker was abandoned and the base image was created by using the Pangeo base image with the `ONBUILD` parts removed. This approach was taken after discussions with Docker image mainainers at NASA who had quickly run into the need for more tailored control of the build process when customizing images. And in fact, the need for customizing the build process became an issue quickly and I was resorting to many "hacks" to circumvent repo2docker default behavior. Given the close relationship between Pangeo base image and repo2docker developers, using the Pangeo base image Docker file still results in a base image that is very similar to that created with repo2docker.
 
-**Why Rocker for the R/RStudio environment?** The Rocker images are the standard for R/RStudio contanier images. They are heavily tested and regularly updated. There is a large developer community that fixes problems and bugs. The stack has gone through major revisions to improve modularity and they constantly innovating (integration for machine-learning, CUDA, BLAS, spatial, etc., etc.). py-rocket is building off that work without using the images directly. Instead it uses the Docker file code and the installation scripts.  There are many other approaches to adding R and RStudio to images that work in JupyterHubs. See [repo2docker-r](https://github.com/boettiger-lab/repo2docker-r) that Carl developed and [r-conda](https://github.com/binder-examples/r-conda) for a conda native approach using repo2docker. py-rocket is not intended to create small images; it is intended to create images that emulate Rocker in the `/rstudio` environment on a JupyterHub.
+**Why Rocker for the R/RStudio environment?** The Rocker images are the standard for R/RStudio contanier images. They are heavily tested and regularly updated. There is a large developer community that fixes problems and bugs. The stack has gone through major revisions to improve modularity and they constantly innovating (integration for machine-learning, CUDA, BLAS, spatial, etc., etc.). py-rocket is building off that work without using the images directly. Instead it uses the Docker file code and the installation scripts.  There are many other approaches to adding R and RStudio to images that work in JupyterHubs. See [repo2docker-r](https://github.com/boettiger-lab/repo2docker-r) that Carl developed and [r-conda](https://github.com/binder-examples/r-conda) for a conda native approach using repo2docker. py-rocket is not intended to create small images; it is intended to create images that emulate Rocker (verse and geospatial) in the R environments (whether in Jupyter Lab, RStudio or VSCode) on a JupyterHub.
 
 ## Building the documentation
 

book/customizing.qmd

@@ -11,6 +11,7 @@ How to use the helper scripts is shown in [configuration files](configuration_fi
 * `install-conda-packages.sh`
 * `install-pip-packages.sh`
 * `install-r-packages.sh`
+* `install-geospatial-r.sh`
 * `install-apt-packages.sh`
 * `install-vscode-extensions.sh`
 * `install-desktop.sh`
@@ -33,12 +34,23 @@ RUN /pyrocket_scripts/install-conda-packages.sh environment.yml && \
   rm environment.yml
 ```
 
-Rocker scripts do not take arguments. Note that PATH must be given since rocker installation scripts will fail with conda on the path. The path specification will apply only within the specific RUN context and must be in the same RUN that the rocker script is run. The scripts should be run as root.
+If you have multiple files in the repo:
+```markdown
+COPY . /tmp2/
+RUN /pyrocket_scripts/install-conda-packages.sh /tmp2/environment.yml || echo "install-conda-packages.sh failed" || true
+RUN /pyrocket_scripts/install-r-packages.sh /tmp2/install.R || echo "install-r-package.sh failed" || true
+RUN /pyrocket_scripts/install-apt-packages.sh /tmp2/apt.txt || echo "install-apt-packages.sh failed" || true
+RUN /pyrocket_scripts/install-desktop.sh /tmp2/Desktop|| echo "setup-desktop.sh failed" || true
+RUN /pyrocket_scripts/install-geospatial-r.sh || echo "setup-desktop.sh failed" || true
+RUN rm -rf /tmp2
+```
+
+Rocker scripts do not take arguments. Note that PATH must be given since rocker installation scripts will fail with conda on the path. The path specification will apply only within the specific RUN context and must be in the same RUN that the rocker script is run. The scripts should be run as root. Getting R packages to install to the system library, as opposed to /home which will be removed in a Jupyter Hub, can be tricky. See the `/pyrocket_scripts/install-geospatial-r.sh` script for an example.
 
 ```markdown
 USER root
 RUN PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin && \
-    /rocker_scripts/install_geospatial.sh
+    /rocker_scripts/install_texlive.sh
 USER ${NB_USER}
 ```
 

extensions/README.md

@@ -0,0 +1 @@
+Download extensions here as a backup if open-vsx.org is down

extensions/quarto-1.120.0.vsix

@@ -0,0 +1,20 @@
+#!/bin/bash
+set -e
+
+# Ensure script is being run as root
+if [ "$(id -u)" -ne 0 ]; then
+  echo "pyrocket_scripts/install-geospatial-r.sh script must be run as root." >&2
+  exit 1
+fi
+
+# Temporary R profile setup for system-wide library path
+echo '.libPaths("/usr/local/lib/R/site-library")' > /etc/littler.r
+echo '.libPaths("/usr/local/lib/R/site-library")' > /tmp/rprofile.site
+
+# Run rocker-provided geospatial install script with temporary profile
+R_PROFILE=/tmp/rprofile.site \
+PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
+/rocker_scripts/install_geospatial.sh
+
+# Cleanup
+rm -f /etc/littler.r /tmp/rprofile.site

scripts/install-geospatial-r.sh

@@ -142,6 +142,7 @@ mkdir -p "${R_LIBS_USER}" && chown ${NB_USER}:staff "${R_LIBS_USER}"
 echo 'if (!dir.exists(Sys.getenv("R_LIBS_USER"))) dir.create(Sys.getenv("R_LIBS_USER"), recursive = TRUE)' >> "$RPROFILE_SITE"
 echo '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()))' >> "$RPROFILE_SITE"
 
+# Install the R kernel for JupyterLab and VSCode so they point to the rocker installed R
 Rscript - <<EOF
 install.packages('IRkernel', lib = .Library) # install in system library
 Sys.setenv(PATH = paste("/srv/conda/envs/notebook/bin", Sys.getenv("PATH"), sep = ":"))
@@ -167,6 +168,9 @@ setHook(packageEvent("reticulate", "attach"), function(...) {
 })
 EOF
 
+# Install for VSCode. languageserver is required for VSCode. others make R nicer in VSCode and JLab.
+Rscript -e "install.packages(c('languageserver', 'httpgd', 'quarto'), lib = .Library)"
+
 # Ensure jovyan can modify Rprofile.site and Renviron.site because start will need this, and allow user to alter rserver.conf
 # to set the gh-scoped-cred variables if they are present
 chown ${NB_USER}:staff /etc/rstudio/rserver.conf

scripts/install-rocker.sh

@@ -1,8 +1,10 @@
 #!/bin/bash
-# Required User: NB_USER
+set -euo pipefail
+IFS=$'\n\t'
+# Required User: root
 
 # Install VSCode extensions. 
-# These get installed to $CONDA_PREFIXshare/code-server/extensions/
+# These get installed to ${NB_PYTHON_PREFIX}/share/code-server/extensions/
 
 # Check if a filename argument is provided
 if [ -z "$1" ]; then
@@ -11,7 +13,7 @@ if [ -z "$1" ]; then
     exit 1
 fi
 
-# Check if the script is run as root; folders will be made in /home and there are issues with this if user is jovyan
+# Check if the script is run as root; 
 # Since some prior installs might have created .local as root. Easier just to install vscode extensions as root
 if [[ $(id -u) -ne 0 ]]; then
     echo "Error: install-vscode-extensions.sh must be run as root. Please use 'USER root' in your Dockerfile before running this script."
@@ -36,6 +38,7 @@ EXT_DIR="${NB_PYTHON_PREFIX}/share/code-server/extensions"
 install -o ${NB_USER} -g ${NB_USER} -m 755 -d "${EXT_DIR}"
 
 # Install each extension listed in the file; skip empty lines or comments
+FAILED=0
 while IFS= read -r EXT; do
     # Remove comments and leading/trailing whitespace
     EXT=$(echo "$EXT" | sed 's/#.*//; s/^[[:space:]]*//; s/[[:space:]]*$//')
@@ -47,5 +50,16 @@ while IFS= read -r EXT; do
         echo "  Successfully installed extension: $EXT"
     else
         echo "  Failed to install extension: $EXT" >&2
+        FAILED=1
     fi
 done < "$ext_file"
+
+# Fix ownership so user can install/uninstall extensions later
+echo "Fixing ownership of installed extensions..."
+chown -R ${NB_USER}:${NB_USER} "${EXT_DIR}"
+chmod -R u+rwX,go+rX "${EXT_DIR}"
+
+if [ "$FAILED" -ne 0 ]; then
+  echo "One or more VSCode extensions failed to install. Exiting." >&2
+  exit 1
+fi

scripts/install-vscode-extensions.sh

@@ -32,4 +32,25 @@ if [ -d "${REPO_DIR}/childstart/" ]; then
         fi
     done
 fi
+
+# --- Set default VSCode code-server settings if missing ---
+DEFAULT_SETTINGS_DIR="/usr/local/share/code-server/User"
+USER_SETTINGS_DIR="/home/jovyan/.local/share/code-server/User"
+
+# Ensure User settings directory exists
+mkdir -p "${USER_SETTINGS_DIR}"
+
+if [ ! -f "${USER_SETTINGS_DIR}/settings.json" ]; then
+    echo "Creating default VSCode settings.json..."
+    cat > "${USER_SETTINGS_DIR}/settings.json" <<'EOF'
+{
+  "remote.autoForwardPorts": true,
+  "remote.autoForwardPortsSource": "process",
+  "remote.restoreForwardedPorts": false
+}
+EOF
+    chown -R jovyan:jovyan "${USER_SETTINGS_DIR}"
+else
+    echo "User already has VSCode settings.json, not overwriting."
+fi
 exec "$@"

start

@@ -1,2 +1,4 @@
 ms-python.python
+ms-toolsai.jupyter
 quarto.quarto
+anwar.papyrus-pdf