jupyterhub-groups-exporter

An externally-managed JupyterHub service to export user group memberships to Prometheus.

Overview

The jupyterhub-groups-exporter project provides a service that integrates with Zero to JupyterHub documentation to export user group memberships as Prometheus metrics. This enables JupyterHub administrators to monitor usage and costs segmented by user groups using visualization tools like Grafana.

Features

• Exports user group memberships from JupyterHub to Prometheus.
• Supports integration with Kubernetes clusters via Helm charts.

Installation

This repository provides packaged Helm charts for the jupyterhub-groups-exporter project. These charts are hosted in the gh-pages branch and made available as a Helm chart repository.

Prerequisites

• Kubernetes cluster
• Helm installed (Helm documentation)

Steps

1. Add the Helm chart repository:

   helm repo add jupyterhub-groups-exporter https://2i2c.org/jupyterhub-groups-exporter
   helm repo update

2. Install the Helm chart:

   helm install jupyterhub-groups-exporter https://2i2c.org/jupyterhub-groups-exporter --version <version>

Usage

Once installed, the jupyterhub-groups-exporter service will interact with the JupyterHub API to retrieve user group memberships and expose them as Prometheus metrics. These metrics can then be queried with PromQL and visualized in Grafana dashboards.

Configuration

The exporter supports the following argument options:

• --port: Port to listen on for the groups exporter. Default is 9090.
• --update_exporter_interval: Time interval (in seconds) between each update of the JupyterHub groups exporter. Default is 3600.
• --allowed_groups: List of allowed user groups to be exported. If not provided, all groups will be exported.
• --default_group: Default group to account usage against for users with multiple group memberships. Default is "other".
• --hub_url: JupyterHub service URL, e.g., http://localhost:8000 for local development. Default is constructed using environment variables HUB_SERVICE_HOST and HUB_SERVICE_PORT.
• --api_token: Token to authenticate with the JupyterHub API. Default is fetched from the environment variable JUPYTERHUB_API_TOKEN.
• --jupyterhub_namespace: Kubernetes namespace where the JupyterHub is deployed. Default is fetched from the environment variable NAMESPACE.
• --jupyterhub_metrics_prefix: Prefix/namespace for the JupyterHub metrics for Prometheus. Default is "jupyterhub".
• --log_level: Logging level for the exporter service. Options are DEBUG, INFO, WARNING, ERROR, and CRITICAL. Default is "INFO".

Exporting user group memberships to Prometheus

The jupyterhub-groups-exporter project provides a service that integrates with JupyterHub to export user group memberships as Prometheus metrics. This component is readily deployable as part of any JupyterHub instance, such as a standalone deployment or a Zero to JupyterHub deployment on Kubernetes.

The exporter provides a Gauge metric called jupyterhub_user_group_info, which contain the following labels:

• namespace – the Kubernetes namespace where the JupyterHub is deployed
• usergroup – the name of the user group
• username – the unescaped username of the user
• username_escape – the escaped username

Escaped usernames are useful because Kubernetes pods have characterset limits for valid pod label names (this limit does not apply to pod annotations). Storing both types of usernames allows us to join escaped versions with their more human-readable unescaped usernames.

Exposing this metric as an endpoint for Prometheus to scrape allows us to query and join groups data with a range of usage metrics to gain powerful group-level insights. Here is an example PromQL query that retrieves the memory usage by user group:

sum(
  container_memory_working_set_bytes{name!="", pod=~"jupyter-.*", namespace=~"$hub_name"}
    * on (namespace, pod) group_left(annotation_hub_jupyter_org_username, usergroup)
    group(
        kube_pod_annotations{namespace=~"$hub_name", annotation_hub_jupyter_org_username=~".*", pod=~"jupyter-.*"}
    ) by (pod, namespace, annotation_hub_jupyter_org_username)
    * on (namespace, annotation_hub_jupyter_org_username) group_left(usergroup)
    group(
      label_replace(jupyterhub_user_group_info{namespace=~"$hub_name", username=~".*", usergroup=~"$user_group"},
        "annotation_hub_jupyter_org_username", "$1", "username", "(.+)")
    ) by (annotation_hub_jupyter_org_username, usergroup, namespace)
) by (usergroup, namespace)

Visualizing user group resource usage with Grafana

The PromQL query above is rather long and complex to construct! However, you can benefit from an upstream contribution to the jupyterhub/grafana-dashboards project where we have encapsulated the PromQL queries as Jsonnet code and represented them as Grafana Dashboard visualizations (also known as Grafonnet).

Contributing

Contributions to the jupyterhub-groups-exporter project are welcome! Please follow the standard GitHub workflow:

1. Fork the repository.
2. Create a feature branch.
3. Submit a pull request.

License

This project is licensed under the BSD 3-Clause License.

Resources

• Helm Chart Repository
• Zero to JupyterHub Documentation
• jupyterhub/grafana-dashboards