Skip to content

Environment variables as cluster option #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sebastian-luna-valero wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Environment variables as cluster option #127

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1ba95d9
Environment variables as cluster option
sebastian-luna-valero Sep 11, 2025
5f39017
follow 2i2c example
sebastian-luna-valero Sep 22, 2025
cc5454e
add DASK_GATEWAY__CLUSTER__OPTIONS__IMAGE
sebastian-luna-valero Oct 7, 2025
89f6acc
bugfix
sebastian-luna-valero Oct 7, 2025
20e7662
DASK_GATEWAY__CLUSTER__OPTIONS__IMAGE conflicts when setting the work…
sebastian-luna-valero Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions providers/kubernetes/daskhub.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,23 +19,27 @@ dask-gateway:
c.ClusterConfig.cluster_max_workers = 4
c.ClusterConfig.idle_timeout = 1800
optionHandler: |
from dask_gateway_server.options import Options, Integer, Float, String
from dask_gateway_server.options import Options, Integer, Float, String, Mapping

def options_handler(options):
if ":" not in options.image:
raise ValueError("When specifying an image you must also provide a tag")
return {
"worker_cores": options.worker_cores,
"worker_memory": int(options.worker_memory * 2 ** 30),
"image": options.image,
}
def cluster_options(user):
def option_handler(options):
if ":" not in options.image:
raise ValueError("When specifying an image you must also provide a tag")

c.Backend.cluster_options = Options(
Integer("worker_cores", default=1, min=1, max=8, label="Worker Cores"),
Float("worker_memory", default=2, min=2, max=32, label="Worker Memory (GiB)"),
String("image", default="quay.io/pangeo/pangeo-notebook:2025.06.02", label="Image"),
handler=options_handler,
)
return {
"worker_cores": options.worker_cores,
"worker_memory": int(options.worker_memory * 2 ** 30),
"image": options.image,
"environment_vars": options.environment_vars,
}
return Options(
Integer("worker_cores", default=1, min=1, max=1, label="Worker Cores"),
Float("worker_memory", default=2, min=2, max=7, label="Worker Memory (GiB)"),
String("image", default="quay.io/pangeo/pangeo-notebook:2025.06.02", label="Image"),
Mapping("environment_vars", {}, label="Environment Variables"),
handler=options_handler,
)
c.Backend.cluster_options = cluster_options
prefix: /services/dask-gateway
backend:
scheduler:
Expand Down