Skip to content
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
f698f21
feat: Include base requirements in requirements-dev.txt
sarthakag Jan 5, 2026
57affd3
Cleans up duplicate pip install
sarthakag Jan 7, 2026
41ad5db
Bump crispy-bootstrap5 from 2025.6 to 2026.3 in /community/front-end/ofe
dependabot[bot] Mar 1, 2026
081fff6
Update cachetools requirement in /community/front-end/ofe
dependabot[bot] Mar 1, 2026
1aa18a0
add ttl label to remaining ml and gke focused blueprints (#5294)
saara-tyagi27 Mar 11, 2026
1d44d83
Bump django from 6.0.2 to 6.0.3 in /community/front-end/ofe (#5310)
dependabot[bot] Mar 11, 2026
3ebf170
Docs: update storage-gke deployment instructions (#5338)
shubpal07 Mar 11, 2026
36dd0b4
remove redundant build files
sudheer-quad Mar 11, 2026
38fc190
feat: full_quota_impl (#5140)
kvenkatachala333 Mar 12, 2026
c3a9d4d
Replace cat with jq for safe JSON generation
aslam-quad Mar 12, 2026
123e8ad
Revert "feat: full_quota_impl" (#5344)
Neelabh94 Mar 12, 2026
f156981
Add robust destroy to integration test (#5345)
LAVEEN Mar 12, 2026
d9d3773
Merge V1.84.0 into Develop (#5349)
siddhartha-quad Mar 13, 2026
09b1693
Add Managed Lustre integration in gke a4x-max (#5337)
parulbajaj01 Mar 13, 2026
b8e93dd
update terraform-provider version to >= 6.9.0, <= 7.23.0
siddhartha-quad Mar 13, 2026
369215d
Replace cat with jq in publish_release.sh (#5342)
aslam-quad Mar 13, 2026
7d9e86d
Bump virtualenv from 20.36.1 to 21.1.0 in /community/front-end/ofe (#…
dependabot[bot] Mar 13, 2026
ad3511a
Include base requirements in requirements-dev.txt (#5064)
sarthakag Mar 13, 2026
b62b016
Remove redundant build files (#5339)
sudheer-quad Mar 13, 2026
9607e6e
Update cachetools requirement from <7.0,>=2.0 to >=2.0,<8.0 in /commu…
aslam-quad Mar 13, 2026
88cf01b
Bump go.opentelemetry.io/otel/sdk from 1.39.0 to 1.40.0 (#5302)
dependabot[bot] Mar 13, 2026
84513a6
Bump crispy-bootstrap5 from 2025.6 to 2026.3 in /community/front-end/…
aslam-quad Mar 13, 2026
1517cdb
Bump pyjwt from 2.11.0 to 2.12.0 in /community/front-end/ofe
dependabot[bot] Mar 14, 2026
8116d96
use constant for google provider version in Go files
aslam-quad Mar 15, 2026
47549cd
update terraform-provider version to >= 6.9.0, <= 7.23.0 (#5355)
aslam-quad Mar 16, 2026
059af7d
Merge branch 'develop' into update-terraform-provider-version
aslam-quad Mar 16, 2026
f44cdd5
use constant for google provider version in Go files (#5361)
aslam-quad Mar 16, 2026
8f07f2d
Fix the network name (#5315)
LAVEEN Mar 16, 2026
71695be
fix(slurm): robustly convert enable_oslogin boolean to metadata strin…
kvenkatachala333 Mar 16, 2026
2966694
pin shfmt version to resolve go version conflict
kadupoornima Mar 17, 2026
ee40474
pin golang.org/x/tools/cmd/goimports to v0.42.0
kadupoornima Mar 17, 2026
5188838
Merge branch 'develop' into go-vers
kadupoornima Mar 17, 2026
d473900
Pin shfmt and goimports version to resolve Go version conflict (#5365)
kadupoornima Mar 17, 2026
1d4a735
Update cloud_dns_config to default to KUBE_DNS (CoreDNS) (#5336)
SwarnaBharathiMantena Mar 17, 2026
4e5adde
Refactor block topology (#5359)
Neelabh94 Mar 17, 2026
1cca13e
Add Binary dependencies downloading script (#5354)
scaliby Mar 17, 2026
f52ea95
Bump pyasn1 from 0.6.2 to 0.6.3 in /community/front-end/ofe
dependabot[bot] Mar 17, 2026
ae2ed3e
Bump the ofe-minor-and-patch-updates group in /community/front-end/of…
dependabot[bot] Mar 18, 2026
cd49cd6
Update requirements.txt
aslam-quad Mar 18, 2026
8574aef
Merge branch 'develop' into dependabot/pip/community/front-end/ofe/py…
aslam-quad Mar 18, 2026
c7936cb
Bump pyasn1 from 0.6.2 to 0.6.3 in /community/front-end/ofe (#5366)
aslam-quad Mar 18, 2026
8c02083
feat(storage): Enable GCS zonal bucket capability with RAPID storage.…
Neelabh94 Mar 18, 2026
5c1087a
Bump pyjwt from 2.11.0 to 2.12.0 in /community/front-end/ofe (#5358)
aslam-quad Mar 18, 2026
dd5375a
Refine gcluster deploy flag checks to only consider local flags. (#5372)
scaliby Mar 21, 2026
df5f30f
Support future reservation in name check validator (#5252)
saara-tyagi27 Mar 23, 2026
a1cba83
Fix: use relative path for slurm_files build_dir (#5253)
mtibben Mar 23, 2026
f65ba73
Update project cleanup schedule to run every 4 hours (#5381)
saara-tyagi27 Mar 23, 2026
3db9949
Fix: use cleaned relative path instead of absolute path for local mod…
mtibben Mar 23, 2026
16473c8
Increase version to 1.85.0
siddhartha-quad Mar 24, 2026
fc84605
Update Toolkit release to v1.85.0 (#5390)
siddhartha-quad Mar 24, 2026
869fd52
added check condition
siddhartha-quad Mar 24, 2026
4e76995
Update pkg/dependencies/downloader.go
LAVEEN Mar 24, 2026
50e1e7b
Added check condition in pkg/dependencies/downloader.go (#5394)
siddhartha-quad Mar 24, 2026
eed7b09
feat: multi-arch build support and README updates (#5388)
kvenkatachala333 Mar 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,10 @@ install-dev-deps: warn-terraform-version warn-packer-version check-pre-commit ch
go install github.com/fzipp/gocyclo/cmd/gocyclo@latest
go install github.com/go-critic/go-critic/cmd/gocritic@latest
go install github.com/google/addlicense@latest
go install mvdan.cc/sh/v3/cmd/shfmt@latest
go install golang.org/x/tools/cmd/goimports@latest
go install mvdan.cc/sh/v3/cmd/shfmt@v3.12.0
go install golang.org/x/tools/cmd/goimports@v0.42.0
go install honnef.co/go/tools/cmd/staticcheck@latest
go install github.com/jstemmer/go-junit-report/v2@latest
pip install -r community/modules/scheduler/schedmd-slurm-gcp-v6-controller/modules/slurm_files/scripts/requirements.txt
pip install -r community/modules/scheduler/schedmd-slurm-gcp-v6-controller/modules/slurm_files/scripts/requirements-dev.txt
pip install mypy==1.18.2

Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,13 @@ hpc-slurm/
See
[Cloud Docs on Installing Dependencies](https://cloud.google.com/cluster-toolkit/docs/setup/install-dependencies).

When running commands like `deploy`, `destroy`, or `export-outputs`, the toolkit can automatically download missing dependencies. You can control this behavior using the `--download-dependencies` flag:

* `--download-dependencies`: Automatically download missing dependencies without prompting.
* `--download-dependencies=false`: Fail immediately if any required dependencies are missing.

If the flag is not provided, you will be interactively asked to confirm the download.

### Notes on Packer

The Toolkit supports Packer templates in the contemporary [HCL2 file
Expand Down
54 changes: 54 additions & 0 deletions cmd/dependencies.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// Copyright 2026 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cmd

import (
"hpc-toolkit/pkg/dependencies"
"hpc-toolkit/pkg/logging"

"github.com/spf13/cobra"
"github.com/spf13/pflag"
)

var downloadDependencies bool
var ensureDependenciesFn = dependencies.EnsureDependencies

func addDependenciesFlags(flagset *pflag.FlagSet) {
flagset.BoolVar(&downloadDependencies, "download-dependencies", false, "Automatically download missing dependencies. Pass --download-dependencies=false to fail if missing.")
}

func initDependencies(cmd *cobra.Command) {
allowedCmds := map[string]bool{
"deploy": true,
"destroy": true,
"export-outputs": true,
}
if !allowedCmds[cmd.Name()] {
return
}

decision := dependencies.DownloadDecisionAsk
if cmd.Flags().Changed("download-dependencies") {
if downloadDependencies {
decision = dependencies.DownloadDecisionYes
} else {
decision = dependencies.DownloadDecisionNo
}
}

if err := ensureDependenciesFn(decision); err != nil {
logging.Fatal("Failed to setup dependencies: %v", err)
}
}
84 changes: 84 additions & 0 deletions cmd/dependencies_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/*
Copyright 2026 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cmd

import (
"testing"

"hpc-toolkit/pkg/dependencies"

"github.com/spf13/cobra"
"github.com/spf13/pflag"
)

func TestAddDependenciesFlags(t *testing.T) {
flags := pflag.NewFlagSet("test", pflag.ContinueOnError)
addDependenciesFlags(flags)

flag := flags.Lookup("download-dependencies")
if flag == nil {
t.Fatalf("Expected 'download-dependencies' flag to be added, but it was not")
}

if flag.DefValue != "false" {
t.Errorf("Expected default value to be 'false', got '%s'", flag.DefValue)
}
}

func TestInitDependenciesIgnoresCommands(t *testing.T) {
cmd := &cobra.Command{Use: "unrelated-command"}
flags := pflag.NewFlagSet("test", pflag.ContinueOnError)
addDependenciesFlags(flags)
cmd.Flags().AddFlagSet(flags)

called := false
originalFn := ensureDependenciesFn
ensureDependenciesFn = func(d dependencies.DownloadDecision) error {
called = true
return nil
}
defer func() { ensureDependenciesFn = originalFn }()

initDependencies(cmd)

if called {
t.Errorf("Expected ensureDependenciesFn not to be called")
}
}

func TestInitDependenciesAllowedCommands(t *testing.T) {
for _, cmdName := range []string{"deploy", "destroy", "export-outputs"} {
cmd := &cobra.Command{Use: cmdName}
flags := pflag.NewFlagSet("test", pflag.ContinueOnError)
addDependenciesFlags(flags)
cmd.Flags().AddFlagSet(flags)

called := false
originalFn := ensureDependenciesFn
ensureDependenciesFn = func(d dependencies.DownloadDecision) error {
called = true
return nil
}
defer func() { ensureDependenciesFn = originalFn }()

initDependencies(cmd)

if !called {
t.Errorf("Expected ensureDependenciesFn to be called for command %s", cmdName)
}
}
}
2 changes: 1 addition & 1 deletion cmd/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ func runDeployCmd(cmd *cobra.Command, args []string) {
deplRoot = args[0]
// check that no "create" flags were specified
cmd.Flags().VisitAll(func(f *pflag.Flag) {
if f.Changed && createCmd.Flag(f.Name) != nil {
if f.Changed && createCmd.LocalFlags().Lookup(f.Name) != nil {
checkErr(fmt.Errorf("cannot specify flag %q with DEPLOYMENT_DIRECTORY provided", f.Name), nil)
}
})
Expand Down
4 changes: 3 additions & 1 deletion cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,15 +52,17 @@ var (
logging.Fatal("cmd.Help function failed: %s", err)
}
},
Version: "v1.84.0",
Version: "v1.85.0",
Annotations: annotation,
}
)

func init() {
addDependenciesFlags(rootCmd.PersistentFlags())
addColorFlag(rootCmd.PersistentFlags())
rootCmd.PersistentPreRun = func(cmd *cobra.Command, args []string) {
initColor()
initDependencies(cmd)
}
}

Expand Down
52 changes: 26 additions & 26 deletions community/front-end/ofe/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,51 +1,51 @@
altgraph==0.17.5
archspec==0.2.5
argcomplete==3.6.3
asgiref==3.11.0
astroid==4.0.3
asgiref==3.11.1
astroid>=4.0.2,<4.1.0
attrs==25.4.0
# This should be supported by zoneinfo in Python 3.9+
backports.zoneinfo==0.2.1;python_version<"3.9"
cachetools>=2.0,<7.0
certifi==2026.1.4
cachetools>=2.0,<8.0
certifi==2026.2.25
cffi>=2.0.0
cfgv==3.5.0
charset-normalizer==3.4.4
click==8.3.1
crispy-bootstrap5==2025.6
crispy-bootstrap5==2026.3
cryptography==46.0.5
decorator==5.2.1
defusedxml==0.7.1
dill==0.4.1
distlib==0.4.0
# django-revproxy==0.11.0 released but not yet in pypi
git+https://github.com/jazzband/django-revproxy.git@d2234005135dc0771b7c4e0bb0465664ccfa5787
Django==6.0.2
django-allauth==65.14.1
django-crispy-forms==2.5
Django==6.0.3
django-allauth==65.14.3
django-crispy-forms==2.6
django-extensions==3.2.3
djangorestframework==3.16.1
filelock==3.20.3
google-api-core==2.29.0
google-api-python-client==2.188.0
filelock==3.24.3
google-api-core==2.30.0
google-api-python-client==2.190.0
google-auth==2.48.0
google-auth-httplib2==0.3.0
google-cloud-artifact-registry==1.19.0
google-cloud-artifact-registry==1.20.0
google-cloud-secret-manager==2.26.0
google-cloud-build==3.35.0
google-cloud-billing==1.18.0
google-cloud-core==2.5.0
google-cloud-pubsub==2.34.0
google-cloud-compute==1.43.0
google-cloud-storage==3.8.0
google-cloud-pubsub==2.35.0
google-cloud-compute==1.44.0
google-cloud-storage==3.9.0
google-cloud-secret-manager==2.26.0
google-crc32c==1.8.0
google-resumable-media==2.8.0
googleapis-common-protos==1.72.0
grafana-api==1.0.3
grpc-google-iam-v1==0.14.3
grpcio==1.76.0
grpcio-status==1.76.0
grpcio==1.78.0
grpcio-status==1.78.0
h11==0.16.0
httplib2==0.31.2
identify==2.6.16
Expand All @@ -65,15 +65,15 @@ nodeenv==1.10.0
oauthlib==3.3.1
path==17.1.1
pkgutil_resolve_name==1.3.10
platformdirs==4.5.1
platformdirs==4.9.2
pre-commit==4.5.1
proto-plus==1.27.0
proto-plus==1.27.1
protobuf>=6.31.1,<7.0.0
pyasn1==0.6.2
pyasn1==0.6.3
pyasn1-modules==0.4.2
pycparser==3.0
PyJWT==2.11.0
pylint==4.0.4
PyJWT==2.12.0
pylint==4.0.5
pylint-django==2.7.0
pylint-plugin-utils==0.9.0
pyparsing==3.3.2
Expand All @@ -99,9 +99,9 @@ typing-inspect==0.9.0
typing_extensions==4.15.0
uritemplate==4.2.0
urllib3==2.6.3
uvicorn==0.40.0
virtualenv==20.36.1
wrapt==2.1.0
xmltodict==1.0.2
uvicorn==0.41.0
virtualenv==21.1.0
wrapt==2.1.1
xmltodict==1.0.4
yq==3.4.3
zipp==3.23.0
2 changes: 1 addition & 1 deletion community/modules/compute/gke-nodeset/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:gke-nodeset/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:gke-nodeset/v1.85.0"
}
}
2 changes: 1 addition & 1 deletion community/modules/compute/gke-partition/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:gke-partition/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:gke-partition/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ terraform {
}

provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:htcondor-execute-point/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:htcondor-execute-point/v1.85.0"
}
}
2 changes: 1 addition & 1 deletion community/modules/compute/mig/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:mig/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:mig/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
terraform {
required_version = "= 1.12.2"
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset-dynamic/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset-dynamic/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ terraform {
required_version = "= 1.12.2"

provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset-tpu/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset-tpu/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,6 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-nodeset/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ terraform {
required_version = "= 1.12.2"

provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-partition/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:schedmd-slurm-gcp-v6-partition/v1.85.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:slurm-cloudsql-federation/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:slurm-cloudsql-federation/v1.85.0"
}
provider_meta "google-beta" {
module_name = "blueprints/terraform/hpc-toolkit:slurm-cloudsql-federation/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:slurm-cloudsql-federation/v1.85.0"
}

required_version = "= 1.12.2"
Expand Down
2 changes: 1 addition & 1 deletion community/modules/file-system/nfs-server/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:nfs-server/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:nfs-server/v1.85.0"
}

required_version = "= 1.12.2"
Expand Down
4 changes: 2 additions & 2 deletions community/modules/files/fsi-montecarlo-on-batch/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ terraform {
}
}
provider_meta "google" {
module_name = "blueprints/terraform/hpc-toolkit:fsi-montecarlo-on-batch/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:fsi-montecarlo-on-batch/v1.85.0"
}
provider_meta "google-beta" {
module_name = "blueprints/terraform/hpc-toolkit:fsi-montecarlo-on-batch/v1.84.0"
module_name = "blueprints/terraform/hpc-toolkit:fsi-montecarlo-on-batch/v1.85.0"
}
}
Loading
Loading