Commit 95096e8
Release v3 (#109)
* Introduce helm charts for hyperpod inference operator
* Introduce helm charts for hyperpod inference operator
* Introduce helm charts for hyperpod inference operator
* Update Helm charts for inference operator, clean up to remove bedrock references.
* Changes to
1. update image tag
2. Remove IAM policies for execution role
3. Rename to hyperpod-inference-operator prefix instead of deploymentoperator prefix
* Removed binary from the code base.
* Nit: Update the app name labels for sample yaml files.
* Merge pull request #29 from mbnavali/main
Introduce helm charts for hyperpod inference operator
* Add crds, service account and region (#32)
* Add CRDs and setup for region
* Change annotation for SA
* Remove default region
* Add hyperpod inference classes
**Description**
Support jumpstart and custom model endpoints
**Testing Done**
Tested manually, will add unit tests in next few PRs
* Refactor create inference function
**Description**
Refactor ModelEndpoint classes to let create happen in separate method instead of constructor
**Testing Done**
Manually tested in demo notebook
* Add List, Delete, Describe endpoint features
Tested manually in demo jupyter notebook
* Add unit test and update class names
**Description**
**Testing Done**
Unit test passes
* Add end and setup.py
* Update gitignore
* Add setup.cfg
* Fix HPEndpoint class and add optional values
* remove utils.py
* Make function classmethod and update unit tests
* Fix bugs for inference endpoint
* Small fixes
* build: add mountpoint s3 csi driver, keda + cert-manager controllers as dependencies
feat: add pv and pvc creation as part of helm
* chore: add inference operator as dependency for HP Helm Chart, default disabled
* feat: add support for jumpstart gated models
* fix: remove stray symbol
* fix: rename inference operator chart to match name in parent
* change: sync charts with latest version of operator
* doc: update readme.md identifying the inference operator as a subchart
* Add HyperpodPytorchJob class (#39)
* Add HyperpodPytorchJob
* update to class methods
* update to class methods
* Address feedback
* Fix bug
---------
Co-authored-by: pintaoz <[email protected]>
* Add tlsConfig to quick create
* Revert "Add tlsConfig to quick create"
This reverts commit 574351e.
* Add tls config
* Update CRD configs and minor updates
* Add model_location to HPEndpont
* Adding observability command to fetch details of grafana, prometheus and list of enabled metrics.
* Training CLI implementation: create
* Adding observability SDK experience and updating CLI command signature
* Rename CLI commands to be consistent with SDK
* Training CLI for Launch
* Training CLI for Launch
* Training CLI for Launch
* Training CLI for Launch
* Update JumpStartModel interface (#51)
* Update JumpStartModel interface
Tested in Jupyter notebook that endpoint can be successfully invoked
* Add refresh method
* remove debugging print
* Update HPEndpoint classes
Tested using example notebooks
* Add example notebooks
These notebooks haven't been cleaned up and they are for internal review only. Commands are supposed to change later
* Add metadata class
* Get Cluster Context
* Update to HyperPodManager call
* Cleanup import
* Training CLI for Launch
* Training CLI for Launch
* Training CLI for Launch
* Update HyperPodPytorchJob (#52)
* Add HyperpodPytorchJob
* update to class methods
* update to class methods
* Address feedback
* Fix bug
* Update HyperPodPytorchJob
* Fix dependency
* Add status
* Add list_pods and get_logs_from_pod
* Add error handling and metadata
* Add example notebook
* Fix bug
---------
Co-authored-by: pintaoz <[email protected]>
* E2E testing done for inference CLI
* delete build
* Revert accidental submodule pointer change
* Update inference example notebook and fix bugs
* Reformat code with black
* Add get_logs function for inference
* Update HyperPodPytorchJob to not use _HyperPodPytorchJob object (#63)
* Add HyperpodPytorchJob
* update to class methods
* update to class methods
* Address feedback
* Fix bug
* Update HyperPodPytorchJob
* Fix dependency
* Add status
* Add list_pods and get_logs_from_pod
* Add error handling and metadata
* Add example notebook
* Fix bug
* Hide _HyperPodPytorchJob from user
* Fix merge conflicts
---------
Co-authored-by: pintaoz <[email protected]>
* Update get_logs function to accept since_hour
Tested in notebook
* Separate get_logs and get_operator_logs methods
* Update get_logs to class method
* Add container name to get_logs function
* Add container in get_logs_from_pod (#66)
Co-authored-by: pintaoz <[email protected]>
* change inference CLI directory, add inference CLI notebook, add get-logs and get-operator-logs
* delete build
* Training CLI for Launch - Changes per SDK HyperPodPytorchJob constructor (#64)
* Training CLI for Launch
* Training CLI for Launch
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* | * d2453d6 (rig-dev) Add notes about HMA patching
* add cloudwatchtrigger and autoscalingspec to model.py and schema.json
* Add exception handling and update example notebooks (#71)
* Add exception handling and update example notebooks
* Update HPEndpoint get status
* Add unit tests for training sdk
* Update util tests
* Add training cli example notebook (#72)
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Address comments
* fix tls flag issue, fsx endpoint successfully created with cli notebook
* clear notebook outputs
* minor update in notebook
* minor change to notebook
* Move Metadata model to common (#75)
Co-authored-by: pintaoz <[email protected]>
* REstructure HPCLI
* Fix training cli unit tests
* Fix list jobs test
* Fixed logger
Logger sometimes does not function properly.
Tested in example notebook
* Updates from Testing
* Update import path
* Revert lines from readme (should not have been updated)
* unit test for inference CLI done
* resolve merge conflicts
* rebase with master
* clean up
* clean up recipes
* Merging hyp and hyperpod commands in a common entry point as hyp
* Removing not relevant directories and updating setup and pyproject (#87)
* Add unit test and fix HyperPod Manager (#84)
* Add unit test and fix HyperPod Manager
1. Default namespace can be set by HyperpodManager.set_context()
2. Added unit tests for inference
* Remove debug print
* update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook (#85)
* Append uuid to endpoint name (#90)
* Add unit test and fix HyperPod Manager
1. Default namespace can be set by HyperpodManager.set_context()
2. Added unit tests for inference
* Remove debug print
* Append uuid to model name and endpoint name
* minor fix in create method
* Fix set_context in HyperPodManager (#91)
* Add unit test and fix HyperPod Manager
1. Default namespace can be set by HyperpodManager.set_context()
2. Added unit tests for inference
* Remove debug print
* Append uuid to model name and endpoint name
* minor fix in create method
* Fix set_context in HyperPodManager
* Add logging info for delete()
* Remove Self from type hint (#92)
* Add unit test and fix HyperPod Manager
1. Default namespace can be set by HyperpodManager.set_context()
2. Added unit tests for inference
* Remove debug print
* Append uuid to model name and endpoint name
* minor fix in create method
* Fix set_context in HyperPodManager
* Add logging info for delete()
* Remove Self in type hint
This only supports python version 3.11+
* Minor documentation fixes for RIG Helm (#93)
* Bug fix: Fixed create command job error (#94)
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* [HyperPod Inference] Update RBAC with perms for KEDA, allow direct provision of operator image repository (#44)
* change: add rbac perms for KEDA scaledobject
* change: allow image.repository to be set directly via flag
* change: consistently use namePrefix for app name and resources
* fix: remove empty string as default value
* fix: reference correct value for tls cert bucket URI
fix: override empty image.repository values from domain map
change: use shorter prefix for namespace
change: do not require sageMakerEndpoint
* Adding dynamic flag for dependencies installation (#95)
* Add utils unit tests for training cli (#97)
* Bug fix: Fixed create command job error
* Add utils unit tests for training cli
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Add instance type validation for JS model (#98)
* Adding observability notebook (#96)
* Inference dogfood notebook update (#99)
* update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook
* change hyperpod to hyp in inferece cli notebook
* update inference CLI notebook to reflect uuid change
* Unique job name: Append uuid to training job name (#101)
* Bug fix: Fixed create command job error
* Add utils unit tests for training cli
* Unique job name: Append uuid to training job name
* Unique job name: Append uuid to training job name
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Inference CLI update after dogfood (#102)
* update print for inference CLI for list and describe, bug fix for since-hours flag to support float, minor update to notebook
* change hyperpod to hyp in inferece cli notebook
* update inference CLI notebook to reflect uuid change
* update list and describe after dogfood callout, remove get_logs for inference CLI, update help text for CLI
* Lookup standard Helm release name for RIG Helm installation (1ff9c) (#104)
* Minor negative case update for Helm release name lookup during RIG Helm installation (#105)
* Add JumpStart PublicHub model visualization utilities. (#106)
* Add JumpStart PublicHub model visualization utilities.
* Add JumpStart PublicHub model visualization utilities.
* Update cli command noun to hyp-*, logging, list_jobs bug fix (#107)
* Bug fix: Fixed create command job error
* Add utils unit tests for training cli
* Unique job name: Append uuid to training job name
* Unique job name: Append uuid to training job name
* Update command verb name to hyp, logging, list_jobs bug fix
* Update command verb name to hyp, logging, list_jobs bug fix
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Make metadata name same as endpoint name; Updated instance type validation (#110)
Unit test passes and verified in jupyter notebook
* Add integ test for training CLI and SDK (#100)
* Add integ test for training cli
* Add integ test for training sdk
* relax pydantic version
* fix pydantic version
* return latest cluster and fix set cluster context test
---------
Co-authored-by: adishaa <[email protected]>
* baseline inference integration test for CLI and SDK, minor bug fixes (#111)
* baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output
* clean up merge header
* Remove UUID from training and Inference (#108)
* Remove UUID from training and Inference
* Fixes and PR comments
* Fix
* Fix logging
* Fix
* Update inference logging setup similar to training (#113)
* Bug fix: Fixed create command job error
* Add utils unit tests for training cli
* Unique job name: Append uuid to training job name
* Unique job name: Append uuid to training job name
* Update command verb name to hyp, logging, list_jobs bug fix
* Update command verb name to hyp, logging, list_jobs bug fix
* Update inference logging setup similar to training
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Change hp-pytorch-job to hyp-pytorch-job (#115)
Co-authored-by: adishaa <[email protected]>
* Add methods for list pods and namespaces (#114)
Added unit test and tested in notebook
* Minor change in training cli notebook: UUID removed (#117)
* Bug fix: Fixed create command job error
* Add utils unit tests for training cli
* Unique job name: Append uuid to training job name
* Unique job name: Append uuid to training job name
* Update command verb name to hyp, logging, list_jobs bug fix
* Update command verb name to hyp, logging, list_jobs bug fix
* Update inference logging setup similar to training
* Minor change in training cli notebook: UUID removed
---------
Co-authored-by: Roja Reddy Sareddy <[email protected]>
* Cleaner error messading for Endpoint invoke (#112)
* Invoke Validation check
* Fix
* Bumping kubernetes python client version and updating observability command (#116)
* change: add prefix to convert bucket name to s3 URI (#109)
* Added type check on commands before invoking subprocess run (#118)
* Bring HyperPodManager class util functions (#119)
* Bring HyperPodManager class util functions
Unit tests pass and verified in notebook
* Update init
* Add list_pods and get_logs for CLI (Update notebook, integ test, unit test) (#120)
* baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output
* update integ test in progress
* update cli code, notebook, integ and unit test to add list_pods and get_logs
* clean up merge header
* Update inference and training to only check kubeconfig on the first time (#122)
Updated unit tests and verified in notebook
* Update Readme to include Inference and Training (#121)
* Update Readme to include Inference and Training
* Update readme command
* Documentation updates
* Doc Updates
* Move observability utils and constants; Rename set_context/get_context (#125)
* Update inference and training to only check kubeconfig on the first time
Updated unit tests and verified in notebook
* Remove old unit tests
* Revert "Remove old unit tests"
This reverts commit e728e9864c853635f724e9a377fbe870f0f2e2a4.
* Move observability utils and constants; Rename set_context/get_context
* Updating template packages name and structure (#126)
* Changelog updates (#128)
* Changelog updates
* Rebase and update
* Fix
* Readme update (#129)
* Update Readme to include Inference and Training
* Update readme command
* Documentation updates
* Doc Updates
* Readme updates
* Fix README.md
* Remove Orchestrator from List Cluster
* Changes to README.md
* Fix the link
* Remove orchestrator from README.md
* Unit test fix (#127)
* use unique basename for test file modules
* fix unit tests
* remove append_uuid test
* fix failing test_invoke tests
---------
Co-authored-by: adishaa <[email protected]>
* Fix get_cluster_context runtime error (#130)
* Remove Py38 Tests (#131)
* Fix get_cluster_context runtime error
* Remove Py38 fromtests
* UNit test fixes (#132)
* Fix get_cluster_context runtime error
* Remove Py38 fromtests
* Fix
* Unit test fixes
* Inference integ tests all passed in Chait's account (#135)
* baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output
* update integ test in progress
* update cli code, notebook, integ and unit test to add list_pods and get_logs
* clean up merge header
* inference integ tests all passing in chait's account
* Update operator namespace string (#137)
* Inference integ test passed on beta account (#140)
* baseline inference integration test for CLI and SDK, minor bug fix for inference cli, clear inference sdk notebook output
* update integ test in progress
* update cli code, notebook, integ and unit test to add list_pods and get_logs
* clean up merge header
* inference integ tests all passing in chait's account
* integ test passing on beta account
* is_kubeconfig_loaded Fix (#139)
* Test PR
* Fix is_kubeconfig_loaded Class attribute bug
* Include main branch in pull request target
---------
Co-authored-by: Mahadeva N <[email protected]>
Co-authored-by: Shantanu Tripathi <[email protected]>
Co-authored-by: Zhaoqi <[email protected]>
Co-authored-by: jzhaoqwa <[email protected]>
Co-authored-by: Rahul Sahu <[email protected]>
Co-authored-by: rvasahu-amazon <[email protected]>
Co-authored-by: pintaoz-aws <[email protected]>
Co-authored-by: pintaoz <[email protected]>
Co-authored-by: Molly He <[email protected]>
Co-authored-by: Amarjeet LNU <[email protected]>
Co-authored-by: Roja Reddy Sareddy <[email protected]>
Co-authored-by: rsareddy0329 <[email protected]>
Co-authored-by: Chris Chan <[email protected]>
Co-authored-by: adishaa <[email protected]>
Co-authored-by: Aditi Sharma <[email protected]>
Co-authored-by: chnnmz <[email protected]>1 parent 1ffc962 commit 95096e8
File tree
201 files changed
+21510
-4494
lines changed- .github/workflows
- examples
- inference
- CLI
- SDK
- observability
- training
- CLI
- SDK
- helm_chart
- HyperPodHelmChartForRIG
- HyperPodHelmChart
- charts/inference-operator
- config
- crd
- manager
- metrics
- network-policy
- prometheus
- rbac
- samples
- templates
- hyperpod-custom-inference-template
- hyperpod_custom_inference_template
- v1_0
- hyperpod-jumpstart-inference-template
- hyperpod_jumpstart_inference_template
- v1_0
- hyperpod-pytorch-job-template
- hyperpod_pytorch_job_template
- v1_0
- src
- hyperpod_cli
- commands
- sagemaker
- hyperpod
- cli
- clients
- commands
- constants
- service
- telemetry
- templates
- validators
- common
- config
- inference
- config
- observability
- training
- config
- test
- integration_tests
- cli
- sdk
- unit_tests
- clients
- cli
- common
- inference
- service
- training
- validators
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
201 files changed
+21510
-4494
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
50 | | - | |
| 51 | + | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
66 | 67 | | |
67 | 68 | | |
68 | 69 | | |
69 | | - | |
| 70 | + | |
70 | 71 | | |
71 | 72 | | |
72 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
21 | 25 | | |
22 | 26 | | |
23 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
| 1 | + | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
Large diffs are not rendered by default.
Whitespace-only changes.
This file was deleted.
Lines changed: 162 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
0 commit comments