Master albatross smd #5141
Closed
Master albatross smd #5141
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* fix: skip TF tests for unsupported versions * flake8
* feat: add pytorch-tgi-inference 2.4.0 * add tgi 3.0.1 image * skip faulty test * formatting * formatting * add hf pytorch training 4.46 * update version alias * add py311 to training version * update tests with pyversion 311 * formatting --------- Co-authored-by: Erick Benitez-Ramos <[email protected]>
…mage (aws#4992) Co-authored-by: Erick Benitez-Ramos <[email protected]>
* Fix ssh host policy * Filter policy by algo- * Add docstring * Fix pylint * Fix docstyle summary * Unit test * Fix unit test * Change to unit test * Fix unit tests * Test comment out flaky tests * Readd the flaky tests * Remove flaky asserts * Remove flaky asserts --------- Co-authored-by: Erick Benitez-Ramos <[email protected]>
* change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions --------- Co-authored-by: Roja Reddy Sareddy <[email protected]>
* implemented multi-node distribution with @Remote function * completed unit tests * added distributed training with CPU and torchrun * backwards compatibility nproc_per_node * fixing code: permissions for non-root users, integration tests * fixed docstyle * refactor nproc_per_node for backwards compatibility * refactor nproc_per_node for backwards compatibility * pylint fix, newlines * added unit tests for bootstrap_environment remote * added mpirun protocol for distributed training with @Remote decorator * aligned mpi_utils_remote.py to mpi_utils.py for estimator * updated docstring for sagemaker sdk doc --------- Co-authored-by: Erick Benitez-Ramos <[email protected]>
* feat: Add support for deeepseek recipes * pylint * add unit test
…ws#5002) * fix: fix ValueError when updating a data quality monitoring schedule * Add unit test * black formatting --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: parknate@ <[email protected]>
Co-authored-by: Keshav Chandak <[email protected]>
* Add cleanup logic to model builder integ tests for endpoints * Fix endpoint api call
…lly (aws#5014) * fix: bug in get latest version was getting the max sorted alphabetically instead of sem-ver * handle invalid sev ver and incompatible sagemaker versions --------- Co-authored-by: Eli Davidson <[email protected]> Co-authored-by: parknate@ <[email protected]>
Co-authored-by: pintaoz <[email protected]>
* Fix sourcedir.tar.gz filenames in docstrings * Fix pylint --------- Co-authored-by: pintaoz <[email protected]>
Co-authored-by: pintaoz <[email protected]>
Co-authored-by: pintaoz <[email protected]>
Co-authored-by: pintaoz <[email protected]>
Co-authored-by: pintaoz <[email protected]>
* Fix all type hint and docstrings for callable * Fix codestyle --------- Co-authored-by: pintaoz <[email protected]>
* fix: keep sagemaker_session from being overridden to None, add unit/integ tests * remove commented code * fix styling issues --------- Co-authored-by: Zhaoqi <[email protected]>
* added ap-southeast-7 and mx-central-1 for Jumpstart * added BKK dlc to djl-neuronx --------- Co-authored-by: Isha Chidrawar <[email protected]>
…DK (aws#5050) Co-authored-by: malavhs <[email protected]>
…aws#5052) * Add backward compatbility for RecordSerializer and RecordDeserializer * fix circular import * fix test --------- Co-authored-by: pintaoz <[email protected]>
* fix altconfig hubcontent and reenable integ test * linting * update exception thrown * feat: Add support for TGI Neuronx 0.0.27 and HF PT 2.3.0 image in PySDK (aws#5050) Co-authored-by: malavhs <[email protected]> * add test * update predictor spec accessor * lint * set custom field from HCD config to model spec data class * lint * remove logs * last update --------- Co-authored-by: Malav Shastri <[email protected]> Co-authored-by: malavhs <[email protected]>
* fix: make configs safer * fix: safer destructor in ModelTrainer * format * Update error message * pylint * Create BaseConfig
…ws#5058) * Remove main function entrypoint in ModelBuilder dependency manager. * Remove main function entrypoint in ModelBuilder dependency manager. --------- Co-authored-by: Joseph Zhang <[email protected]>
…aining script which can misguide users. (aws#5057) * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * documentation: Removed a line about python version requirements of training script which can misguide users.Training script can be of latest version based on the support provided by framework_version of the container --------- Co-authored-by: Roja Reddy Sareddy <[email protected]>
Co-authored-by: pintaoz <[email protected]>
…ing one (aws#5063) Co-authored-by: Keshav Chandak <[email protected]>
Co-authored-by: pintaoz <[email protected]>
…ws#5062) * Fix error when there is no session to call _create_model_request() * Fix codestyle --------- Co-authored-by: pintaoz <[email protected]>
* Ensure Model.is_repack() returns a boolean * update test --------- Co-authored-by: pintaoz <[email protected]>
* Allow ModelTrainer to accept hyperparameter file and create Hyperparameter class * pylint * Detect hyperparameters from contents rather than file extension * pylint * change: add integs * change: add integs * change: remove custom hyperparameter tooling * Add tests for hp contracts * change: add unit tests and remove unreachable condition * fix integs * doc check fix * fix tests * fix tox.ini * add unit test
…urated Hub Phase 2 (aws#5070) * change: update image_uri_configs 01-27-2025 06:18:13 PST * fix: skip TF tests for unsupported versions (aws#5007) * fix: skip TF tests for unsupported versions * flake8 * change: update image_uri_configs 01-29-2025 06:18:08 PST * chore: add new images for HF TGI (aws#5005) * feat: add pytorch-tgi-inference 2.4.0 * add tgi 3.0.1 image * skip faulty test * formatting * formatting * add hf pytorch training 4.46 * update version alias * add py311 to training version * update tests with pyversion 311 * formatting --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * feat: use jumpstart deployment config image as default optimization image (aws#4992) Co-authored-by: Erick Benitez-Ramos <[email protected]> * prepare release v2.238.0 * update development version to v2.238.1.dev0 * Fix ssh host policy (aws#4966) * Fix ssh host policy * Filter policy by algo- * Add docstring * Fix pylint * Fix docstyle summary * Unit test * Fix unit test * Change to unit test * Fix unit tests * Test comment out flaky tests * Readd the flaky tests * Remove flaky asserts * Remove flaky asserts --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * change: Allow telemetry only in supported regions (aws#5009) * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions * change: Allow telemetry only in supported regions --------- Co-authored-by: Roja Reddy Sareddy <[email protected]> * mpirun protocol - distributed training with @Remote decorator (aws#4998) * implemented multi-node distribution with @Remote function * completed unit tests * added distributed training with CPU and torchrun * backwards compatibility nproc_per_node * fixing code: permissions for non-root users, integration tests * fixed docstyle * refactor nproc_per_node for backwards compatibility * refactor nproc_per_node for backwards compatibility * pylint fix, newlines * added unit tests for bootstrap_environment remote * added mpirun protocol for distributed training with @Remote decorator * aligned mpi_utils_remote.py to mpi_utils.py for estimator * updated docstring for sagemaker sdk doc --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * feat: Add support for deepseek recipes (aws#5011) * feat: Add support for deeepseek recipes * pylint * add unit test * feat: [JumpStart] Add access configs and training instance type variants artifact uri handling for Curated Hub Phase 2 training integration (aws#1653) * Add access config to training input for Curated Hub Training Integration * Add support to retrieve instance specific training artifact keys * Fix some typos and naming issues * Fix more typos * fix formatting issues with black * modify access config logic so accept_eula is passed into fit * update black formatting * Add more unit tests for passing access configs * fix style errors * fix for failing integ test * fix styles and integ test error * skip blocking integ test * fix formatting * remove env vars when access configs are being used * fix docstyle issue * update usage of access configs, remove conversion of training artifact key to uri * fix styling issues * fix styling issues * fix unit tests * fix adding hubaccessconfig only if hubcontentarn exists * move logic to JumpStartEstimator from Job * Fix styling issues * Remove unused code * fix styling issues * fix unit test failure * fix some formatting, add comments * remove typing for estimator in get_access_configs function * fix circular import dependency * fix styling issues --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * Always add code channel, regardless of network isolation (aws#1657) * fix formatting issue * fix formatting issue * fix formatting issue * fix tensorflow file --------- Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: Erick Benitez-Ramos <[email protected]> Co-authored-by: varunmoris <[email protected]> Co-authored-by: Gary Wang <[email protected]> Co-authored-by: ci <ci> Co-authored-by: parknate@ <[email protected]> Co-authored-by: rsareddy0329 <[email protected]> Co-authored-by: Roja Reddy Sareddy <[email protected]> Co-authored-by: Bruno Pistone <[email protected]>
* feat: Make DistributedConfig Extensible * pylint * Include none types when creating config jsons for safer reference * fix: update test to account for changes * format * Add integ test * pylint * prepare release v2.240.0 * update development version to v2.240.1.dev0 * Fix key error in _send_metrics() (aws#5068) Co-authored-by: pintaoz <[email protected]> * fix: Added check for the presence of model package group before creating one (aws#5063) Co-authored-by: Keshav Chandak <[email protected]> * Use sagemaker session's s3_resource in download_folder (aws#5064) Co-authored-by: pintaoz <[email protected]> * remove union * fix merge artifact * Change dir path to distributed_drivers * update paths --------- Co-authored-by: ci <ci> Co-authored-by: pintaoz-aws <[email protected]> Co-authored-by: pintaoz <[email protected]> Co-authored-by: Keshav Chandak <[email protected]> Co-authored-by: Keshav Chandak <[email protected]>
Co-authored-by: pintaoz <[email protected]>
…image. (aws#1661) * feature:support custom workflow deployment in ModelBuilder using SMD inference image. * Rename test case and pass session. * Address PR comments. * Tweak resource cleanup logic in integ test. * Fixing CodeBuild integ test failures. * Renamed integ test. * Remove unused integ test, restore once GA. --------- Co-authored-by: Joseph Zhang <[email protected]>
* Remove property@ decorator from ABC definition. * Cache client as instance attribute in @Property. * Fix flake8 issue. --------- Co-authored-by: Joseph Zhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
unique_name_from_baseto create resource names in integ tests (if appropriate)By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.