|
438 | 438 | "training_job.wait()"
|
439 | 439 | ]
|
440 | 440 | },
|
441 |
| - { |
442 |
| - "cell_type": "markdown", |
443 |
| - "metadata": {}, |
444 |
| - "source": [ |
445 |
| - "## [Private Preview Feedback needed] De-nesting simplifications\n", |
446 |
| - "\n", |
447 |
| - "As noticed in the TrainingJob.create call above, the users need to configure API mirrored pythonic structures instead of the raw dictionaries while making an API request. While this simplifies the user experience in terms of the object-oriented experience we envision to provide, there is a proposal on table to simplify the experience further by de-nesting the complexity of nested structures in a very simplified and opinionated key-value pair inputs. With these simplifications above TrainingJob.create would look like:\n", |
448 |
| - "\n", |
449 |
| - "```\n", |
450 |
| - "training_job_using_simplified_create = TrainingJob.create(\n", |
451 |
| - " training_job_name='xgboost-iris-' + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime()),\n", |
452 |
| - " hyper_parameters={\n", |
453 |
| - " 'objective': 'multi:softmax',\n", |
454 |
| - " 'num_class': '3',\n", |
455 |
| - " 'num_round': '10',\n", |
456 |
| - " 'eval_metric': 'merror'\n", |
457 |
| - " },\n", |
458 |
| - " training_image=image,\n", |
459 |
| - " training_input_mode='File',\n", |
460 |
| - " role_arn=role,\n", |
461 |
| - " input_data_config=[\n", |
462 |
| - " CustomChannel(\n", |
463 |
| - " channel_name='train',\n", |
464 |
| - " content_type='csv',\n", |
465 |
| - " compression_type='None',\n", |
466 |
| - " record_wrapper_type='None',\n", |
467 |
| - " data_source=CustomS3DataSource(\n", |
468 |
| - " s3_data_type='S3Prefix',\n", |
469 |
| - " s3_uri=s3_input_path,\n", |
470 |
| - " s3_data_distribution_type='FullyReplicated'\n", |
471 |
| - " )\n", |
472 |
| - " )\n", |
473 |
| - " ],\n", |
474 |
| - " output_data_config_s3_output_path=s3_output_path,\n", |
475 |
| - " instance_type='ml.m4.xlarge',\n", |
476 |
| - " instance_count=1,\n", |
477 |
| - " volume_size_in_gb=30,\n", |
478 |
| - " stopping_condition_max_runtime=600,\n", |
479 |
| - ")\n", |
480 |
| - "```\n", |
481 |
| - "Note: Not all structures are completely de-nested. There are technical blockers to de-nest list structures.\n", |
482 |
| - "\n", |
483 |
| - "This would reduce the number of lines of code and number of class structures to be configured by users to improvise the experience.\n", |
484 |
| - "\n", |
485 |
| - "Further we can set SDK specific defaults [not to be confused with intelligent defaults which are User set defaults] like the following to make parameters optional, to reduce the number of user-configured-params further:\n", |
486 |
| - "\n", |
487 |
| - "```\n", |
488 |
| - "training_job_name=training_image + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime()) + Unique_ID,\n", |
489 |
| - "volume_size_in_gb=5\n", |
490 |
| - "instance_count=1\n", |
491 |
| - "enable_network_isolation=False,\n", |
492 |
| - "enable_inter_container_traffic_encryption=False,\n", |
493 |
| - "enable_managed_spot_training=False,\n", |
494 |
| - "retry_strategy_max_retry=3,\n", |
495 |
| - "enable_remote_debug=False,\n", |
496 |
| - "enable_infra_check=False,\n", |
497 |
| - "```\n", |
498 |
| - "\n", |
499 |
| - "The above experience can be provided at the very Basic auto-gen layer resource classes by modifying the experience of TrainingJob.create [Option1] or provide a seperate TrainingJob.create_lite alternate interface [Option2]. Details on both approaches and pros/cons are discussed in detail in https://quip-amazon.com/QH1wAlPEsEPW/Simplifying-sagemaker-core-resource-classes.\n", |
500 |
| - "\n", |
501 |
| - "Though the above simplifications would extremely benefit the Core layer user experience, there are concerns around the customer positioning where in if we were to introduce a higher level training abstraction (like Estimator) we would be introducing multiple abstracted choices, making it difficult for customers to learn and use.\n", |
502 |
| - "\n", |
503 |
| - "With this in mind please share your thoughts in this [sim](https://issues.amazon.com/issues/CODEX-41935) comments whether this thin layer of abstraction can co-exist with higher level abstractions. Or +1 the sim feature request if you support.\n", |
504 |
| - "\n", |
505 |
| - "\n", |
506 |
| - "\n", |
507 |
| - "\n", |
508 |
| - "\n", |
509 |
| - "\n", |
510 |
| - "\n" |
511 |
| - ] |
512 |
| - }, |
513 | 441 | {
|
514 | 442 | "cell_type": "markdown",
|
515 | 443 | "metadata": {},
|
|
0 commit comments