Skip to content

Commit 8bc72cf

Browse files
mollyheamazonrsareddy0329
mollyheamazon
authored and
rsareddy0329
committed
add example notebooks for init experience, update README to match with new documentation (#250)
* add example notebooks for init experience, update README to match with new documentation * clear output
1 parent 7697ded commit 8bc72cf

9 files changed

+1539
-106
lines changed

‎README.md‎

Lines changed: 503 additions & 81 deletions
Large diffs are not rendered by default.

‎examples/inference/CLI/inference-fsx-model-e2e-cli.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"id": "2d55c8b9",
66
"metadata": {},
77
"source": [
8-
"## Inference Operator CLI E2E Expereience (S3 custom model)"
8+
"## Inference Operator CLI E2E Expereience (FSX custom model)"
99
]
1010
},
1111
{

‎examples/inference/CLI/inference-jumpstart-init-experience.ipynb‎

Lines changed: 323 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,323 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# SageMaker HyperPod Jumpstart Endpoint - Init Experience\n",
8+
"\n",
9+
"This notebook demonstrates the complete end-to-end workflow for creating a SageMaker HyperPod Jumpstart Endpoint using the HyperPod CLI. The init experience provides a guided approach to create Hyperpod Jumpstart Endpoint with validation and configuration management.\n",
10+
"\n",
11+
"## Prerequisites\n",
12+
"\n",
13+
"- SageMaker HyperPod CLI installed (`pip install sagemaker-hyperpod`)\n",
14+
"- Hyperpod jumpstart inference template installed (`pip install hyperpod-jumpstart-inference-template`)\n",
15+
"- Hyperpod inference operator installed in your hyperpod cluster\n",
16+
"- Python 3.8+ environment\n",
17+
"\n",
18+
"## Workflow Overview\n",
19+
"\n",
20+
"1. **Initialize** - Create initial jumpstart endpoint configuration\n",
21+
"2. **Configure** - Customize jumpstart endpoint parameters\n",
22+
"3. **Validate** - Verify configuration before deployment\n",
23+
"4. **Create** - Deploy the jumpstart endpoint creation\n",
24+
"5. **Monitor** - Check jumpstart endpoint status and manage lifecycle\n"
25+
]
26+
},
27+
{
28+
"cell_type": "markdown",
29+
"metadata": {},
30+
"source": [
31+
"## Step 0: Connect to your Hyperpod cluster\n",
32+
"\n",
33+
"Make sure you have installed hyperpod inference operator in your hyperpod cluster.\n"
34+
]
35+
},
36+
{
37+
"cell_type": "code",
38+
"execution_count": null,
39+
"metadata": {},
40+
"outputs": [],
41+
"source": [
42+
"# List all available SageMaker HyperPod clusters in your account\n",
43+
"!hyp list-cluster"
44+
]
45+
},
46+
{
47+
"cell_type": "code",
48+
"execution_count": null,
49+
"metadata": {},
50+
"outputs": [],
51+
"source": [
52+
"# Configure your local kubectl environment to interact with a specific SageMaker HyperPod cluster (and namespace)\n",
53+
"!hyp set-cluster-context --cluster-name ml-cluster-integ-test"
54+
]
55+
},
56+
{
57+
"cell_type": "markdown",
58+
"metadata": {},
59+
"source": [
60+
"## Step 1: Initialize Jumpstart Endpoint Configuration\n",
61+
"\n",
62+
"The `hyp init hyp-jumpstart-endpoint` command creates a new configuration template with default settings. This generates a `config.yaml` file that serves as the foundation for your deployment.\n",
63+
"\n",
64+
"**What this does:**\n",
65+
"- Creates a `config.yaml` with default jumpstart endpoint settings.\n",
66+
"- Creates a `k8s.jinja` which is a reference to the k8s payload that is going to be submitted with. Users can refer this to understand how the parameters are being used. \n",
67+
"- Creates a `README.md` which is a detailed explanation of the init experience.\n"
68+
]
69+
},
70+
{
71+
"cell_type": "code",
72+
"execution_count": null,
73+
"metadata": {},
74+
"outputs": [],
75+
"source": [
76+
"# Initialize a new jumpstart endpoint configuration in the current directory\n",
77+
"!hyp init hyp-jumpstart-endpoint"
78+
]
79+
},
80+
{
81+
"cell_type": "markdown",
82+
"metadata": {},
83+
"source": [
84+
"## Step 2: Configure Jumpstart Endpoint Settings\n",
85+
"\n",
86+
"The `hyp configure` command allows you to customize your jumpstart endpoint configuration.\n",
87+
"\n",
88+
"**Key configuration options:**\n",
89+
"- **model_id**: Unique identifier of the model within the SageMakerPublicHub\n",
90+
"- **instance_type**: EC2 instance type for the inference server\n",
91+
"- **endpoint_name**: Name of SageMaker endpoint"
92+
]
93+
},
94+
{
95+
"cell_type": "code",
96+
"execution_count": null,
97+
"metadata": {},
98+
"outputs": [],
99+
"source": [
100+
"!hyp configure --endpoint-name my-jumpstart-endpoint"
101+
]
102+
},
103+
{
104+
"cell_type": "markdown",
105+
"metadata": {},
106+
"source": [
107+
"### View Current Configuration\n",
108+
"\n",
109+
"Let's examine the generated configuration to understand what will be deployed:"
110+
]
111+
},
112+
{
113+
"cell_type": "code",
114+
"execution_count": null,
115+
"metadata": {},
116+
"outputs": [],
117+
"source": [
118+
"# Display the current configuration\n",
119+
"!cat config.yaml | head -50"
120+
]
121+
},
122+
{
123+
"cell_type": "markdown",
124+
"metadata": {},
125+
"source": [
126+
"## Step 3: Validate Configuration\n",
127+
"\n",
128+
"The `hyp validate` command performs syntax validation of your jumpstart endpoint configuration before deployment. This helps catch configuration errors early and ensures all prerequisites are met.\n"
129+
]
130+
},
131+
{
132+
"cell_type": "code",
133+
"execution_count": null,
134+
"metadata": {},
135+
"outputs": [],
136+
"source": [
137+
"# Validate the jumpstart endpoint configuration\n",
138+
"# This checks for potential issues before deployment\n",
139+
"!hyp validate"
140+
]
141+
},
142+
{
143+
"cell_type": "markdown",
144+
"metadata": {},
145+
"source": [
146+
"## Step 4: Reset Configuration (Optional)\n",
147+
"\n",
148+
"The `hyp reset` command allows you to reset your configuration to defaults or clean up any partial deployments. This is useful when you want to start fresh or if validation reveals issues that require a clean slate.\n",
149+
"\n",
150+
"**Use cases for reset:**\n",
151+
"- Starting over with a clean configuration\n",
152+
"- Cleaning up after failed deployments\n",
153+
"- Switching between different jumpstart endpoint configurations\n"
154+
]
155+
},
156+
{
157+
"cell_type": "code",
158+
"execution_count": null,
159+
"metadata": {},
160+
"outputs": [],
161+
"source": [
162+
"# Reset configuration if needed (uncomment to use)\n",
163+
"# !hyp reset\n",
164+
"\n",
165+
"print(\"Reset command available if configuration changes are needed\")"
166+
]
167+
},
168+
{
169+
"cell_type": "markdown",
170+
"metadata": {},
171+
"source": [
172+
"## Step 5: Create the Jumpstart Endpoint\n",
173+
"\n",
174+
"The `hyp create` command deploys your HyperPod jumpstart endpoint with configurations in the config.yaml. A timestamped folder is created in the `runs` folder, where the config.yaml and the values-injected k8s.yaml kubernates payload is saved.\n",
175+
"\n",
176+
"**Note:** The sagemaker jumpstart endpoint typically takes 10-15 minutes to be created.\n"
177+
]
178+
},
179+
{
180+
"cell_type": "code",
181+
"execution_count": null,
182+
"metadata": {},
183+
"outputs": [],
184+
"source": [
185+
"# Create the jumpstart endpoint\n",
186+
"!hyp create"
187+
]
188+
},
189+
{
190+
"cell_type": "markdown",
191+
"metadata": {},
192+
"source": [
193+
"## Step 6: Monitor Jumpstart Endpoint Creation\n",
194+
"\n",
195+
"While the jumpstart endpoint is being created, you can monitor its progress using the describe and list commands. These provide real-time status updates on the deployment process."
196+
]
197+
},
198+
{
199+
"cell_type": "code",
200+
"execution_count": null,
201+
"metadata": {},
202+
"outputs": [],
203+
"source": [
204+
"# Check jumpstart endpoint creation status\n",
205+
"import time\n",
206+
"\n",
207+
"print(\"Monitoring jumpstart endpoint progress...\")\n",
208+
"for i in range(5):\n",
209+
" print(f\"\\n--- Status Check {i+1} ---\")\n",
210+
" !hyp describe hyp-jumpstart-endpoint --name my-jumpstart-endpoint\n",
211+
" time.sleep(30) # Wait 30 seconds between checks"
212+
]
213+
},
214+
{
215+
"cell_type": "markdown",
216+
"metadata": {},
217+
"source": [
218+
"## Step 7: Invoke Sagemaker Endpoint\n",
219+
"\n",
220+
"After the sagemaker endpoint is successfully created, you can use `hyp invoke hyp-jumpstart-endpoint` command to do basic invocation of sagemaker endpoint."
221+
]
222+
},
223+
{
224+
"cell_type": "code",
225+
"execution_count": null,
226+
"metadata": {},
227+
"outputs": [],
228+
"source": [
229+
"!hyp invoke hyp-jumpstart-endpoint --endpoint-name my-jumpstart-endpoint --body '{\"inputs\":\"What is the capital of USA?\"}'"
230+
]
231+
},
232+
{
233+
"cell_type": "markdown",
234+
"metadata": {},
235+
"source": [
236+
"## Step 8: Describe Jumpstart Endpoint\n",
237+
"\n",
238+
"The `hyp describe hyp-jumpstart-endpoint` command provides detailed information about your jumpstart endpoint deployment status and sagemaker endpoint status."
239+
]
240+
},
241+
{
242+
"cell_type": "code",
243+
"execution_count": null,
244+
"metadata": {},
245+
"outputs": [],
246+
"source": [
247+
"# Get detailed information about the jumpstart endpoint\n",
248+
"!hyp describe hyp-jumpstart-endpoint --name my-jumpstart-endpoint"
249+
]
250+
},
251+
{
252+
"cell_type": "markdown",
253+
"metadata": {},
254+
"source": [
255+
"## Step 9: List All Jumpstart Endpoints\n",
256+
"\n",
257+
"The `hyp list hyp-jumpstart-endpoint` command shows all HyperPod jumpstart endpoints in your account. This is useful for managing multiple jumpstart endpoint deployments and getting an overview of your deployments.\n"
258+
]
259+
},
260+
{
261+
"cell_type": "code",
262+
"execution_count": null,
263+
"metadata": {},
264+
"outputs": [],
265+
"source": [
266+
"# List all jumpstart endpoints in your account\n",
267+
"!hyp list hyp-jumpstart-endpoint"
268+
]
269+
},
270+
{
271+
"cell_type": "markdown",
272+
"metadata": {},
273+
"source": [
274+
"## Next Steps\n",
275+
"\n",
276+
"After successfully creating your HyperPod Jumpstart Endpoint, you can:\n",
277+
"\n",
278+
"1. **Monitor Resources**: Check pod status with `hyp list-pods hyp-jumpstart-endpoint`\n",
279+
"2. **Access Logs**: View pod logs with `hyp get-logs hyp-jumpstart-endpoint`\n",
280+
"\n",
281+
"\n",
282+
"## Troubleshooting\n",
283+
"\n",
284+
"If you encounter issues during Jumpstart Endpoint creation:\n",
285+
"\n",
286+
"- Use `hyp get-operator-logs hyp-jumpstart-endpoint` to check potential operator log errors\n",
287+
"- Verify AWS credentials and permissions\n",
288+
"- Ensure resource quotas are sufficient\n",
289+
"- Review the configuration file for syntax errors\n",
290+
"- Use `hyp validate` to identify configuration issues\n",
291+
"\n",
292+
"## Cleanup\n",
293+
"\n",
294+
"To avoid ongoing charges, remember to delete your jumpstart endpoint when no longer needed:\n",
295+
"\n",
296+
"```bash\n",
297+
"hyp delete hyp-jumpstart-endpoint --name my-jumpstart-endpoint\n",
298+
"```\n"
299+
]
300+
}
301+
],
302+
"metadata": {
303+
"kernelspec": {
304+
"display_name": "Python 3",
305+
"language": "python",
306+
"name": "python3"
307+
},
308+
"language_info": {
309+
"codemirror_mode": {
310+
"name": "ipython",
311+
"version": 3
312+
},
313+
"file_extension": ".py",
314+
"mimetype": "text/x-python",
315+
"name": "python",
316+
"nbconvert_exporter": "python",
317+
"pygments_lexer": "ipython3",
318+
"version": "3.12.2"
319+
}
320+
},
321+
"nbformat": 4,
322+
"nbformat_minor": 4
323+
}

0 commit comments

Comments
 (0)