Skip to content

ipywidgets #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ipywidgets #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
392 changes: 392 additions & 0 deletions demo-notebooks/guided-demos/ipywidgets.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,392 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "8d4a42f6",
"metadata": {},
"source": [
"In this notebook, we will go through the basics of using the SDK to:\n",
" - Spin up a Ray cluster with our desired resources\n",
" - View the status and specs of our Ray cluster\n",
" - Take down the Ray cluster when finished"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "301094f1",
"metadata": {},
"outputs": [],
"source": [
"%pip uninstall codeflare-sdk -y\n",
"%pip install ../../dist/codeflare_sdk-0.0.0.dev0-py3-none-any.whl"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a",
"metadata": {},
"outputs": [],
"source": [
"# Import pieces from codeflare-sdk\n",
"from codeflare_sdk import Cluster, ClusterConfiguration, TokenAuthentication, list_cluster_details"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "614daa0c",
"metadata": {},
"outputs": [],
"source": [
"# Create authentication object for user permissions\n",
"# IF unused, SDK will automatically check for default kubeconfig, then in-cluster config\n",
"# KubeConfigFileAuthentication can also be used to specify kubeconfig path manually\n",
"auth = TokenAuthentication(\n",
" token = \"XXXXX\",\n",
" server = \"XXXXX\",\n",
" skip_tls=False\n",
")\n",
"auth.login()"
]
},
{
"cell_type": "markdown",
"id": "bc27f84c",
"metadata": {},
"source": [
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
"\n",
"NOTE: We must specify the `image` which will be used in our RayCluster, we recommend you bring your own image which suits your purposes. \n",
"The example here is a community image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f4bc870-091f-4e11-9642-cba145710159",
"metadata": {},
"outputs": [],
"source": [
"# Create and configure our cluster object\n",
"# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
"cluster = Cluster(ClusterConfiguration(\n",
" name='raytest1', \n",
" namespace='default', # Update to your namespace\n",
" head_gpus=0, # For GPU enabled workloads set the head_gpus and num_gpus\n",
" num_gpus=0,\n",
" num_workers=1,\n",
" min_cpus=1,\n",
" max_cpus=1,\n",
" min_memory=2,\n",
" max_memory=2,\n",
" image=\"quay.io/rhoai/ray:2.23.0-py39-cu121\",\n",
" write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
" # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
"))"
]
},
{
"cell_type": "markdown",
"id": "12eef53c",
"metadata": {},
"source": [
"Next, we want to bring our cluster up, so we call the `up()` function below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200",
"metadata": {},
"outputs": [],
"source": [
"# Bring up the cluster\n",
"cluster.up()"
]
},
{
"cell_type": "markdown",
"id": "657ebdfb",
"metadata": {},
"source": [
"Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "df71c1ed",
"metadata": {},
"outputs": [],
"source": [
"def format_status(status):\n",
" if status == \"Ready\":\n",
" return '<span style=\"color: green;\">Ready ✓</span>'\n",
" elif status == \"Suspended\":\n",
" return '<span style=\"color: orange;\">Suspended ~</span>'\n",
" elif status == \"Starting\":\n",
" return '<span style=\"color: purple;\">Starting ⌛</span>'\n",
" elif status == \"Failed\":\n",
" return '<span style=\"color: red;\">Failed ✗</span>'\n",
" else:\n",
" return status\n",
"\n",
"import ipywidgets as widgets\n",
"import pandas as pd\n",
"from IPython.display import display, HTML\n",
"data = {\n",
" \"name\": [\"RayTest1\", \"RayTest2\", \"RayTest3\", \"RayTest4\"],\n",
" \"namespace\": [\"default\", \"usernamespace\", \"usernamespace\", \"usernamespace\"],\n",
" \"head_gpu\": [0, 1, 2, 0],\n",
" \"worker_gpu\": [2, 0, 1, 0],\n",
" \"min_memory\": [2, 4, 4, 2],\n",
" \"max_memory\": [2, 4, 8, 4],\n",
" \"min_cpu\": [1, 2, 4, 2],\n",
" \"max_cpu\": [1, 4, 8, 2],\n",
" \"status\": [\"Ready\", \"Starting\", \"Suspended\", \"Failed\"]\n",
"}\n",
"df = pd.DataFrame(data)\n",
"\n",
"# format to add icons\n",
"df['status'] = df['status'].apply(format_status)\n",
"\n",
"my_output = widgets.Output()\n",
"my_output\n",
"classification_widget = widgets.ToggleButtons(\n",
" options=['RayTest1', \"RayTest2\", \"RayTest3\", \"RayTest4\"],\n",
" description='Select an existing cluster:',\n",
")\n",
"\n",
"def on_click(change):\n",
" new_value = change[\"new\"]\n",
" my_output.clear_output()\n",
" with my_output:\n",
" display(HTML(df[df[\"name\"]==new_value][[\"name\", \"namespace\", \"head_gpu\", \"worker_gpu\", \"min_memory\", \"max_memory\", \"min_cpu\", \"max_cpu\", \"status\"]].to_html(escape=False, index=False, border=2)))\n",
"\n",
"classification_widget.observe(on_click, names=\"value\")\n",
"display(widgets.VBox([classification_widget, my_output]))\n",
"\n",
"\n",
"list_jobs_button = widgets.Button(\n",
" description='View Jobs',\n",
" icon='suitcase'\n",
" )\n",
"delete_button = widgets.Button(\n",
" description='Delete Cluster',\n",
" icon='trash'\n",
" )\n",
"ray_dashboard_button = widgets.Button(\n",
" description='Open Ray Dashboard',\n",
" icon='dashboard',\n",
" layout=widgets.Layout(width='auto'),\n",
" )\n",
"view_yaml_button = widgets.Button(\n",
" description='View YAML',\n",
" icon='file'\n",
" )\n",
"display(widgets.HBox([delete_button, list_jobs_button, view_yaml_button, ray_dashboard_button]))\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "24e612ff",
"metadata": {},
"outputs": [],
"source": [
"list_cluster_details()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9dda874b",
"metadata": {},
"outputs": [],
"source": [
"def format_status(status):\n",
" if status == \"Ready\":\n",
" return '<span style=\"color: green;\">Ready ✓</span>'\n",
" elif status == \"Suspended\":\n",
" return '<span style=\"color: orange;\">Suspended ~</span>'\n",
" elif status == \"Starting\":\n",
" return '<span style=\"color: purple;\">Starting ⌛</span>'\n",
" elif status == \"Failed\":\n",
" return '<span style=\"color: red;\">Failed ✗</span>'\n",
" else:\n",
" return status\n",
"\n",
"import ipywidgets as widgets\n",
"import pandas as pd\n",
"from IPython.display import display, HTML\n",
"data = {\n",
" \"name\": [\"RayTest1\", \"RayTest2\", \"RayTest3\", \"RayTest4\"],\n",
" \"namespace\": [\"default\", \"usernamespace\", \"usernamespace\", \"usernamespace\"],\n",
" \"head_gpu\": [0, 1, 2, 0],\n",
" \"worker_gpu\": [2, 0, 1, 0],\n",
" \"min_memory\": [2, 4, 4, 2],\n",
" \"max_memory\": [2, 4, 8, 4],\n",
" \"min_cpu\": [1, 2, 4, 2],\n",
" \"max_cpu\": [1, 4, 8, 2],\n",
" \"status\": [\"Ready\", \"Starting\", \"Suspended\", \"Failed\"],\n",
" \"pods\": [\n",
" [{\"pod\": \"head\", \"name\": \"head-raytest1\", \"status\": \"Ready\"}, {\"pod\": \"worker\", \"name\": \"worker-raytest1-a\", \"status\": \"Ready\"}, {\"pod\": \"worker\", \"name\": \"worker-raytest1-b\", \"status\": \"Ready\"}],\n",
" [{\"pod\": \"head\", \"name\": \"head-raytest2\", \"status\": \"Ready\"}, {\"pod\": \"worker\", \"name\": \"worker-raytest2a\", \"status\": \"Starting\"}],\n",
" [{\"pod\": \"head\", \"name\": \"head-raytest3\", \"status\": \"Suspended\"}, {\"pod\": \"worker\", \"name\": \"worker-raytest3a\", \"status\": \"Suspended\"}],\n",
" [{\"pod\": \"head\", \"name\": \"head-raytest4\", \"status\": \"Failed\"}, {\"pod\": \"worker\", \"name\": \"worker-raytest4a\", \"status\": \"Failed\"}]\n",
" ]\n",
"}\n",
"df = pd.DataFrame(data)\n",
"\n",
"# format to add icons\n",
"df['status'] = df['status'].apply(format_status)\n",
"\n",
"my_output = widgets.Output()\n",
"my_output\n",
"classification_widget = widgets.ToggleButtons(\n",
" options=['RayTest1', \"RayTest2\", \"RayTest3\", \"RayTest4\"],\n",
" description='Select an existing cluster:',\n",
")\n",
"\n",
"def on_click(change):\n",
" new_value = change[\"new\"]\n",
" my_output.clear_output()\n",
" with my_output:\n",
" selected_data = df[df[\"name\"] == new_value]\n",
" main_table = selected_data[[\"name\", \"namespace\", \"head_gpu\", \"worker_gpu\", \"min_memory\", \"max_memory\", \"min_cpu\", \"max_cpu\", \"status\"]].to_html(escape=False, index=False)\n",
" pod_rows = \"\"\n",
" for pod in selected_data[\"pods\"].values[0]:\n",
" pod_rows += f'<tr><td>{pod[\"pod\"]}</td><td>{pod[\"name\"]}</td><td>{format_status(pod[\"status\"])}</td></tr>'\n",
" pods_table = f'<div style=\"border:1px solid black; margin-top: 10px; margin-left: 10px; display: inline-block;\"><table><tr><th>Pod</th><th>Name</th><th>Status</th></tr>{pod_rows}</table></div>'\n",
" display(HTML(f'<div style=\"border:1px solid black; display: inline-block; padding-bottom: 10px;\">{main_table}{pods_table}</div>'))\n",
"\n",
"classification_widget.observe(on_click, names=\"value\")\n",
"display(widgets.VBox([classification_widget, my_output]))\n",
"\n",
"\n",
"list_jobs_button = widgets.Button(\n",
" description='View Jobs',\n",
" icon='suitcase'\n",
" )\n",
"delete_button = widgets.Button(\n",
" description='Delete Cluster',\n",
" icon='trash'\n",
" )\n",
"ray_dashboard_button = widgets.Button(\n",
" description='Open Ray Dashboard',\n",
" icon='dashboard',\n",
" layout=widgets.Layout(width='auto'),\n",
" )\n",
"view_yaml_button = widgets.Button(\n",
" description='View YAML',\n",
" icon='file'\n",
" )\n",
"display(widgets.HBox([delete_button, list_jobs_button, view_yaml_button, ray_dashboard_button]))"
]
},
{
"cell_type": "markdown",
"id": "b3a55fe4",
"metadata": {},
"source": [
"Let's quickly verify that the specs of the cluster are as expected."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f1ab7ff",
"metadata": {},
"outputs": [],
"source": [
"with my_output:\n",
" display(cluster.details())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084",
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import HTML, display\n",
"import ipywidgets as widgets\n",
"\n",
"def on_click(change):\n",
" new_value = change[\"new\"]\n",
" my_output.clear_output()\n",
" with my_output:\n",
" display(HTML(f'<div style=\"border:1px solid black;\">{df[df[\"name\"]==new_value][[\"name\", \"namespace\", \"head_gpu\", \"worker_gpu\", \"min_memory\", \"max_memory\", \"min_cpu\", \"max_cpu\", \"status\"]].to_html(escape=False, index=False)}</div>'))\n",
"\n",
"classification_widget.observe(on_click, names=\"value\")\n",
"display(widgets.VBox([classification_widget, my_output], layout=widgets.Layout(border='2px solid black')))\n",
"\n",
"list_jobs_button = widgets.Button(description='View Jobs', icon='suitcase')\n",
"delete_button = widgets.Button(description='Delete Cluster', icon='trash')\n",
"ray_dashboard_button = widgets.Button(description='Open Ray Dashboard', icon='dashboard', layout=widgets.Layout(width='auto'))\n",
"view_yaml_button = widgets.Button(description='View YAML', icon='file')\n",
"buttons_container = widgets.HBox([delete_button, list_jobs_button, view_yaml_button, ray_dashboard_button], layout=widgets.Layout(border='2px solid black'))\n",
"\n",
"display(buttons_container)\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "5af8cd32",
"metadata": {},
"source": [
"Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57",
"metadata": {},
"outputs": [],
"source": [
"cluster.down()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d41b90e",
"metadata": {},
"outputs": [],
"source": [
"auth.logout()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
},
"vscode": {
"interpreter": {
"hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading