Skip to content

Commit bb572f8

Browse files
committed
Update Blog “deploying-a-hugging-face-llm-in-hpe-private-cloud-ai”
1 parent 354ce23 commit bb572f8

File tree

1 file changed

+10
-10
lines changed

1 file changed

+10
-10
lines changed

content/blog/deploying-a-hugging-face-llm-in-hpe-private-cloud-ai.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,19 +11,19 @@ tags:
1111
- vllm
1212
- SLM
1313
---
14-
Deploying new language models for for users to interact with can be challenging for beginners. HPE has developed Private Cloud AI to help users get to productive AI work quickly and easily.
14+
Deploying new language models for for users to interact with can be challenging for beginners. HPE developed Private Cloud AI to help users set up and implement AI solutions quickly and easily.
1515

16-
Here we will show how to use HPE Machine Learning Inference Service as a part of HPE Private Cloud AI to add a new packaged model from a Hugging Face repository and create an endpoint to query the model. This is done in a Jupyter Notebook.
16+
In this post, we will show how to use the HPE Machine Learning Inference Service (MLIS) as a part of HPE Private Cloud AI to add a new packaged model from a Hugging Face repository and create an endpoint to query the model. This is done using a Jupyter Notebook.
1717

1818
### Prerequisites
1919

2020
This tutorial uses the [HPE Private Cloud AI](https://www.hpe.com/us/en/private-cloud-ai.html) (PCAI) platform. A PCAI system is required for these steps to work. It is assumed that the PCAI system is physically installed, patched and running with user accounts provisioned.
2121

22-
### Steps to Deploy
22+
### Steps to deploy
2323

24-
First we need to choose a model to deploy, in this case we've chosen a model hosted on Hugging Face called [SmolLM2 1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct). This is a compact model that can solve a wide range of problems even though it is relatively diminutive at 1.7B parameters.
24+
First, you will need to choose a model to deploy. In this case, we've chosen a model hosted on Hugging Face called [SmolLM2 1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct). This is a compact model that can solve a wide range of problems even though it is relatively diminutive at 1.7B parameters.
2525

26-
### Launching the Interface
26+
### Launching the interface
2727

2828
![Computer screen showing the HPE Private Cloud AI user interface and the HPE MLIS tile is highlighted.](/img/mlis.png)
2929

@@ -49,7 +49,7 @@ When the process is complete an endpoint will be provided.
4949

5050
![Endpoint provided by MLIS system](/img/endpoint.png)
5151

52-
Next up let's take the now deployed model that's ready for inference and connect to it and interact with it from a Jupyter Notebook.
52+
Next up, let's take the now deployed model that's ready for inference and connect to it and interact with it from a Jupyter Notebook.
5353

5454
### Building the Jupyter Notebook
5555

@@ -61,7 +61,7 @@ First install `openai` if you do not already have it and import.
6161
from openai import OpenAI
6262
```
6363

64-
Then using the endpoint and key genered by HPE MLIS enter them into your Jupyter Notebook. Be sure to append /v1 to the URL.
64+
Then using the endpoint and key generated by HPE MLIS enter them into your Jupyter Notebook. Be sure to append /v1 to the URL.
6565

6666
```python
6767
# Grab endpoint URL and API key from MLIS, remember to include "/v1" for latest version of the OpenAI-compatible API
@@ -70,7 +70,7 @@ openai_api_base = "https://smollm2-1-7b-vllm-predictor-dave-wright-hpe-1073f7cd.
7070
openai_api_key = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3Mzk5MzgzMzAsImlzcyI6ImFpb2xpQGhwZS5jb20iLCJzdWIiOiI5MjNhM2JhOC1mMGU4LTQxOTQtODNkMS05ZWY4NzNjZGYxOWYiLCJ1c2VyIjoiZGF2ZS53cmlnaHQtaHBlLmNvbSJ9.YwH9gGPxTWxy4RSdjnQA9-U3_u7P0OIcarqw25DV8bOiftU1L4IvvyERHspj2lMGtZWbff1F3uh84wjAePHaHDcDTLoGtq6gJYwo_qRU03xV8Q2lwBetCCLUE4OHqS608gjJ-j1SLyqwxFxlXkqMOtnBY5_nswlAwCzHV28P8u8XxxfWuXFmoJpSA1egCWVVfEoTuK8CTz9kUJJ5opSp6m8qdqJmC2qxH0igcpKmL2H_MZ-62UHfEf240VRtc0DRNlOjeCoDM79aVPs3SjCtGeVkeEHimJwJbfGFIcu3LibX3QjbABUzWb5BPPZjzyEYUVM5ak12_sJ8j1mUW-r0sA"
7171
```
7272

73-
Now we need an OpenAI client interface.
73+
Now, we need an OpenAI client interface.
7474

7575
```python
7676
# create OpenAI client interface
@@ -124,9 +124,9 @@ def chat():
124124

125125
![Jupyter Notebook showing imported model endpoint and API key.](/img/jupyter.png)
126126

127-
Then interacting with the model is done through a simple chat.
127+
Then, interacting with the model is done through a simple chat.
128128

129-
![Interaction with the SmolLM2 small language model in a Jupyter Notebook](/img/chat-interface.png)
129+
![Interaction with the SmolLM2 Small Language Model in a Jupyter Notebook](/img/chat-interface.png)
130130

131131
For a recorded demonstration that shows the process real time see this screen recording. <https://www.youtube.com/watch?v=oqjc-2c1Vtk>
132132

0 commit comments

Comments
 (0)