You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/blog/deploying-a-hugging-face-llm-in-hpe-private-cloud-ai.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,19 +11,19 @@ tags:
11
11
- vllm
12
12
- SLM
13
13
---
14
-
Deploying new language models for for users to interact with can be challenging for beginners. HPE has developed Private Cloud AI to help users get to productive AI work quickly and easily.
14
+
Deploying new language models for for users to interact with can be challenging for beginners. HPE developed Private Cloud AI to help users set up and implement AI solutions quickly and easily.
15
15
16
-
Here we will show how to use HPE Machine Learning Inference Service as a part of HPE Private Cloud AI to add a new packaged model from a Hugging Face repository and create an endpoint to query the model. This is done in a Jupyter Notebook.
16
+
In this post, we will show how to use the HPE Machine Learning Inference Service (MLIS) as a part of HPE Private Cloud AI to add a new packaged model from a Hugging Face repository and create an endpoint to query the model. This is done using a Jupyter Notebook.
17
17
18
18
### Prerequisites
19
19
20
20
This tutorial uses the [HPE Private Cloud AI](https://www.hpe.com/us/en/private-cloud-ai.html) (PCAI) platform. A PCAI system is required for these steps to work. It is assumed that the PCAI system is physically installed, patched and running with user accounts provisioned.
21
21
22
-
### Steps to Deploy
22
+
### Steps to deploy
23
23
24
-
First we need to choose a model to deploy, in this case we've chosen a model hosted on Hugging Face called [SmolLM2 1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct). This is a compact model that can solve a wide range of problems even though it is relatively diminutive at 1.7B parameters.
24
+
First, you will need to choose a model to deploy. In this case, we've chosen a model hosted on Hugging Face called [SmolLM2 1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct). This is a compact model that can solve a wide range of problems even though it is relatively diminutive at 1.7B parameters.
25
25
26
-
### Launching the Interface
26
+
### Launching the interface
27
27
28
28

29
29
@@ -49,7 +49,7 @@ When the process is complete an endpoint will be provided.
49
49
50
50

51
51
52
-
Next up let's take the now deployed model that's ready for inference and connect to it and interact with it from a Jupyter Notebook.
52
+
Next up, let's take the now deployed model that's ready for inference and connect to it and interact with it from a Jupyter Notebook.
53
53
54
54
### Building the Jupyter Notebook
55
55
@@ -61,7 +61,7 @@ First install `openai` if you do not already have it and import.
61
61
from openai import OpenAI
62
62
```
63
63
64
-
Then using the endpoint and key genered by HPE MLIS enter them into your Jupyter Notebook. Be sure to append /v1 to the URL.
64
+
Then using the endpoint and key generated by HPE MLIS enter them into your Jupyter Notebook. Be sure to append /v1 to the URL.
65
65
66
66
```python
67
67
# Grab endpoint URL and API key from MLIS, remember to include "/v1" for latest version of the OpenAI-compatible API
0 commit comments