Skip to content

Commit 75245a1

Browse files
santhoshb-msftsanthoshb-msft
authored
Update deploy-models-jamba.md
fixed feedback from TG
1 parent 0b4f365 commit 75245a1

File tree

1 file changed

+40
-29
lines changed

1 file changed

+40
-29
lines changed

‎articles/ai-studio/how-to/deploy-models-jamba.md

Lines changed: 40 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -21,30 +21,24 @@ In this article, you learn how to use Azure AI Studio to deploy AI21's Jamba fam
2121
The Jamba family models are AI21's production-grade Mamba-based large language model (LLM) which leverages AI21's hybrid Mamba-Transformer architecture. It's an instruction-tuned version of AI21's hybrid structured state space model (SSM) transformer Jamba model. The Jamba family models are built for reliable commercial use with respect to quality and performance.
2222

2323
> [!TIP]
24-
> See our announcements of AI21's Jamba family models available now on Azure AI Model Catalog through [AI21's blog](https://aka.ms/ai21-jamba-instruct-blog) and [Microsoft Tech Community Blog](https://aka.ms/ai21-jamba-instruct-announcement).
24+
> See our announcements of AI21's Jamba family models available now on Azure AI Model Catalog through [AI21's blog](https://aka.ms/ai21-jamba-1.5-large-announcement) and [Microsoft Tech Community Blog](https://aka.ms/ai21-jamba-1.5-large-microsoft-annnouncement).
2525
2626
## Deploy the Jamba family models as a serverless API
2727

2828
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing, providing a way to consume them as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
2929

30-
# [AI21 Jamba 1.5 Mini](#tab/ai21-jamba-1-5)
31-
32-
The [AI21 Jamba 1.5 mini model](https://aka.ms/aistudio/landing/ai21-labs-jamba-1.5) deployed as a serverless API with pay-as-you-go billing is [offered by AI21 through Microsoft Azure Marketplace](https://aka.ms/azure-marketplace-offer-ai21-jamba-1.5). AI21 can change or update the terms of use and pricing of this model.
33-
34-
To get started with Jamba 1.5 mini deployed as a serverless API, explore our integrations with [LangChain](https://aka.ms/ai21-jamba-1.5-langchain-sample), [LiteLLM](https://aka.ms/ai21-jamba-1.5-litellm-sample), [OpenAI](https://aka.ms/ai21-jamba-1.5-openai-sample) and the [Azure API](https://aka.ms/ai21-jamba-1.5-azure-api-sample).
35-
3630
# [AI21 Jamba 1.5 Large](#tab/ai21-jamba-1-5-large)
3731

38-
The [AI21-Jamba 1.5 large model](https://aka.ms/aistudio/landing/ai21-labs-jamba-1.5-large) deployed as a serverless API with pay-as-you-go billing is [offered by AI21 through Microsoft Azure Marketplace](https://aka.ms/azure-marketplace-offer-ai21-jamba-1.5-large). AI21 can change or update the terms of use and pricing of this model.
32+
The [AI21-Jamba 1.5 Large model](https://aka.ms/aistudio/landing/ai21-labs-jamba-1.5-large) deployed as a serverless API with pay-as-you-go billing is [offered by AI21 through Microsoft Azure Marketplace](https://aka.ms/azure-marketplace-offer-ai21-jamba-1.5-large). AI21 can change or update the terms of use and pricing of this model.
3933

4034
To get started with Jamba 1.5 large deployed as a serverless API, explore our integrations with [LangChain](https://aka.ms/ai21-jamba-1.5-large-langchain-sample), [LiteLLM](https://aka.ms/ai21-jamba-1.5-large-litellm-sample), [OpenAI](https://aka.ms/ai21-jamba-1.5-large-openai-sample) and the [Azure API](https://aka.ms/ai21-jamba-1.5-large-azure-api-sample).
4135

4236

43-
# [AI21 Jamba Instruct](#tab/ai21-jamba-instruct)
37+
# [AI21 Jamba 1.5 Mini](#tab/ai21-jamba-1-5)
4438

45-
The [AI21 Jamba Instruct model](https://aka.ms/aistudio/landing/ai21-labs-jamba-instruct) deployed as a serverless API with pay-as-you-go billing is [offered by AI21 through Microsoft Azure Marketplace](https://aka.ms/azure-marketplace-offer-ai21-jamba-instruct). AI21 can change or update the terms of use and pricing of this model.
39+
The [AI21 Jamba 1.5 Mini model](https://aka.ms/aistudio/landing/ai21-labs-jamba-1.5-mini) deployed as a serverless API with pay-as-you-go billing is [offered by AI21 through Microsoft Azure Marketplace](https://aka.ms/azure-marketplace-offer-ai21-jamba-1.5-mini). AI21 can change or update the terms of use and pricing of this model.
4640

47-
To get started with Jamba Instruct deployed as a serverless API, explore our integrations with [LangChain](https://aka.ms/ai21-jamba-instruct-langchain-sample), [LiteLLM](https://aka.ms/ai21-jamba-instruct-litellm-sample), [OpenAI](https://aka.ms/ai21-jamba-instruct-openai-sample) and the [Azure API](https://aka.ms/ai21-jamba-instruct-azure-api-sample).
41+
To get started with Jamba 1.5 mini deployed as a serverless API, explore our integrations with [LangChain](https://aka.ms/ai21-jamba-1.5-mini-langchain-sample), [LiteLLM](https://aka.ms/ai21-jamba-1.5-mini-litellm-sample), [OpenAI](https://aka.ms/ai21-jamba-1.5-mini-openai-sample) and the [Azure API](https://aka.ms/ai21-jamba-1.5-mini-azure-api-sample).
4842

4943
---
5044

@@ -85,16 +79,16 @@ To get started with Jamba Instruct deployed as a serverless API, explore our int
8579

8680
### Create a new deployment
8781

88-
These steps demonstrate the deployment of AI21-Jamba family models. To create a deployment:
82+
These steps demonstrate the deployment of `AI21 Jamba 1.5 Large` or `AI21 Jamba 1.5 Mini` models. To create a deployment:
8983

9084
1. Sign in to [Azure AI Studio](https://ai.azure.com).
9185
1. Select **Model catalog** from the left sidebar.
92-
1. Search for and select a AI21 model like `AI21 Jamba 1.5 Mini` or `AI21 Jamba 1.5 Large` or `AI21 Jamba Instruct` to open its Details page.
86+
1. Search for and select a AI21 model like `AI21 Jamba 1.5 Large` or `AI21 Jamba 1.5 Mini` or `AI21 Jamba Instruct` to open its Details page.
9387
1. Select **Deploy** to open a serverless API deployment window for the model.
9488
1. Alternatively, you can initiate a deployment by starting from your project in AI Studio.
9589
1. From the left sidebar of your project, select **Components** > **Deployments**.
9690
1. Select **+ Create deployment**.
97-
1. Search for and select a AI21 model like `AI21 Jamba 1.5 Mini` or `AI21 Jamba 1.5 Large` or `AI21 Jamba Instruct` to open the Model's Details page.
91+
1. Search for and select a AI21 model like `AI21 Jamba 1.5 Large` or `AI21 Jamba 1.5 Mini` or `AI21 Jamba Instruct` to open the Model's Details page.
9892
1. Select **Confirm** to open a serverless API deployment window for the model.
9993
1. Select the project in which you want to deploy your model. To deploy the AI21-Jamba family models, your project must be in one of the regions listed in the [Prerequisites](#prerequisites) section.
10094
1. In the deployment wizard, select the link to **Azure Marketplace Terms**, to learn more about the terms of use.
@@ -177,7 +171,7 @@ Payload is a JSON formatted string containing the following parameters:
177171

178172
| Key | Type | Required/Default | Allowed values | Description |
179173
| ------------- | -------------- | :-----------------:| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
180-
| `model` | `string` | Y | Must be `jamba-1.5` or `jamba-1.5-large` or `jamba-instruct`|
174+
| `model` | `string` | Y | Must be `jamba-1.5-large` or `jamba-1.5-mini` or `jamba-instruct`|
181175
| `messages` | `list[object]` | Y | A list of objects, one per message, from oldest to newest. The oldest message can be role `system`. All later messages must alternate between user and assistant roles. See the message object definition below.|
182176
| `max_tokens` | `integer` | N <br>`4096` | 0 – 4096 | The maximum number of tokens to allow for each generated response message. Typically the best way to limit output length is by providing a length limit in the system prompt (for example, "limit your answers to three sentences")|
183177
| `temperature` | `float` | N <br>`1` | 0.0 – 2.0 | How much variation to provide in each answer. Setting this value to 0 guarantees the same response to the same question every time. Setting a higher value encourages more variation. Modifies the distribution from which tokens are sampled. We recommend altering this or `top_p`, but not both. |
@@ -212,11 +206,11 @@ The `document` object has the following fields:
212206

213207
#### Request example
214208

215-
__Single-turn example Jamba 1.5 mini and Jamba 1.5 large__
209+
__Single-turn example Jamba 1.5 large and Jamba 1.5 mini__
216210

217211
```JSON
218212
{
219-
"model_name":"model-name", <jamba-1.5|jamba-1.5-large>
213+
"model":"jamba-1.5-mini", <jamba-1.5-large|jamba-1.5-mini>
220214
"messages":[
221215
{
222216
"role":"user",
@@ -230,27 +224,44 @@ __Single-turn example Jamba 1.5 mini and Jamba 1.5 large__
230224
"stream":false
231225
}
232226
```
233-
234-
__Single-turn example Jamba Instruct__
227+
__Single-turn example Jamba 1.5 large and Jamba 1.5 mini with documents__
235228

236229
```JSON
237230
{
238-
"model": "jamba-instruct",
239-
"messages": [
240-
{
241-
"role":"user",
242-
"content":"Who was the first emperor of rome?"}
243-
],
244-
"temperature": 0.8,
245-
"max_tokens": 512
231+
"model":"jamba-1.5-mini", <jamba-1.5-large|jamba-1.5-mini>
232+
"messages":[
233+
{
234+
"role":"system",
235+
"content":'''<documents>
236+
# Documents
237+
238+
You can use the following documents for reference:
239+
240+
## Document ID: 0
241+
Text: Harry Potter is a series of seven fantasy novels written by British author J. K. Rowling.
242+
243+
## Document ID: 1
244+
Text: The Great Gatsby is a novel by American writer F. Scott Fitzgerald.
245+
</documents>'''},
246+
247+
{
248+
"role":"user",
249+
"content":"Who wrote Harry Potter?"
250+
}
251+
],
252+
"temperature":0.4,
253+
"top_p":1,
254+
"n":1,
255+
"stop":"\n",
256+
"stream":false
246257
}
247258
```
248259

249260
__Chat example (fourth request containing third user response)__
250261

251262
```JSON
252263
{
253-
"model": "model-name", <jamba-1.5|jamba-1.5-large|jamba-instruct>
264+
"model": "jamba-instruct",
254265
"messages": [
255266
{"role": "system",
256267
"content": "You are a helpful genie just released from a bottle. You start the conversation with 'Thank you for freeing me! I grant you one wish.'"},
@@ -389,4 +400,4 @@ Models deployed as a serverless API are protected by Azure AI content safety. Wi
389400

390401
- [What is Azure AI Studio?](../what-is-ai-studio.md)
391402
- [Azure AI FAQ article](../faq.yml)
392-
- [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)
403+
- [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)

0 commit comments

Comments
 (0)