Merge pull request #3176 from MicrosoftDocs/main

2/25/2025 PM Publish

Author: Taojunshen

articles/ai-foundry/model-inference/quotas-limits.md

@@ -17,7 +17,7 @@ This article contains a quick reference and a detailed description of the quotas
 
 ## Quotas and limits reference
 
-The following sections provide you with a quick guide to the default quotas and limits that apply to Azure AI model's inference service in Azure AI services:
+Azure uses quotas and limits to prevent budget overruns due to fraud, and to honor Azure capacity constraints. Consider these limits as you scale for production workloads. The following sections provide you with a quick guide to the default quotas and limits that apply to Azure AI model's inference service in Azure AI services:
 
 ### Resource limits
 
@@ -28,12 +28,18 @@ The following sections provide you with a quick guide to the default quotas and
 
 ### Rate limits
 
-| Limit name | Limit value |
-| ---------- | ----------- |
-| Tokens per minute (Azure OpenAI models)   | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
-| Tokens per minute (rest of models)        | 200.000 |
-| Requests per minute (Azure OpenAI models) | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
-| Requests per minute (rest of models)      | 1.000   |
+| Limit name           | Applies to          | Limit value |
+| -------------------- | ------------------- | ----------- |
+| Tokens per minute    | Azure OpenAI models | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
+| Requests per minute  | Azure OpenAI models | Varies per model and SKU. See [limits for Azure OpenAI](../../ai-services/openai/quotas-limits.md). |
+| Tokens per minute    | DeepSeek models     | 5.000.000 |
+| Requests per minute  | DeepSeek models     | 5.000     |
+| Concurrent requests  | DeepSeek models     | 300        |
+| Tokens per minute    | Rest of models      | 200.000   |
+| Requests per minute  | Rest of models      | 1.000     |
+| Concurrent requests  | Rest of models      | 300       |
+
+You can [request increases to the default limits](#request-increases-to-the-default-limits). Due to high demand, limit increase requests can be submitted and evaluated per request.
 
 ### Other limits
 
@@ -49,6 +55,28 @@ Global Standard deployments use Azure's global infrastructure, dynamically routi
 
 The Usage Limit determines the level of usage above which customers might see larger variability in response latency. A customer's usage is defined per model and is the total tokens consumed across all deployments in all subscriptions in all regions for a given tenant.
 
+## Request increases to the default limits
+
+Limit increase requests can be submitted and evaluated per request. [Open an online customer support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest/). When requesting for endpoint limit increase, provide the following information:
+
+1. When opening the support request, select **Service and subscription limits (quotas)** as the **Issue type**.
+
+1. Select the subscription of your choice.
+
+1. Select **Cognitive Services** as **Quota type**.
+
+1. Select **Next**.
+
+1. On the **Additional details** tab, you need to provide detailed reasons for the limit increase in order for your request to be processed. Be sure to add the following information into the reason for limit increase:
+
+   * Model name, model version (if applicable), and deployment type (SKU).
+   * Description of your scenario and workload.
+   * Rationale for the requested increase.
+   * Provide the target throughput: Tokens per minute, requests per minute, etc.
+   * Provide planned time plan (by when you need increased limits).
+
+1. Finally, select **Save and continue** to continue.
+
 ## General best practices to remain within rate limits
 
 To minimize issues related to rate limits, it's a good idea to use the following techniques:
@@ -58,10 +86,6 @@ To minimize issues related to rate limits, it's a good idea to use the following
 - Test different load increase patterns.
 - Increase the quota assigned to your deployment. Move quota from another deployment, if necessary.
 
-### Request increases to the default quotas and limits
-
-Quota increase requests can be submitted and evaluated per request. [Submit a service request](../../ai-services/cognitive-services-support-options.md?context=/azure/ai-services/openai/context/context).
-
 ## Next steps
 
-* Learn more about the [models available in the Azure AI model's inference service](./concepts/models.md)
+* Learn more about the [models available in the Azure AI model's inference service](./concepts/models.md)

articles/ai-services/openai/concepts/model-retirements.md

@@ -4,7 +4,7 @@ titleSuffix: Azure OpenAI
 description: Learn about the model deprecations and retirements in Azure OpenAI.
 ms.service: azure-ai-openai
 ms.topic: conceptual
-ms.date: 02/24/2025
+ms.date: 02/25/2025
 ms.custom: 
 manager: nitinme
 author: mrbullwinkle
@@ -93,7 +93,7 @@ These models are currently available for use in Azure OpenAI Service.
 
 | Model | Version | Retirement date | Suggested replacements |
 | ---- | ---- | ---- | --- |
-| `dall-e-3` | 3 | No earlier than April 30, 2025 | |
+| `dall-e-3` | 3 | No earlier than June 30, 2025 | |
 | `gpt-35-turbo-16k`| 0613 | April, 30, 2025 | `gpt-35-turbo` (0125) <br><br> `gpt-4o-mini`|
 | `gpt-35-turbo` | 1106 | No earlier than May 31, 2025 <br><br> Deployments set to [**Auto-update to default**](/azure/ai-services/openai/how-to/working-with-models?tabs=powershell#auto-update-to-default) will be automatically upgraded to version: `0125`, starting on January 21, 2025. | `gpt-35-turbo` (0125) <br><br> `gpt-4o-mini` |
 | `gpt-35-turbo` | 0125 | No earlier than May 31, 2025 | `gpt-4o-mini` |
@@ -171,6 +171,10 @@ If you're an existing customer looking for information about these models, see [
 
 ## Retirement and deprecation history
 
+## February 25, 2025
+
+-  `dalle-3` updated to no earlier than June 30, 2025.
+
 ## February 20, 2025
 
 - `o1-preview` updated to no earlier than April 2, 2025.

articles/ai-services/openai/how-to/fine-tuning-functions.md

@@ -1,24 +1,118 @@
 ---
 title: Fine-tuning function calls with Azure OpenAI Service
-description: Learn how to improve function calling performance with Azure OpenAI fine-tuning
+description: Learn how to improve tool calling performance with Azure OpenAI fine-tuning
 #services: cognitive-services
 manager: nitinme
 ms.service: azure-ai-openai
 ms.topic: how-to
-ms.date: 09/05/2024
+ms.date: 02/20/2025
 author: mrbullwinkle
 ms.author: mbullwin
 ---
 
 
-# Fine-tuning and function calling
+# Fine-tuning and tool calling
 
-Models that use the chat completions API support [function calling](../how-to/function-calling.md). Unfortunately, functions defined in your chat completion calls don't always perform as expected. Fine-tuning your model with function calling examples can improve model output by enabling you to:
+Models that use the chat completions API support [tool calling](../how-to/function-calling.md). Unfortunately, functions defined in your chat completion calls don't always perform as expected. Fine-tuning your model with tool calling examples can improve model output by enabling you to:
 
 * Get similarly formatted responses even when the full function definition isn't present. (Allowing you to potentially save money on prompt tokens.)
 * Get more accurate and consistent outputs.
 
-## Constructing a training file
+> [!NOTE]
+> `function_call` and `functions` have been deprecated in favor of `tools`. 
+> It is recommended to use the `tools` parameter instead.
+
+
+## Tool calling (recommended)
+### Constructing a training file
+
+When constructing a training file of tool calling examples, you would take a function definition like this:
+
+```json
+{
+    "messages": [
+        { "role": "user", "content": "What is the weather in San Francisco?" },
+        {
+            "role": "assistant",
+            "tool_calls": [
+                {
+                    "id": "call_id",
+                    "type": "function",
+                    "function": {
+                        "name": "get_current_weather",
+                        "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"
+                    }
+                }
+            ]
+        }
+    ],
+    "tools": [
+        {
+            "type": "function",
+            "function": {
+                "name": "get_current_weather",
+                "description": "Get the current weather",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "location": {
+                            "type": "string",
+                            "description": "The city and country, eg. San Francisco, USA"
+                        },
+                        "format": { "type": "string", "enum": ["celsius", "fahrenheit"] }
+                    },
+                    "required": ["location", "format"]
+                }
+            }
+        }
+    ]
+}
+```
+
+And express the information as a single line within your `.jsonl` training file as below:
+
+```jsonl
+{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]}],"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country, eg. San Francisco, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
+```
+
+As with all fine-tuning training your example file requires at least 10 examples.
+
+### Optimize for cost
+
+OpenAI recommends that if you're trying to optimize to use fewer prompt tokens post fine-tuning your model on the full function definitions you can experiment with:
+
+* Omit function and parameter descriptions: remove the description field from function and parameters.
+* Omit parameters: remove the entire properties field from the parameters object.
+* Omit function entirely: remove the entire function object from the functions array.
+
+### Optimize for quality
+
+Alternatively, if you're trying to improve the quality of the tool calling output, it's recommended that the function definitions present in the fine-tuning training dataset and subsequent chat completion calls remain identical.
+
+### Customize model responses to function outputs
+
+Fine-tuning based on tool calling examples can also be used to improve the model's response to function outputs. To accomplish this, you include examples consisting of function response messages and assistant response messages where the function response is interpreted and put into context by the assistant.
+
+```json
+{
+    "messages": [
+        {"role": "user", "content": "What is the weather in San Francisco?"},
+        {"role": "assistant", "tool_calls": [{"id": "call_id", "type": "function", "function": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]}
+        {"role": "tool", "tool_call_id": "call_id", "content": "21.0"},
+        {"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"}
+    ],
+    "tools": [] // same as before
+}
+```
+
+As with the example before, this example is artificially expanded for readability. The actual entry in the `.jsonl` training file would be a single line:
+
+```jsonl
+{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]},{"role":"tool","tool_call_id":"call_id","content":"21.0"},{"role":"assistant","content":"It is 21 degrees celsius in San Francisco, CA"}],"tools":[]}
+```
+
+## Function calling
+### Constructing a training file
 
 When constructing a training file of function calling examples, you would take a function definition like this:
 
@@ -51,19 +145,19 @@ And express the information as a single line within your `.jsonl` training file
 
 As with all fine-tuning training your example file requires at least 10 examples.
 
-## Optimize for cost
+### Optimize for cost
 
 OpenAI recommends that if you're trying to optimize to use fewer prompt tokens post fine-tuning your model on the full function definitions you can experiment with:
 
 * Omit function and parameter descriptions: remove the description field from function and parameters.
 * Omit parameters: remove the entire properties field from the parameters object.
 * Omit function entirely: remove the entire function object from the functions array.
 
-## Optimize for quality
+### Optimize for quality
 
 Alternatively, if you're trying to improve the quality of the function calling output, it's recommended that the function definitions present in the fine-tuning training dataset and subsequent chat completion calls remain identical.
 
-## Customize model responses to function outputs
+### Customize model responses to function outputs
 
 Fine-tuning based on function calling examples can also be used to improve the model's response to function outputs. To accomplish this, you include examples consisting of function response messages and assistant response messages where the function response is interpreted and put into context by the assistant.
 
@@ -85,6 +179,7 @@ As with the example before, this example is artificially expanded for readabilit
 {"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}, {"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celcius\"}"}}, {"role": "function", "name": "get_current_weather", "content": "21.0"}, {"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"}], "functions": []}
 ```
 
+
 ## Next steps
 
 * [Function calling fine-tuning scenarios](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/fine-tuning-with-function-calling-on-azure-openai-service/ba-p/4065968).