Skip to content

Commit cc4fc5b

Browse files
committed
updates
1 parent 85e9273 commit cc4fc5b

File tree

2 files changed

+26
-22
lines changed

2 files changed

+26
-22
lines changed

articles/ai-services/openai/concepts/provisioned-throughput.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,10 @@ You should consider switching from standard deployments to provisioned managed d
5353

5454
## Key concepts
5555

56+
### Provisioned Throughput Units (PTU)
57+
58+
Provisioned throughput units (PTUs) are generic units of model processing capacity that you can use to size provisioned deployments to achieve the required throughput for processing prompts and generating completions. Provisioned throughput units are granted to a subscription as quota, and used to define costs. Each quota is specific to a region and defines the maximum number of PTUs that can be assigned to deployments in that subscription and region. For information about the costs associated with the provision managed offering and PTUs, see [Understanding costs associated with PTU](../how-to/provisioned-throughput-onboarding.md).
59+
5660
### Deployment types
5761

5862
When creating a provisioned deployment in Azure AI Foundry, the deployment type on the Create Deployment dialog can be set to the Global Provisioned-Managed, DataZone Provisioned-Managed, or regional Provisioned-Managed deployment type depending on the data processing needs for the given workload.

articles/ai-services/openai/how-to/provisioned-throughput-onboarding.md

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Use this article to learn about calculating and understanding costs associated w
1717
> [!NOTE]
1818
> In function calling and agent use cases, token usage can be variable. You should understand your expected Tokens Per Minute (TPM) usage in detail prior to migrating workloads to PTU.
1919
20-
#### Provisioned throughput units
20+
## Provisioned throughput units
2121

2222
Provisioned throughput units (PTUs) are generic units of model processing capacity that you can use to size provisioned deployments to achieve the required throughput for processing prompts and generating completions. Provisioned throughput units are granted to a subscription as quota. Each quota is specific to a region and defines the maximum number of PTUs that can be assigned to deployments in that subscription and region.
2323

@@ -30,6 +30,27 @@ The hourly model is useful for short-term deployment needs, such as validating n
3030
> [!NOTE]
3131
> Azure OpenAI Provisioned customers onboarded prior to the August self-service update use a purchase model called the Commitment model. These customers can continue to use this older purchase model alongside the Hourly/reservation purchase model. The Commitment model is not available for new customers or new models introduced after August 2024. For details on the Commitment purchase model and options for coexistence and migration, please see the [Azure OpenAI Provisioned August Update](../concepts/provisioned-migration.md).
3232
33+
34+
## Model independent quota
35+
36+
Unlike the Tokens Per Minute (TPM) quota used by other Azure OpenAI offerings, PTUs are model-independent. The PTUs might be used to deploy any supported model/version in the region.
37+
38+
:::image type="content" source="../media/provisioned/model-independent-quota.png" alt-text="Diagram of model independent quota with one pool of PTUs available to multiple Azure OpenAI models." lightbox="../media/provisioned/model-independent-quota.png":::
39+
40+
Quota for provisioned deployments shows up in Azure AI Foundry as the following deployment types: [global provisioned](../how-to/deployment-types.md#global-provisioned), [data zone provisioned](../how-to/deployment-types.md#data-zone-provisioned) and [standard provisioned](../how-to/deployment-types.md#provisioned).
41+
42+
|deployment type |Quota name |
43+
|---------|---------|
44+
|[provisioned](../how-to/deployment-types.md#provisioned) | Provisioned Managed Throughput Unit |
45+
|[global provisioned](../how-to/deployment-types.md#global-provisioned) | Global Provisioned Managed Throughput Unit |
46+
|[data zone provisioned](../how-to/deployment-types.md#data-zone-provisioned) | Data Zone Provisioned Managed Throughput Unit |
47+
48+
:::image type="content" source="../media/provisioned/ptu-quota-page.png" alt-text="Screenshot of quota UI for Azure OpenAI provisioned." lightbox="../media/provisioned/ptu-quota-page.png":::
49+
50+
51+
> [!NOTE]
52+
> Global provisioned and data zone provisioned deployments are only supported for gpt-4o and gpt-4o-mini models at this time. For more information on model availability, review the [models documentation](./models.md).
53+
3354
## Hourly usage
3455

3556
Provisioned, Data Zone Provisioned, and Global Provisioned deployments are charged an hourly rate ($/PTU/hr) on the number of PTUs that have been deployed.  For example, a 300 PTU deployment will be charged the hourly rate times 300.  All Azure OpenAI pricing is available in the Azure Pricing Calculator.
@@ -81,27 +102,6 @@ A few high-level considerations:
81102
- Generations require more capacity than prompts
82103
- For GPT-4o and later models, the TPM per PTU is set for input and output tokens separately. For older models, larger calls are progressively more expensive to compute. For example, 100 calls of with a 1000 token prompt size requires less capacity than one call with 100,000 tokens in the prompt. This tiering means that the distribution of these call shapes is important in overall throughput. Traffic patterns with a wide distribution that includes some large calls might experience lower throughput per PTU than a narrower distribution with the same average prompt & completion token sizes.
83104

84-
85-
### Model independent quota
86-
87-
Unlike the Tokens Per Minute (TPM) quota used by other Azure OpenAI offerings, PTUs are model-independent. The PTUs might be used to deploy any supported model/version in the region.
88-
89-
:::image type="content" source="../media/provisioned/model-independent-quota.png" alt-text="Diagram of model independent quota with one pool of PTUs available to multiple Azure OpenAI models." lightbox="../media/provisioned/model-independent-quota.png":::
90-
91-
Quota for provisioned deployments shows up in Azure AI Foundry as the following deployment types: [global provisioned](../how-to/deployment-types.md#global-provisioned), [data zone provisioned](../how-to/deployment-types.md#data-zone-provisioned) and [standard provisioned](../how-to/deployment-types.md#provisioned).
92-
93-
|deployment type |Quota name |
94-
|---------|---------|
95-
|[provisioned](../how-to/deployment-types.md#provisioned) | Provisioned Managed Throughput Unit |
96-
|[global provisioned](../how-to/deployment-types.md#global-provisioned) | Global Provisioned Managed Throughput Unit |
97-
|[data zone provisioned](../how-to/deployment-types.md#data-zone-provisioned) | Data Zone Provisioned Managed Throughput Unit |
98-
99-
:::image type="content" source="../media/provisioned/ptu-quota-page.png" alt-text="Screenshot of quota UI for Azure OpenAI provisioned." lightbox="../media/provisioned/ptu-quota-page.png":::
100-
101-
102-
> [!NOTE]
103-
> Global provisioned and data zone provisioned deployments are only supported for gpt-4o and gpt-4o-mini models at this time. For more information on model availability, review the [models documentation](./models.md).
104-
105105
### Obtaining PTU quota
106106

107107
PTU quota is available by default in many regions. If more quota is required, customers can request quota via the Request Quota link. This link can be found to the right of the designated provisioned deployment type quota tabs in Azure AI Foundry The form allows the customer to request an increase in the specified PTU quota for a given region. The customer receives an email at the included address once the request is approved, typically within two business days.

0 commit comments

Comments
 (0)