You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/openai/how-to/provisioned-throughput-onboarding.md
+11-4Lines changed: 11 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,13 +72,20 @@ Customers that require long-term usage of provisioned, data zoned provisioned, a
72
72
> Charges for deployments on a deleted resource will continue until the resource is purged. To prevent this, delete a resource’s deployment before deleting the resource. For more information, see [Recover or purge deleted Azure AI services resources](../../recover-purge-resources.md).
73
73
74
74
## How much throughput per PTU you get for each model
75
-
The amount of throughput (measured in tokens per minute or TPM) a deployment gets per PTU is a function of the input and output tokens in a given minute.
76
75
77
-
Generating output tokens requires more processing than input tokens. For the models specified in the table below, 1 output token counts as 3 input tokens towards your TPM-per-PTU limit. The service dynamically balances the input & output costs, so users do not have to set specific input and output limits. This approach means your deployment is resilient to fluctuations in the workload.
78
76
79
-
To help with simplifying the sizing effort, the following table outlines the TPM-per-PTU for the specified models. To understand the impact of output tokens on the TPM-per-PTU limit, use the 3 input token to 1 output token ratio.
77
+
To understand how much throughput (TPU) you get, keep the following in mind:
78
+
79
+
* The amount of throughput (measured in tokens per minute or TPM) a deployment gets per PTU is a function of the input and output tokens in a given minute.
80
+
* Generating output tokens requires more processing than input tokens. Provisioned-Managed matches the Standard offering for gpt-4.1 models and later.
81
+
* 1 output token now counts as 4 input tokens towards your TPM-per-PTU limit.
82
+
* In the standard offering, 1 output token is 4 times expensive as an input token. See the [pricing page for details](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/).
83
+
84
+
To help simplify the sizing effort, the following table outlines the TPM-per-PTU for the specified models.
85
+
To understand the impact of output tokens on the TPM-per-PTU limit, use the 4 input token to 1 output token ratio.
86
+
87
+
For a detailed understanding of how different ratios of input and output tokens impact the throughput your workload needs, see the [Azure OpenAI capacity calculator](https://ai.azure.com/resource/calculator). The table also shows the [Service Level Agreement (SLA)](https://www.microsoft.com/licensing/docs/view/Service-Level-Agreements-SLA-for-Online-Services?lang=1) Latency Target Values per model.
80
88
81
-
For a detailed understanding of how different ratios of input and output tokens impact the throughput your workload needs, see the [Azure OpenAI capacity calculator](https://ai.azure.com/resource/calculator). The table also shows Service Level Agreement (SLA) Latency Target Values per model. For more information about the SLA for Azure OpenAI Service, see the [Service Level Agreements (SLA) for Online Services page](https://www.microsoft.com/licensing/docs/view/Service-Level-Agreements-SLA-for-Online-Services?lang=1)
0 commit comments