Skip to content

Commit 986ce0b

Browse files
committed
adding project pacific content
1 parent 61cb707 commit 986ce0b

File tree

2 files changed

+259
-1
lines changed

2 files changed

+259
-1
lines changed
Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
# Azure Resource Graph (ARG) GET/LIST API
2+
3+
ARG GET/LIST API significantly reduces READ throttling by serving all incoming GET and LIST calls against ARG platform with smart routing controls in control plane. The API aligns with the existing request and response contracts of Azure control plane APIs while addressing throttling issues for customers.
4+
5+
ARG GET/LIST provides a default quota of 4k /min /user / subscription on a moving window. This is not a hard limit; we can support more than this based on scenario requirement. The API provides a response header “x-ms-user-quota-remaining" indicating remaining quota and "x-ms-user-quota-resets-after" indicating the time for a full quota reset based on which you can understand your quota consumption.
6+
7+
> [!NOTE]
8+
> Please keep in mind that the Azure Resource Manager (ARM) quota applies to these calls. Read about the ARM limits here : Understand how Azure Resource Manager throttles requests - Azure Resource Manager | Microsoft Learn. Refer to section "Migrating to regional throttling and token bucket algorithm" , these are the new limits which ARM follows for Azure Public cloud. 
9+
10+
## Using the ARG GET/LIST API
11+
12+
Using ARG GET/LIST API is very straightforward. If you feel your scenario matches the conditions mentioned here <link to when to use ARG GET/LIST API >, you can simply append the flag &useResourceGraph=true to your control plane API calls, and the request will be routed to the ARG GET/LIST API backend.
13+
14+
Contact the ARG Product group by sending an email to Azure Resource Graph team sharing a brief overview of your scenario and the ARG team will reach out to you with next steps. Callers must also design appropriate retry logic and implement fallback mechanisms to ensure smooth and reliable experience. This opt-in model has been deliberately chosen to allow the Azure Resource Graph team to better understand customer usage patterns and make improvements as needed.
15+
16+
Refer to some known limitations [here](#known-limitations) and [frequently asked questions](#frequently-asked-questions).
17+
18+
## ARG GET/LIST API Contract
19+
20+
### Resource Point Get
21+
22+
This request is used for looking up a single resource by providing resource Id.
23+
24+
**Traditional Point Get Request:**
25+
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/{providerNamespace}/{resourceType}/{resourceName}?api-version={apiVersion}
26+
27+
**ARG Point Get Request:**
28+
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/{providerNamespace}/{resourceType}/{resourceName}?api-version={apiVersion}&**useResourceGraph=true**
29+
30+
## Subscription Collection Get-
31+
32+
This request is used for listing all resources under a single resource type in a subscription.
33+
34+
**Traditional Subscription Collection Get Request:**
35+
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/{providerNamespace}/{resourceType}?api-version={apiVersion}
36+
37+
**ARG Subscription Collection Get Request:**
38+
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/{providerNamespace}/{resourceType}?api-version={apiVersion}&**useResourceGraph=true**
39+
40+
## Resource Group Collection Get
41+
42+
This request is used for listing all resources under a single resource type in a resource group.
43+
44+
**Traditional Point Get Request:**
45+
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/{providerNamespace}/{resourceType}?api-version={apiVersion}
46+
47+
**ARG GET/LIST Point Get Request:**
48+
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/{providerNamespace}/{resourceType}?api-version={apiVersion}&**useResourceGraph=true**
49+
50+
## Some frequently used examples
51+
52+
### Scenario: Get VM with InstanceView
53+
54+
For this specific example, use the following requests to get a virtual machine with InstanceView.
55+
56+
#### ARG GET/LIST Request
57+
58+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}?api-version=2024-07-01&$expand=instanceView&useResourceGraph=true
59+
60+
#### ARG Query
61+
62+
Resources
63+
64+
| where type =~ 'microsoft.compute/virtualmachines'
65+
| where id =~ '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}'
66+
67+
#### CRP Request
68+
69+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}?api-version=2024-07-01&$expand=instanceView
70+
71+
72+
### List VMs under ResourceGroup
73+
74+
For this specific example, use the following requests to retrieve the list of virtual machines under the resource group.
75+
76+
#### ARG GET/LIST Request
77+
78+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines?api-version=2024-07-01&$expand=instanceView&useResourceGraph=true
79+
80+
#### ARG Query
81+
82+
Resources
83+
84+
| where resourceGroup =~ ‘{resourceGroup}’
85+
| where type =~ 'microsoft.compute/virtualmachines'
86+
87+
### CRP Request
88+
89+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines?api-version=2024-07-01
90+
91+
### List VMSS VM (Uniform) with InstanceView
92+
93+
#### ARG GET/LIST Request
94+
95+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}/virtualmachines?api-version=2024-07-01&$expand=instanceView&useResourceGraph=true
96+
97+
#### ARG Query
98+
99+
ComputeResources
100+
101+
| where type =~ 'microsoft.compute/virtualmachinescalesets/virtualmachines'
102+
| where id startswith ‘/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}’
103+
104+
### CRP Request
105+
106+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}/virtualmachines?api-version=2024-07-01&$expand=instanceView
107+
108+
## List VMSS VM (Flex) with InstanceView
109+
110+
For this specific example, use the following requests to retrieve the list of VMSS VM with InstanceView.
111+
112+
### ARG GET/LIST Request
113+
114+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}/virtualmachines?api-version=2024-07-01&$filter=’virtualMachineScaleSet/id’ eq ‘/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}’&$expand=instanceView&useResourceGraph=true
115+
116+
### ARG Query
117+
118+
Resources
119+
120+
| where type =~ ‘microsoft.compute/virtualmachines’
121+
| where properties.virtualMachineScaleSet.id =~‘/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}’
122+
123+
### CRP Request
124+
125+
Virtual Machines - List - REST API (Azure Compute) | Microsoft Learn
126+
127+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines?api-version=2024-07-01&$expand=instanceView&$filter=’virtualMachineScaleSet/id’ eq ‘/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachinescalesets/{vmss}’
128+
129+
### List storage accounts in subscription
130+
131+
For this specific example, use the following requests to retrieve the list of storage accounts in the subscription.
132+
133+
#### ARG GET/LIST request
134+
135+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.storage/storageAccounts?api-version=2024-01-01&useResourceGraph=true
136+
137+
#### ARG Query
138+
139+
Resources
140+
141+
| where type =~ ‘microsoft.storage/storageAccounts’
142+
143+
#### SRP request
144+
145+
HTTP GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.storage/storageAccounts?api-version=2024-01-01
146+
147+
148+
# Known Limitations
149+
150+
1. **VMSS VM Health status** is not supported by default. If you have a requirement for that, do let us know by emailing Azure Resource Graph team.
151+
2. **Supported Resources** - The ARG GET/LIST API supports resources part of ‘resources’ and ‘computeresources’ table. If you have a requirement for a specific resource type outside of these tables do let us know by emailing Azure Resource Graph team.
152+
3. **Single API Version Support** - ARG GET/LIST today only supports a single API version for each resource type. This is generally the latest non-preview version of the type that exists in the Azure REST API spec. The `api-version` parameter in the request URL is ignored by ARG GET/LIST today. ARG GET/LIST returns `apiVersion` field in resource payloads in responses; this is the version that ARG GET/LIST indexed the resource in. Callers can leverage this field to understand the apiVersion to use , if its relevant for their scenario.
153+
4. **Client Support** - Azure REST API: Supported | Azure SDK: Supported [sample .NET code shared below](#sample-sdk-code)| PowerShell: Currently not supported | Azure CLI: Currently not supported | Azure Portal: Currently not supported
154+
5. **Client Deserialization Concerns**
155+
- If a client uses REST API to issue GET calls, there should generally be no concerns regarding deserialization due to API version differences.
156+
- If a client uses Azure SDK to issue GET calls, due to relaxed deserialization setting across all languages, the deserialization issue should not be a concern, unless there are contract breaking changes among different versions for the target resource type.
157+
6. **Unprocessable Resource Request**
158+
- There are rare scenarios where ARG GET/LIST is not able to index a resource correctly, other than the existence of the resource. To not sacrifice data quality, ARG GET/LIST will refuse serving GET calls for these resources and return an error code of HTTP 422.
159+
- Clients of ARG GET/LIST should treat HTTP 422 as a permanent error. Clients should retry by falling back to RP (by removing useResourceGraph=true flag). Since the error is applicable specifically to ARG GET/LIST, fallback to RPs should result in an E2E success.
160+
7. **Consistency**
161+
- As an index of Azure resources, ARG GET/LIST ingests the latest updates within seconds. ARG GET/LIST therefore provides a strict bounded staleness consistency level for all resources. This differs from the strong consistency model of Resource Providers.
162+
- In resource Point Get responses, ARG GET/LIST provides a response header x-ms-arg-snapshot-timestamp that indicates the timestamp when the returned resource snapshot was indexed. The value of the header is UTC time in ISO8601 format. (An example, "x-ms-arg-snapshot-timestamp" : "2023-01-20T18:55:59.5610084Z").
163+
164+
### Sample SDK code
165+
166+
The following is a .NET Code sample to call ARG GET/LIST API by creating an ARMClient with policy that adds the flag `useResourceGraph=true` to each call:
167+
168+
First, We create custom ArmClientOption with policy that adds the `useResourceGraph=True` flag per call
169+
170+
```bicep
171+
var ArmClientOptions = new ArmClientOptions();
172+
173+
ArmClientOptions.AddPolicy(new ARG GET/LISTHttpPipelinePolicy(),
174+
175+
HttpPipelinePosition.PerCall);
176+
177+
```
178+
179+
Then, we create ArmClient using the custom ArmClientOptions
180+
181+
```bicep
182+
ArmClient client = new ArmClient(new DefaultAzureCredential(), null,
183+
184+
ArmClientOptions);
185+
```
186+
187+
What if we want to create an ARMClient using a default subscription? We would set the ArmClient client value to:
188+
189+
```bicep
190+
new ArmClient(new DefaultAzureCredential(), defaultSubId,
191+
ArmClientOptions);
192+
```
193+
194+
Then use this policy to add query parameters for every request through the client:
195+
196+
internal class ARG GET/LISTHttpPipelinePolicy : HttpPipelineSynchronousPolicy
197+
198+
```bicep
199+
{
200+
public override void OnSendingRequest(HttpMessage message)
201+
{
202+
// Adds the required query param to explicitly query ARG GET/LIST
203+
message.Request.Uri.AppendQuery("useResourceGraph", bool.TrueString);
204+
}
205+
}
206+
```
207+
208+
# Frequently asked questions
209+
210+
1. How do you ensure the response is returned by ARG GET/LIST API?
211+
212+
There are a few ways that you can identify when a request is served by ARG GET/LIST: 
213+
- In the response body, the `apiVersion` field of resources will be populated, if served by ARG GET/LIST. 
214+
- ARG GET/LIST/ARG returns some additional response headers, some of which are:  
215+
- x-ms-arg-snapshot 
216+
- x-ms-user-quota-remaining 
217+
- x-ms-user-quota-resets-after 
218+
- x-ms-resource-graph-request-duration 
219+
220+
2. How do you know which API version was used by ARG GET/LIST?
221+
222+
This is returned in “apiVersion” field in resource response today. 
223+
224+
3. What happens if a caller calls ARG GET/LIST API with useResourceGraph=true flag for a resource not supported by ARG GET/LIST?  
225+
226+
Any unsupported/unrouteable requests will result in “useResourceGraph=true” ignored and the call will be automatically routed to be Resource Provider. User does not have to take any action.  
227+
228+
4. What permissions are required for querying ARG GET/LIST APIs?
229+
230+
No special permissions are required for ARG GET/LIST APIs; ARG GET/LIST APIs are equivalent to native Resource provider based GET APIs and therefore, standard RBAC applies. Callers need to have at least READ permissions to resources/scopes they are trying to access. 
231+
232+
5. What is the rollback strategy, if we find issues while using ARG GET/LIST APIs?  
233+
234+
If onboarded through the flag “useResourceGraph=true”, the caller may choose to remove the flag (or) “useResourceGraph =false” and the call will be automatically routed to be Resource Provider. 
235+
236+
6. What if you're getting a 404 Not Found when trying to get a resource from ARG GET/LIST that was recently created?
237+
238+
This is a common scenario with a lot of services where customers create resources and immediately issue a GET in 1-2 secs as part of another WRITE workflow. For example - customers create a new resource and right after trying to create a metric alert that monitors it. The resource might not have been indexed by ARG GET/LIST yet. There are 2 ways to work around this:
239+
- Retry on ARG GET/LIST a few times until it returns status code 200. 
240+
- Retry without ARG GET/LIST flag to fall back on the RP. True 404s will not hit the RP since ARM will return the error directly, whereas a false 404 should be served by RPs to get actual data.

articles/governance/resource-graph/concepts/guidance-for-throttled-requests.md

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -226,9 +226,27 @@ while (!string.IsNullOrEmpty(azureOperationResponse.Body.SkipToken))
226226
}
227227
```
228228

229+
## ARG GET/LIST API
230+
231+
ARG is introducing an alternative approach to the existing Azure control plane GET and List API calls that improve scalability and performance, while addressing throttling issues for Azure customers. This API is currently supported only for resources in the ‘resources’ table and ‘computeresources’ table.
232+
233+
The ARG GET/LIST API is specifically meant to address scenarios where you need a lookup of a single resource by Id, or you’re listing resources under the same type and within a certain scope (subscription, resource group or parent resource).
234+
235+
You should consider Pacific if your service falls into one (or many) of categories below:
236+
237+
- Your service is issuing a large volume of simple GET calls within the context of a single subscription or a single resource group and not batches of records from multiple subscriptions.
238+
- Your service issues a large volume of simple GET requests, and is at risk of:
239+
- Facing throttling
240+
- Competing for throttling quota with other customers
241+
- Your service may be or is prone to issuing a large burst of concurrent GET requests within a short period of time.
242+
- Your service requires high availability and faster performance for GET requests for single
243+
- Resource management or enumeration of a list of resources within a certain scope.
244+
245+
If the resource you’re interested in, is in the ‘resources’ table or ‘computeresources’ table, *and* it falls in one of the above categories, then use the ARG GET/LIST API <link>
246+
229247
## Still being throttled?
230248

231-
If you used this article's recommendations and your Azure Resource Graph queries are still being throttled, contact the [Azure Resource Graph team](mailto:[email protected]). The team supports Azure Resource Graph but doesn't support [Microsoft Graph throttling](/graph/throttling).
249+
If you used this article's recommendations as well as the Azure Resource Graph and your Azure Resource Graph GET/LIST API solution and your queries are still being throttled, contact the [Azure Resource Graph team](mailto:[email protected]). The team supports Azure Resource Graph but doesn't support [Microsoft Graph throttling](/graph/throttling).
232250

233251
Provide these details when you contact the Azure Resource Graph team:
234252

0 commit comments

Comments
 (0)