Skip to content

Commit abdd48a

Browse files
dkershaw10dkershaw10
authored
Merge pull request #490 from microsoft/dkershaw10-idempotentOperations
New upsert pattern
2 parents b090216 + b2a7ced commit abdd48a

File tree

2 files changed

+253
-3
lines changed

2 files changed

+253
-3
lines changed

‎graph/GuidelinesGraph.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,8 @@ If possible, APIs SHOULD use resource-based designs with standard HTTP methods r
250250

251251
| Microsoft Graph rules for modeling behavior |
252252
|------------------------------------------------------------------|
253-
| :heavy_check_mark: **MUST** use POST to create new entities in insertable entity sets or collections.<BR>This approach requires the server to produce system generated identities. |
253+
| :heavy_check_mark: **MUST** use POST to create new entities in insertable entity sets or collections.<BR>This approach requires the service to produce a system-generated key, or for a caller to provide a key in the request payload. |
254+
| :ballot_box_with_check: **SHOULD** additionally use PATCH to create new entities in insertable entity sets or collections.<BR>This [Upsert](./patterns/upsert.md) approach requires the caller to provide a key in the request URL. |
254255
| :heavy_check_mark: **MUST** use PATCH to edit updatable resources. |
255256
| :heavy_check_mark: **MUST** use DELETE to delete deletable resources. |
256257
| :heavy_check_mark: **MUST** use GET for listing and reading resources. |
@@ -383,8 +384,9 @@ The guidelines in previous sections are intentionally brief and provide a jump s
383384
| [Namespace](./patterns/namespace.md) | Organize resource definitions into a logical set. |
384385
| [Navigation properties](./patterns/navigation-property.md) | Model resource relationships |
385386
| [Operations](./patterns/operations.md) | Model complex business operations |
386-
| [Type hierarchy](./patterns/subtypes.md) | Model `is-a` relationships using subtypes.
387-
| [Viewpoint](./patterns/viewpoint.md) | Model user specific properties for a shared resource.
387+
| [Type hierarchy](./patterns/subtypes.md) | Model `is-a` relationships using subtypes. |
388+
| [Upsert](./patterns/upsert.md) | Idempotent operation to create or update a resource using a client-provided key. |
389+
| [Viewpoint](./patterns/viewpoint.md) | Model user specific properties for a shared resource. |
388390

389391
## References
390392

‎graph/patterns/upsert.md

Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
# Upsert
2+
3+
Microsoft Graph API Design Pattern
4+
5+
*The `Upsert` pattern is a non-destructive idempotent operation using a client-provided key, that ensures that system resources can be deployed in a reliable, repeatable, and controlled way, typically used in Infrastructure as Code (IaC) scenarios.*
6+
7+
## Problem
8+
9+
Infrastructure as code (IaC) defines system resources and topologies in a declarative manner that allows teams to manage those resources as they would code.
10+
Practicing IaC helps teams deploy system resources in a reliable, repeatable, and controlled way.
11+
IaC also helps automate deployment and reduces the risk of human error, especially for complex large environments.
12+
Customers want to adopt IaC practices for many of the resources managed through Microsoft Graph.
13+
14+
Most resources' creation operations in Microsoft Graph are not idempotent in nature.
15+
As a consequence, API consumers that want to offer IaC solutions, must create compensation layers that can mimic idempotent behavior.
16+
For example, when creating a resource, the compensation layer must check whether the resource first exists, before trying to create or update the resource.
17+
18+
Additionally, IaC code scripts or templates usually employ client-provided names (or keys) to track resources in a predictable manner, whereas [Microsoft Graph guidelines](../GuidelinesGraph.md#behavior-modeling) suggests use of `POST` to create new entities with service-generated keys.
19+
20+
## Solution
21+
22+
The solution is to use an `Upsert` pattern, to solve for the non-idempotent creation and client-provided naming problems.
23+
24+
* `Upsert` uses `PATCH` with a client-provided key in the URL:
25+
* If there is a natural client-provided key that can serve as the primary key, then the service should support `Upsert` with that key.
26+
* If the primary key is service-generated, the client-provided key should use an [alternate key](./alternate-key.md) to support idempotent creation.
27+
* For a non-existent resource (specified by the client-provided key) the service must handle this as a "create" (aka insert). As part of creation, the service must still generate the primary key value, if appropriate.
28+
* For an existing resource (specified by the client-provided key) the service must handle this as an "update".
29+
* If using an alternate key, then
30+
* for IaC scenarios, the alternate key should be called `uniqueName`, if there isn't already a more natural existing property that could be used as an alternate key.
31+
* the service must also support `GET` using the alternate key pattern.
32+
* Services should always support `POST` to the collection URL.
33+
* For service-generated keys, this should return the server generated key.
34+
* For client-provided keys, the client can provide the key as part of the request payload.
35+
* If a service does not support `Upsert`, then a `PATCH` call against a non-existent resource must result in an HTTP "404 not found" error.
36+
37+
This solution allows for existing resources that follow Microsoft Graph conventions for CRUD operations to add `Upsert` without impacting existing apps or functionality.
38+
39+
Ideally, all new entity types should support an `Upsert` mechanism, especially where they support control-plane APIs, or are used in admin style or IaC scenarios.
40+
41+
## When to use this pattern
42+
43+
This pattern should be adopted for resources that are managed through infrastructure as code or desired state configuration.
44+
45+
## Issues and considerations
46+
47+
* Services with existing APIs that use a client-defined key that want to start supporting the `Upsert` pattern may have concerns about backwards compatibility.
48+
API producers can require clients to opt-in to the `Upsert` pattern, by using the `Prefer: create-if-missing` HTTP request header.
49+
* `Upsert` can also be supported against singletons, using a `PATCH` to the singleton's URL.
50+
* Services that support `Upsert` should allow clients to use the:
51+
* `If-Match=*` request header to explicitly treat an `Upsert` request as an update and not an insert.
52+
* `If-None-Match=*` request header to explicitly treat an `Upsert` request as an insert and not an update.
53+
* The client-provided alternate key must be immutable after being set. If its value is null then it should be settable as a way to backfill existing resources for use in IaC scenarios.
54+
* API producers could use `PUT` operations to create or update, but generally this approach is not recommended due to the destructive nature of `PUT`'s replace semantics.
55+
* API producers may annotate entity sets, singletons and collections to indicate that entities can be "upserted". The example below shows this annotation for the `groups` entity set.
56+
57+
```xml
58+
<EntitySet Name="groups" EntityType="microsoft.graph.group">
59+
<Annotation Term="Org.OData.Capabilities.V1.UpdateRestrictions">
60+
<Record>
61+
<PropertyValue Property="Upsertable" Bool="true"/>
62+
</Record>
63+
</Annotation>
64+
</EntitySet>
65+
```
66+
67+
## Examples
68+
69+
For these examples we'll use the `group` entity type, which defines both a primary (service-generated) key (`id`) and an alternate (client-provided) key (`uniqueName`).
70+
71+
```xml
72+
<EntityType Name="group">
73+
<Key>
74+
<PropertyRef Name="id"/>
75+
</Key>
76+
<Property Name="id" Type="Edm.String"/>
77+
<Property Name="uniqueName" Type="Edm.String"/>
78+
<Property Name="displayName" Type="Edm.String"/>
79+
<Property Name="description" Type="Edm.String"/>
80+
<Annotation Term="Org.OData.Core.V1.AlternateKeys">
81+
<Collection>
82+
<Record Type="Org.OData.Core.V1.AlternateKey">
83+
<PropertyValue Property="Key">
84+
<Collection>
85+
<Record Type="Org.OData.Core.V1.PropertyRef">
86+
<PropertyValue Property="Name" PropertyPath="uniqueName" />
87+
</Record>
88+
</Collection>
89+
</PropertyValue>
90+
</Record>
91+
</Collection>
92+
</Annotation>
93+
</Property>
94+
</EntityType>
95+
```
96+
97+
### Upserting a record (creation path)
98+
99+
Create a new group, with a `uniqueName` of "Group157". In this case, this group does not exist.
100+
101+
```http
102+
PATCH /groups(uniqueName='Group157')
103+
Prefer: return=representation
104+
```
105+
106+
```json
107+
{
108+
"displayName": "My favorite group",
109+
"description": "All my favorite people in the world"
110+
}
111+
```
112+
113+
Response:
114+
115+
```http
116+
201 created
117+
Preference-Applied: return=representation
118+
```
119+
120+
```json
121+
{
122+
"id": "1a89ade6-9f59-4fea-a139-23f84e3aef66",
123+
"displayName": "My favorite group",
124+
"description": "All my favorite people in the world",
125+
"uniqueName": "Group157"
126+
}
127+
```
128+
129+
### Upserting a record (update path)
130+
131+
Create a new group, with a `uniqueName` of "Group157", exactly like before. Except in this case, this group already exists. This is a common scenario in IaC, when a deployment template is re-run multiple times.
132+
133+
```http
134+
PATCH /groups(uniqueName='Group157')
135+
Prefer: return=representation
136+
```
137+
138+
```json
139+
{
140+
"displayName": "My favorite group",
141+
"description": "All my favorite people in the world"
142+
}
143+
```
144+
145+
Response:
146+
147+
```http
148+
200 ok
149+
Preference-Applied: return=representation
150+
```
151+
152+
```json
153+
{
154+
"id": "1a89ade6-9f59-4fea-a139-23f84e3aef66",
155+
"displayName": "My favorite group",
156+
"description": "All my favorite people in the world",
157+
"uniqueName": "Group157"
158+
}
159+
```
160+
161+
Notice how this operation is idempotent in nature, rather than returning a 409 conflict error.
162+
163+
### Updating a record
164+
165+
Update "Group157" group with a new description.
166+
167+
```http
168+
PATCH /groups(uniqueName='Group157')
169+
Prefer: return=representation
170+
```
171+
172+
```json
173+
{
174+
"description": "Some of my favorite people in the world."
175+
}
176+
```
177+
178+
Response:
179+
180+
```http
181+
200 ok
182+
Preference-Applied: return=representation
183+
```
184+
185+
```json
186+
{
187+
"id": "1a89ade6-9f59-4fea-a139-23f84e3aef66",
188+
"displayName": "My favorite group",
189+
"description": "Some of my favorite people in the world.",
190+
"uniqueName": "Group157"
191+
}
192+
```
193+
194+
### Upsert opt-in request
195+
196+
In this case, the group API is a pre-existing API that supports `PATCH` with a client-provided alternate key. To enable `Upsert` behavior,
197+
the client must opt-in using an HTTP request header, to create a new group using `PATCH`.
198+
199+
```http
200+
PATCH /groups(uniqueName='Group157')
201+
Prefer: create-if-missing; return=representation
202+
```
203+
204+
```json
205+
{
206+
"displayName": "My favorite group",
207+
"description": "All my favorite people in the world"
208+
}
209+
```
210+
211+
Response:
212+
213+
```http
214+
201 created
215+
Preference-Applied: create-if-missing; return=representation
216+
```
217+
218+
```json
219+
{
220+
"id": "1a89ade6-9f59-4fea-a139-23f84e3aef66",
221+
"displayName": "My favorite group",
222+
"description": "All my favorite people in the world",
223+
"uniqueName": "Group157"
224+
}
225+
```
226+
227+
### Upsert (create) not supported
228+
229+
Following on from the last example, the same request to create a new group, with a `uniqueName` of "Group157",
230+
without the opt-in header, results in a 404 HTTP response code.
231+
232+
```http
233+
PATCH /groups(uniqueName='Group157')
234+
Prefer: return=representation
235+
```
236+
237+
```json
238+
{
239+
"displayName": "My favorite group",
240+
"description": "All my favorite people in the world"
241+
}
242+
```
243+
244+
Response:
245+
246+
```http
247+
404 not found
248+
```

0 commit comments

Comments
 (0)