|
| 1 | +# Change tracking |
| 2 | + |
| 3 | +Microsoft Graph API Design Pattern |
| 4 | + |
| 5 | +*The change tracking pattern provides the ability for API consumers to request changes in data from Microsoft Graph without having to re-read data that has not changed.* |
| 6 | + |
| 7 | + |
| 8 | +## Problem |
| 9 | + |
| 10 | +API consumers require an efficient way to acquire changes to data in the Microsoft Graph, for example to synchronize an external store or to drive a change-centric business process. |
| 11 | + |
| 12 | +## Solution |
| 13 | + |
| 14 | +API designers can enable the change tracking (delta) capability on a resource in the Microsoft Graph (typically on an entity collection or a parent resource) by declaring a delta function on that resource and applying `Org.OData.Capabilities.V1.ChangeTracking` annotation. |
| 15 | + |
| 16 | +This function returns a delta payload. A delta payload consists of a collection of annotated full or partial Microsoft Graph entities plus either a `nextLink` to further pages of original or change data that are immediately available OR a `deltaLink` to get the next set of changes at some later date. |
| 17 | + |
| 18 | +The `nextLink` provides a mechanism to do server-driven paging through the change data that is currently available. When there are no further pages of changes immediately available, a `deltaLink` is returned instead. |
| 19 | +The `deltaLink` provides a mechanism for the API consumer to catch up on changes since their last request to the delta function. If no changes have happened since the last request, then the deltaLink MUST return an empty collection. |
| 20 | + |
| 21 | +Both `nextLink` and `deltaLink` MUST be considered opaque URLs. The best practice is to make them opaque via encoding. |
| 22 | + |
| 23 | +The pattern requires a sequence of requests on the delta function, for additional details see [Change Tracking](https://learn.microsoft.com/en-us/graph/delta-query-overview?tabs=http#use-delta-query-to-track-changes-in-a-resource-collection): |
| 24 | + |
| 25 | + 1. GET request which returns the first page of the current state of the resources that delta applies to. |
| 26 | + 2. [Optionally] Further GET requests to retrieve more pages of the current state via the `@odata.nextLink` URL. |
| 27 | + 3. After some time, a GET request to see if there are new changes via the `@odata.deltaLink` URL. |
| 28 | + 4. [Optionally] GET requests to retrieve more pages of changes via the `@odata.nextLink` URL. |
| 29 | + |
| 30 | +Delta payload requirements: |
| 31 | + - The payload is a collection of change records using the collection format. |
| 32 | + - The change records are full or partial representations of the resources according to their resource types. |
| 33 | + - When a change representing a resource update is included in the payload the API producer MAY return either the changed properties or the full entity. The ID of the resource MUST be included in every change record. |
| 34 | + - When an entity is deleted, the delta function MUST return the ID of the deleted entity as well as an `@removed` annotation with the reason field. |
| 35 | + - When an entity is deleted, the reason MUST be set to “changed” if the entity can be restored. |
| 36 | + - When an entity is deleted. the reason MUST be set to “deleted” if the entity cannot be restored. |
| 37 | + - There is no mechanism to indicate that a resource has entered or exited the dataset based on a change that causes it to match or no longer match any `$filter` query parameter. |
| 38 | + - When a link to an entity is deleted, when the linked entity is deleted, or when a link to an entity is added, the implementer MUST return a `property@delta` annotation. |
| 39 | + - When a link to an entity is deleted, but the entity still exists, the reason MUST be set to `changed`. |
| 40 | + - When a link to an entity is deleted along with the entity, the reason MUST be set to `deleted`. |
| 41 | + |
| 42 | +API producers MAY choose to collate multiple changes to the same resource into a single change record. |
| 43 | + |
| 44 | +API consumers are expected to differentiate resource adds from updates by interpreting the id property of the change records against the existence of resources in whatever external system is doing the processing. |
| 45 | + |
| 46 | + |
| 47 | +## When to use this pattern |
| 48 | + |
| 49 | +API consumers want a pull mechanism to request and process change to Microsoft Graph data, either via proactive polling or by responding to Microsoft Graph notifications. |
| 50 | + |
| 51 | +API consumers need guaranteed data integrity over the set of changes to Microsoft Graph data. |
| 52 | + |
| 53 | +## Considerations |
| 54 | + |
| 55 | + - API service MAY be able to respond to standard OData query parameters with the initial call to the delta function: |
| 56 | + |
| 57 | + - `$select` to enforce the set of properties on which change is reported. |
| 58 | + - `$filter` to influence the scope of changes returned. |
| 59 | + - `$expand` to include linked resources with the set of changes. |
| 60 | + - `$top` parameter to influence the size of the set of change records. |
| 61 | + |
| 62 | + These query parameters MUST be encoded into subsequent `@odata.nextLink` or `@odata.deltaLink`, such that the same options are preserved through the call sequence without callers respecifying them, which MUST NOT be allowed. OData query parameters must be honored in full, or a 400-error returned. |
| 63 | +- Making a sequence of calls to a delta function followed by the opaque URLs in the `nextLink` and `deltaLink` MUST guarantee that the data at the start time of the call sequence and all changes to the data thereafter will be returned at least once. It is not necessary to avoid duplicates in the sequence. When the delta function is returning changes, they MUST be sequenced chronologically refer to [public documentation](https://learn.microsoft.com/en-us/graph/delta-query-overview?view=graph-rest-1.0) for more details. |
| 64 | +- The delta function can be bound to |
| 65 | + - an entity collection, as with `/users/delta` that returns the changes to the users' collection, or |
| 66 | + - some logical parent resource that returns an entity collection, where the change records are implied to be relative to all collections contained within the parent.For example `/me/planner/all/delta` returns changes to any resource within a planner, which are referenced by 'all' navigation property, and `/communications/onlineMeetings/getAllRecordings/delta` returns changes to any meeting recordings returned by `getAllRecordings` function. |
| 67 | + |
| 68 | +- API service should use `$skipToken` and `$deltaToken` within their implementations of `nextLink` and `deltaLink`, however the URLs are defined as being opaque and the existence of the tokens MUST NOT be documented. It is not a breaking change to modify the structure of `nextLinks` or `deltaLinks`.- |
| 69 | +- `nextLink` and `deltaLink` URLs are valid for a specific period before the client application needs to run a full synchronization again.For `nextLink`, a minimal validity time should be 1 hour. For `deltaLink`, a minimal validity time should be seven days. When a link is no longer valid it must return a standard error with a 410 GONE response code. |
| 70 | +- Although this capability is similar to the OData `$delta` feed capability, it is a different construct. Microsoft Graph APIs MUST provide change tracking through the delta function and MUST NOT implement the OData `$delta` feed when providing change tracking capabilities to ensure the uniformity of the API experience. |
| 71 | +- The Graph delta payload format has some deviations from the OData 4.01 change tracking format to simplify parsing, for example the context annotation is removed. |
| 72 | +- Additional implementation details are documented [internally](https://dev.azure.com/msazure/One/_wiki/wikis/Microsoft%20Graph%20Partners/211718/Deltas). |
| 73 | + |
| 74 | + |
| 75 | +## Alternatives |
| 76 | + |
| 77 | +- Change notifications pattern with rich payloads – for use cases where API consumers would find calling back into Microsoft Graph onerous and absolute integrity guarantees are less critical. |
| 78 | + |
| 79 | + |
| 80 | +## Examples |
| 81 | + |
| 82 | +### Change tracking on entity set |
| 83 | + |
| 84 | +``` |
| 85 | +<Function Name="delta" IsBound="true"> |
| 86 | + <Parameter Name="bindingParameter" Type="Collection(graph.user)" /> |
| 87 | + <ReturnType Type="Collection(graph.user)" /> |
| 88 | +</Function> |
| 89 | +<EntitySet Name="users" EntityType="graph.user"> |
| 90 | + <Annotation Term="Org.OData.Capabilities.V1.ChangeTracking"> |
| 91 | + <Record> |
| 92 | + <PropertyValue Property="Supported" Bool="true" /> |
| 93 | + </Record> |
| 94 | + </Annotation> |
| 95 | +</EntitySet> |
| 96 | +``` |
| 97 | + |
| 98 | +### Change tracking on navigation property |
| 99 | + |
| 100 | +``` |
| 101 | +<EntityType Name="educationRoot"> |
| 102 | + <NavigationProperty Name="classes" Type="Collection(graph.educationClass)" ContainsTarget="true" /> |
| 103 | + <NavigationProperty Name="me" Type="graph.educationUser" ContainsTarget="true" /> |
| 104 | + <NavigationProperty Name="schools" Type="Collection(graph.educationSchool)" ContainsTarget="true" /> |
| 105 | + <NavigationProperty Name="synchronizationProfiles" Type="Collection(graph.educationSynchronizationProfile)" ContainsTarget="true"/> |
| 106 | + <NavigationProperty Name="users" Type="Collection(graph.educationUser)" ContainsTarget="true" /> |
| 107 | +</EntityType> |
| 108 | +<Function Name="delta" IsBound="true"> |
| 109 | + <Parameter Name="bindingParameter" Type="Collection(graph.educationClass)" /> |
| 110 | + <ReturnType Type="Collection(graph.educationClass)" /> |
| 111 | +</Function> |
| 112 | + <Annotations Target="microsoft.graph.educationRoot/classes"> |
| 113 | + <Annotation Term="Org.OData.Capabilities.V1.ChangeTracking"> |
| 114 | + <Record> |
| 115 | + <PropertyValue Property="Supported" Bool="true" /> |
| 116 | + </Record> |
| 117 | + </Annotation> |
| 118 | +</Annotations> |
| 119 | +``` |
| 120 | +### Change tracking on function that return an entity collection |
| 121 | + |
| 122 | +Firstly, an API designer needs to define the function as composable (so that a delta function can be added to it), by adding the `IsComposable` annotation: |
| 123 | + |
| 124 | +```xml |
| 125 | +<Function Name="getAllRecordings" IsBound="true" EntitySetPath="bindingParameter/recordings" IsComposable="true"> |
| 126 | + <Parameter Name="bindingParameter" Type="Collection(self.onlineMeeting)" /> |
| 127 | + <ReturnType Type="Collection(self.meetingRecording)" /> |
| 128 | +</Function> |
| 129 | +``` |
| 130 | + |
| 131 | +Next, define the `delta` function. The binding parameter and the return type of the delta function MUST be the same as the return type of the target `getAllRecordings` function: |
| 132 | + |
| 133 | +```xml |
| 134 | +<Function Name="delta" IsBound="true" EntitySetPath="bindingParameter"> |
| 135 | + <Parameter Name="bindingParameter" Type="Collection(self.meetingRecording)" /> |
| 136 | + <ReturnType Type="Collection(self.meetingRecording)" /> |
| 137 | +</Function> |
| 138 | +``` |
| 139 | +Finally, for the function, the designer needs to add an annotation (either as a child of the entity or by targeting the entity type as below) stating that it supports change tracking (delta query): |
| 140 | + |
| 141 | +```xml |
| 142 | +<Annotations Target="self.getAllRecordings(Collection(self.onlineMeeting))"> |
| 143 | + <Annotation Term="Org.OData.Capabilities.V1.ChangeTracking"> |
| 144 | + <Record> |
| 145 | + <PropertyValue Property="Supported" Bool="true" /> |
| 146 | + </Record> |
| 147 | + </Annotation> |
| 148 | +</Annotations> |
| 149 | +``` |
| 150 | +Here is the HTTP request to start the change tracking process on `getAllRecordings` |
| 151 | + |
| 152 | +```http |
| 153 | +GET https://graph.microsoft.com/v1.0/communications/onlineMeetings/getAllRecordings/delta |
| 154 | +``` |
| 155 | + |
| 156 | +### Delta payload |
| 157 | + |
| 158 | + Here after the initial delta call, a user resource is updated, and there is one user added to and one removed from that user’s directReports collection. Additionally, a second user is deleted. In this case, there are no further pages of change records currently available. For detailed sequence of requests see [Change Tracking](https://learn.microsoft.com/en-us/graph/delta-query-overview?tabs=http#use-delta-query-to-track-changes-in-a-resource-collection). |
| 159 | + |
| 160 | +``` |
| 161 | +GET https://graph.microsoft.com/v1.0/users/delta?$skiptoken=pqwSUjGYvb3jQpbwVAwEL7yuI3dU1LecfkkfLPtnIjvB7XnF_yllFsCrZJ |
| 162 | +
|
| 163 | +{ |
| 164 | + "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users", |
| 165 | + "@odata.deltaLink": "https://graph.microsoft.com/v1.0/users/delta?$deltatoken=mS5DuRZGjVL-abreviated", |
| 166 | + "value": [ |
| 167 | + { |
| 168 | + "businessPhones": ["+1 309 555 0104"], |
| 169 | + "displayName": "Grady Archie", |
| 170 | + "givenName": "Grady", |
| 171 | + "jobTitle": "Designer", |
| 172 | + |
| 173 | + "officeLocation": "19/2109", |
| 174 | + "preferredLanguage": "en-US", |
| 175 | + "surname": "Archie", |
| 176 | + "userPrincipalName": "[email protected]", |
| 177 | + "id": "0baaae0f-b0b3-4645-867d-742d8fb669a2", |
| 178 | + "directReports@delta": [ |
| 179 | + { |
| 180 | + "@odata.type": "#microsoft.graph.user", |
| 181 | + "id": "99789584-a1e1-4232-90e5-866170e3d4e7" |
| 182 | + } , |
| 183 | + { |
| 184 | + "id": "66789583-f1e2-6232-70e5-366170e3d4a6", |
| 185 | + "@removed": { |
| 186 | + "reason": "deleted" |
| 187 | + } |
| 188 | + } |
| 189 | + ] |
| 190 | + }, |
| 191 | + { |
| 192 | + "id": "0bbbbb0f-b0b3-4645-867d-742d8fb669a2", |
| 193 | + "@removed": { |
| 194 | + "reason": "changed" |
| 195 | + } |
| 196 | + } |
| 197 | + ] |
| 198 | +} |
| 199 | +``` |
0 commit comments