Skip to content

Commit da0926d

Browse files
authored
API docs about OData Dataset Service (getodk#788)
* API docs about OData Dataset Service * Added more details to select/filtering * Docs update in response to comments
1 parent 7b7deff commit da0926d

File tree

1 file changed

+217
-3
lines changed

1 file changed

+217
-3
lines changed

docs/api.md

Lines changed: 217 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ Here major and breaking changes to the API are listed by version.
3434

3535
### ODK Central v2023.2
3636

37+
**Added**:
38+
- New [OData Dataset Service](#reference/odata-endpoints/odata-dataset-service) for each `Dataset` that provides a list of `Entities`.
39+
3740
**Changed**:
3841
- The response of `GET`, `POST`, `PUT` and `PATCH` methods of [Submissions](#reference/submissions/listing-all-submissions-on-a-form) endpoint has been updated to include metadata of the `currentVersion` of the Submission.
3942

@@ -3428,15 +3431,15 @@ While OData itself supports data of any sort of structure, Power BI and Tableau
34283431

34293432
In general, the OData standard protocol consists of three API endpoints:
34303433

3431-
* The **Service Document** describes the available resources in the service. We provide one of these for every `Form` in the system. In our case, these are the tables we derive from the `repeat`s in the given Form. The root table is always named `Submissions`.
3434+
* The **Service Document** describes the available resources in the service. We provide one of these for every `Form` in the system. As of version 2023.2, we also provide one for every `Dataset`.
34323435
* The **Metadata Document** is a formal XML-based EDMX schema description of every data object we might return. It is linked in every OData response.
3433-
* The actual data documents, linked from the Service Document, are a simple JSON representation of the submission data, conforming to the schema we describe in our Metadata Document.
3436+
* The actual data documents, linked from the Service Document, are a simple JSON representation of the submission data or entity, conforming to the schema we describe in our Metadata Document.
34343437

34353438
As our focus is on the bulk-export of data from ODK Central so that more advanced analysis tools can handle the data themselves, we do not support most of the features at the Intermediate and above conformance levels, like `$sort` or `$filter`.
34363439

34373440
## OData Form Service [/v1/projects/{projectId}/forms/{xmlFormId}.svc]
34383441

3439-
ODK Central presents one OData service for every `Form` it knows about. Each service might have multiple tables related to that Form. To access the OData service, simply add `.svc` to the resource URL for the given Form.
3442+
ODK Central presents one OData service for every `Form` it knows about. To access the OData service, simply add `.svc` to the resource URL for the given Form.
34403443

34413444
+ Parameters
34423445
+ projectId: `7` (number, required) - The numeric ID of the Project
@@ -3658,6 +3661,217 @@ Because this `/#/dl` path returns a web page that causes a file download rather
36583661

36593662
(html markup data)
36603663

3664+
3665+
## OData Dataset Service [/v1/projects/{projectId}/datasets/{name}.svc]
3666+
3667+
ODK Central presents one OData service for every `Dataset` as a way to get an OData feed of `Entities`. To access the OData service, simply add `.svc` to the resource URL for the given Dataset.
3668+
3669+
+ Parameters
3670+
+ projectId: `7` (number, required) - The numeric ID of the Project
3671+
3672+
+ `name`: `trees` (string, required) - The `name` of the `Dataset` whose OData service you wish to access.
3673+
3674+
### Service Document [GET]
3675+
3676+
The Service Document provides a link to the main source of information in this OData service: the list of `Entities` in this `Dataset`, as well as the Metadata Document describing the schema of this information.
3677+
3678+
This document is available only in JSON format.
3679+
3680+
+ Response 200 (application/json; charset=utf-8; odata.metadata=minimal)
3681+
+ Body
3682+
3683+
{
3684+
"@odata.context": "https://your.odk.server/v1/projects/7/datasets/trees.svc/$metadata",
3685+
"value": [
3686+
{
3687+
"kind": "EntitySet",
3688+
"name": "Entities",
3689+
"url": "Entities"
3690+
},
3691+
]
3692+
}
3693+
3694+
+ Response 403 (application/json)
3695+
+ Attributes (Error 403)
3696+
3697+
+ Response 406 (application/json)
3698+
+ Attributes (Error 406)
3699+
3700+
### Metadata Document [GET /v1/projects/{projectId}/datasets/{name}.svc/$metadata]
3701+
3702+
The Metadata Document describes, in [EDMX CSDL](http://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html), the schema of all the data you can retrieve from the OData Dataset Service in question. Essentially, these are the Dataset properties, or the schema of each `Entity`, translated into the OData format.
3703+
3704+
+ Response 200 (application/xml)
3705+
+ Body
3706+
3707+
<?xml version="1.0" encoding="UTF-8"?>
3708+
<edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">
3709+
<edmx:DataServices>
3710+
<Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="org.opendatakit.entity">
3711+
<ComplexType Name="metadata">
3712+
<Property Name="createdAt" Type="Edm.DateTimeOffset"/>
3713+
<Property Name="creatorId" Type="Edm.String"/>
3714+
<Property Name="creatorName" Type="Edm.String"/>
3715+
</ComplexType>
3716+
</Schema>
3717+
<Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="org.opendatakit.user.trees">
3718+
<EntityType Name="Entities">
3719+
<Key>
3720+
<PropertyRef Name="__id"/>
3721+
</Key>
3722+
<Property Name="__id" Type="Edm.String"/>
3723+
<Property Name="__system" Type="org.opendatakit.entity.metadata"/>
3724+
<Property Name="name" Type="Edm.String"/>
3725+
<Property Name="label" Type="Edm.String"/>
3726+
<Property Name="geometry" Type="Edm.String"/>
3727+
<Property Name="species" Type="Edm.String"/>
3728+
<Property Name="circumference_cm" Type="Edm.String"/>
3729+
</EntityType>
3730+
<EntityContainer Name="trees">
3731+
<EntitySet Name="Entities" EntityType="org.opendatakit.user.trees.Entities">
3732+
<Annotation Term="Org.OData.Capabilities.V1.ConformanceLevel" EnumMember="Org.OData.Capabilities.V1.ConformanceLevelType/Minimal"/>
3733+
<Annotation Term="Org.OData.Capabilities.V1.BatchSupported" Bool="false"/>
3734+
<Annotation Term="Org.OData.Capabilities.V1.CountRestrictions">
3735+
<Record>
3736+
<PropertyValue Property="Countable" Bool="true"/>
3737+
</Record>
3738+
</Annotation>
3739+
<Annotation Term="Org.OData.Capabilities.V1.FilterFunctions">
3740+
<Record>
3741+
<PropertyValue Property="NonCountableProperties">
3742+
<Collection>
3743+
<String>eq</String>
3744+
</Collection>
3745+
</PropertyValue>
3746+
</Record>
3747+
</Annotation>
3748+
<Annotation Term="Org.OData.Capabilities.V1.FilterFunctions">
3749+
<Record>
3750+
<PropertyValue Property="Filterable" Bool="true"/>
3751+
<PropertyValue Property="RequiresFilter" Bool="false"/>
3752+
<PropertyValue Property="NonFilterableProperties">
3753+
<Collection>
3754+
<PropertyPath>geometry</PropertyPath>
3755+
<PropertyPath>species</PropertyPath>
3756+
<PropertyPath>circumference_cm</PropertyPath>
3757+
</Collection>
3758+
</PropertyValue>
3759+
</Record>
3760+
</Annotation>
3761+
<Annotation Term="Org.OData.Capabilities.V1.SortRestrictions">
3762+
<Record>
3763+
<PropertyValue Property="Sortable" Bool="false"/>
3764+
</Record>
3765+
</Annotation>
3766+
<Annotation Term="Org.OData.Capabilities.V1.ExpandRestrictions">
3767+
<Record>
3768+
<PropertyValue Property="Expandable" Bool="false"/>
3769+
</Record>
3770+
</Annotation>
3771+
</EntitySet>
3772+
</EntityContainer>
3773+
</Schema>
3774+
</edmx:DataServices>
3775+
</edmx:Edmx>
3776+
3777+
+ Response 403 (application/json)
3778+
+ Attributes (Error 403)
3779+
3780+
+ Response 406 (application/json)
3781+
+ Attributes (Error 406)
3782+
3783+
### Data Document [GET /v1/projects/{projectId}/datasets/{name}.svc/Entities{?%24skip,%24top,%24count,%24filter,%24select}]
3784+
3785+
A data document is the straightforward JSON representation of all the `Entities` in a `Dataset`.
3786+
3787+
The `$top` and `$skip` querystring parameters, specified by OData, apply `limit` and `offset` operations to the data, respectively. The `$count` parameter, also an OData standard, will annotate the response data with the total row count, regardless of the scoping requested by `$top` and `$skip`. While paging is possible through these parameters, it will not greatly improve the performance of exporting data. ODK Central prefers to bulk-export all of its data at once if possible.
3788+
3789+
The [`$filter` querystring parameter](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358948)can be used to filter by any data field in the system-level schema, but not the Dataset properties. The operators `lt`, `le`, `eq`, `ne`, `ge`, `gt`, `not`, `and`, and `or` are supported. The built-in functions `now`, `year`, `month`, `day`, `hour`, `minute`, `second` are supported.
3790+
3791+
The fields you can query against are as follows:
3792+
3793+
| Entity Metadata | OData Field Name |
3794+
| ------------------------| -------------------- |
3795+
| Entity UUID | `__id` |
3796+
| Entity Name (same as UUID) | `name` |
3797+
| Entity Label | `label` |
3798+
| Entity Creator Actor ID | `__system/creatorId` |
3799+
| Entity Timestamp | `__system/createdAt` |
3800+
3801+
Note that `createdAt` is a time component. This means that any comparisons you make need to account for the full time of the entity. It might seem like `$filter=__system/createdAt le 2020-01-31` would return all results on or before 31 Jan 2020, but in fact only entities made before midnight of that day would be accepted. To include all of the month of January, you need to filter by either `$filter=__system/createdAt le 2020-01-31T23:59:59.999Z` or `$filter=__system/createdAt lt 2020-02-01`. Remember also that you can [query by a specific timezone](https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC).
3802+
3803+
Please see the [OData documentation](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358948) on `$filter` [operations](http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_BuiltinFilterOperations) and [functions](http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_BuiltinQueryFunctions) for more information.
3804+
3805+
The [`$select` query parameter](http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#_Toc31358942) will return just the fields you specify and is supported on `__id`, `__system`, `__system/creatorId` and `__system/createdAt`, as well as on user defined properties.
3806+
3807+
As the vast majority of clients only support the JSON OData format, that is the only format ODK Central offers.
3808+
3809+
+ Parameters
3810+
+ `%24skip`: `10` (number, optional) - If supplied, the first `$skip` rows will be omitted from the results.
3811+
+ `%24top`: `5` (number, optional) - If supplied, only up to `$top` rows will be returned in the results.
3812+
+ `%24count`: `true` (boolean, optional) - If set to `true`, an `@odata.count` property will be added to the result indicating the total number of rows, ignoring the above paging parameters.
3813+
+ `%24filter`: `year(__system/createdAt) lt year(now())` (string, optional) - If provided, will filter responses to those matching the query. Only [certain fields](/reference/odata-endpoints/odata-form-service/data-document) are available to reference. The operators `lt`, `le`, `eq`, `neq`, `ge`, `gt`, `not`, `and`, and `or` are supported, and the built-in functions `now`, `year`, `month`, `day`, `hour`, `minute`, `second`.
3814+
+ `%24select`: `__id, label, name` (string, optional) - If provided, will return only the selected fields.
3815+
3816+
3817+
+ Response 200 (application/json)
3818+
+ Body
3819+
3820+
{
3821+
"@odata.context": "https://your.odk.server/v1/projects/7/datasets/trees.svc/$metadata#Entities"",
3822+
"value": [
3823+
{
3824+
"__id": "0f56bde5-dd05-41f7-8175-c4114eab41c6",
3825+
"name": "0f56bde5-dd05-41f7-8175-c4114eab41c6",
3826+
"label": "25cm purpleheart",
3827+
"__system": {
3828+
"createdAt": "2022-12-09T19:41:16.478Z",
3829+
"creatorId": "39",
3830+
"creatorName": "Tree surveyor"
3831+
},
3832+
"geometry": "32.7413996 -117.1394617 53.80000305175781 13.933",
3833+
"species": "purpleheart",
3834+
"circumference_cm": "25"
3835+
},
3836+
{
3837+
"__id": "aeebd746-3b1e-4a24-ba9d-ed6547bd5ff1",
3838+
"name": "aeebd746-3b1e-4a24-ba9d-ed6547bd5ff1",
3839+
"label": "345cm mora",
3840+
"__system": {
3841+
"createdAt": "2022-11-21T19:17:36.348Z",
3842+
"creatorId": "8",
3843+
"creatorName": "[email protected]"
3844+
},
3845+
"geometry": "47.722581 18.562111 0 0",
3846+
"species": "mora",
3847+
"circumference_cm": "345"
3848+
},
3849+
{
3850+
"__id": "eacb9844-2f88-48b5-b7c0-6333263fe639",
3851+
"name": "eacb9844-2f88-48b5-b7c0-6333263fe639",
3852+
"label": "123cm wallaba",
3853+
"__system": {
3854+
"createdAt": "2022-11-21T18:22:43.759Z",
3855+
"creatorId": "8",
3856+
"creatorName": "[email protected]"
3857+
},
3858+
"geometry": "",
3859+
"species": "wallaba",
3860+
"circumference_cm": "123"
3861+
}
3862+
]
3863+
}
3864+
3865+
+ Response 403 (application/json)
3866+
+ Attributes (Error 403)
3867+
3868+
+ Response 406 (application/json)
3869+
+ Attributes (Error 406)
3870+
3871+
+ Response 501 (application/json)
3872+
+ Attributes (Error 501)
3873+
3874+
36613875
## Draft Testing [/v1/projects/{projectId}/forms/{xmlFormId}/draft.svc]
36623876

36633877
_(introduced: version 0.8)_

0 commit comments

Comments
 (0)