Skip to content

Commit a03bd49

Browse files
handrewskarenetheridge
handrews
and
karenetheridge
committed
Sequential media type support.
The fundamental approach in this change was proposed by Karen Etheridge. This adds "itemSchema" as a schema to apply to each entry in a sequential media type instance. It also defines how to map sequential media tyes for use with "schema", and explains that "schema" applies to complete content only. JSON-based and text/event-stream media types are included. Co-authored-by: Karen Etheridge <[email protected]>
1 parent e889471 commit a03bd49

File tree

1 file changed

+310
-37
lines changed

1 file changed

+310
-37
lines changed

‎src/oas.md‎

Lines changed: 310 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,40 @@ Some examples of possible media type definitions:
8484
application/vnd.github.v3.patch
8585
```
8686

87+
JSON-based and JSON-compatible YAML-based media types can make direct use of the [Schema Object](#schema-object) as the Object uses JSON Schema.
88+
The use of the Schema Object with other media types is handled by mapping them into the JSON Schema [instance data model](https://www.ietf.org/archive/id/draft-bhutton-json-schema-01.html#name-instance-data-model).
89+
These mappings may be implicit based on the media type, or explicit based on the values of particular fields.
90+
Each mapping is addressed where the relevant media type is discussed in this section or under the [Media Type Object](#media-type-object) or [Encoding Object](#encoding-object)
91+
92+
#### Sequential Media Types
93+
94+
Within this specification, a _sequential media type_ is defined as any media type that consists of a repeating structure, separated by some delimiter, without any sort of header, footer, envelope, or other metadata in addition to the sequence.
95+
96+
Some examples of sequential media types (including some that are not IANA-registered but are in common use) are:
97+
98+
```text
99+
application/jsonl
100+
application/x-ndjson
101+
application/json-seq
102+
application/geo+json-seq
103+
text/event-stream
104+
```
105+
106+
In the first three above, the repeating structure is any [JSON value](https://tools.ietf.org/html/rfc8259#section-3).
107+
The fourth repeats `application/geo+json`-structured values, while the last repeats a custom text format related to Server-Sent Events.
108+
109+
Implementations MUST support mapping sequential media types into the JSON Schema data model by treating them as if the values were in an array in the same order.
110+
111+
See [Complete vs Streaming Content](#complete-vs-streaming-content) for more information on handling sequential media type in a streaming context, including special considerations for `text/event-stream` content.
112+
113+
#### Media Type Registry
114+
115+
While the [Schema Object](#schema-object) is designed to describe and validate JSON, several other media types are commonly used in APIs.
116+
Requirements regarding support for other media types are documented in this Media Types section and in several Object sections later in this specification.
117+
For convenience and future extensibility, these are cataloged in the OpenAPI Initiative's [Media Type Registry](https://spec.openapis.org/registry/media-type/), which indicates where in this specification the relevant requirements can be found.
118+
119+
See also the [Media Type Object](#media-type-object) for further information on working with specific media types.
120+
87121
### HTTP Status Codes
88122

89123
The HTTP Status Codes are used to indicate the status of the executed operation.
@@ -279,7 +313,7 @@ The `contentMediaType` keyword is redundant if the media type is already set:
279313

280314
If the [Schema Object](#schema-object) will be processed by a non-OAS-aware JSON Schema implementation, it may be useful to include `contentMediaType` even if it is redundant. However, if `contentMediaType` contradicts a relevant Media Type Object or Encoding Object, then `contentMediaType` SHALL be ignored.
281315

282-
The `maxLength` keyword MAY be used to set an expected upper bound on the length of a streaming payload. The keyword can be applied to either string data, including encoded binary data, or to unencoded binary data. For unencoded binary, the length is the number of octets.
316+
See [Complete vs Streaming Content](#complete-vs-streaming-content) for guidance on streaming binary payloads.
283317

284318
##### Migrating binary descriptions from OAS 3.0
285319

@@ -1602,7 +1636,8 @@ content:
16021636

16031637
#### Media Type Object
16041638

1605-
Each Media Type Object provides schema and examples for the media type identified by its key.
1639+
Each Media Type Object describes content structured in accordance with the media type identified by its key.
1640+
Multiple Media Type Objects can be used to describe content that can appear in any of several different media types.
16061641

16071642
When `example` or `examples` are provided, the example SHOULD match the specified schema and be in the correct format as specified by the media type and its encoding.
16081643
The `example` and `examples` fields are mutually exclusive, and if either is present it SHALL _override_ any `example` in the schema.
@@ -1612,50 +1647,80 @@ See [Working With Examples](#working-with-examples) for further guidance regardi
16121647

16131648
| Field Name | Type | Description |
16141649
| ---- | :----: | ---- |
1615-
| <a name="media-type-schema"></a>schema | [Schema Object](#schema-object) | The schema defining the content of the request, response, parameter, or header. |
1650+
| <a name="media-type-schema"></a>schema | [Schema Object](#schema-object) | A schema describing the complete content of the request, response, parameter, or header. |
1651+
| <a name="media-type-item-schema"></a>itemSchema> | [Schema Object](#schema-object) | A schema describing each item within a [sequential media type](#sequential-media-types). |
16161652
| <a name="media-type-example"></a>example | Any | Example of the media type; see [Working With Examples](#working-with-examples). |
16171653
| <a name="media-type-examples"></a>examples | Map[ `string`, [Example Object](#example-object) \| [Reference Object](#reference-object)] | Examples of the media type; see [Working With Examples](#working-with-examples). |
16181654
| <a name="media-type-encoding"></a>encoding | Map[`string`, [Encoding Object](#encoding-object)] | A map between a property name and its encoding information. The key, being the property name, MUST exist in the schema as a property. The `encoding` field SHALL only apply to [Request Body Objects](#request-body-object), and only when the media type is `multipart` or `application/x-www-form-urlencoded`. If no Encoding Object is provided for a property, the behavior is determined by the default values documented for the Encoding Object. |
16191655

16201656
This object MAY be extended with [Specification Extensions](#specification-extensions).
16211657

1622-
##### Media Type Examples
1658+
See also the [Media Type Registry](#media-type-registry).
16231659

1624-
```json
1625-
{
1626-
"application/json": {
1627-
"schema": {
1628-
"$ref": "#/components/schemas/Pet"
1629-
},
1630-
"examples": {
1631-
"cat": {
1632-
"summary": "An example of a cat",
1633-
"value": {
1634-
"name": "Fluffy",
1635-
"petType": "Cat",
1636-
"color": "White",
1637-
"gender": "male",
1638-
"breed": "Persian"
1639-
}
1640-
},
1641-
"dog": {
1642-
"summary": "An example of a dog with a cat's name",
1643-
"value": {
1644-
"name": "Puma",
1645-
"petType": "Dog",
1646-
"color": "Black",
1647-
"gender": "Female",
1648-
"breed": "Mixed"
1649-
}
1650-
},
1651-
"frog": {
1652-
"$ref": "#/components/examples/frog-example"
1653-
}
1654-
}
1655-
}
1656-
}
1660+
##### Complete vs Streaming Content
1661+
1662+
The `schema` field MUST be applied to the complete content, as defined by the media type and the context ([Request Body Object](#request-body-object), [Response Object](#response-object), [Parameter Object](#parameter-object), or [Header Object](#header-object).
1663+
Unless some sort of streaming JSON Schema processor is available, this requires loading the entire content into memory.
1664+
This poses a challenge for streamed media, particularly streams where the client is intended to choose when to stop reading as there is no well-defined end to the stream.
1665+
1666+
###### Binary Streams
1667+
1668+
The `maxLength` keyword MAY be used to set an expected upper bound on the length of a streaming payload.
1669+
The keyword can be applied to either string data, including encoded binary data, or to unencoded binary data. For unencoded binary, the length is the number of octets.
1670+
For this use case, `maxLength` MAY be implemented outside of regular JSON Schema evaluation as JSON Schema does not directly apply to binary data, and an encoded binary stream may be impractical to store in memory in its entirety.
1671+
1672+
###### Streaming Sequential Media Types
1673+
1674+
The `itemSchema` field is provided to support streaming use case for sequential media types.
1675+
Unlike `schema`, which is applied to the complete content (treated as an array as described in the [sequential media types](#sequential-media-types) section), `itemSchema` MUST be applied to each item in the stream independently, which supports processing each item as it is read from the stream.
1676+
1677+
Both `schema` and `itemSchema` MAY be used in the same Media Type Object, although doing so is unlikely to have significant advantages over using the `items` keyword within the `schema` field.
1678+
OpenAPI Description authors are responsible for avoiding the use of the `schema` in any situation where tooling may not be able to discern when the content is complete.
1679+
For example, if partial content is read from a stream and then passed with the `schema` value to a schema evaluator that is unaware of the stream context, the results are well-defined but will not be meaningful in terms of validating a keyword like `maxItems` as there may be additional items in the stream that are unknown to the schema evaluator.
1680+
1681+
##### Special Considerations for `text/event-stream` Content
1682+
1683+
For `text/event-stream`, each item in the array MUST be treated as if it were a JSON object with property names taken from the left side of the `:`, property values from the right side, and consecutive lines with the same name treated as a single property, with the value combined in accordance with the [`text/event-stream` specification](https://html.spec.whatwg.org/multipage/iana.html#text/event-stream).
1684+
1685+
Field names can be repeated within an item to allow splitting the value across multiple lines; such split values MUST be treated the same as if they were a single field, with newlines added as required by the [`text/event-stream` specification](https://html.spec.whatwg.org/multipage/iana.html#text/event-stream).
1686+
1687+
The `text/event-stream` specification requires that fields with Unknown names, as well as `id` fields where the value contains `U+0000 NULL` be ignored.
1688+
These fields SHOULD NOT be present in the data used with the Schema Object.
1689+
1690+
Field value types MUST be handled as specified by the `text/event-stream` specification (e.g. the `retry` field value is modeled as a JSON number that is expected to be of JSON Schema `type: integer`), and fields not given an explicit value type MUST be handled as strings.
1691+
1692+
Some users of `text/event-stream` use a format such as JSON for field values, particularly the `data` field.
1693+
Use JSON Schema's keywords for working with the [contents of string-encoded data](https://www.ietf.org/archive/id/draft-bhutton-json-schema-validation-01.html#name-a-vocabulary-for-the-conten), particularly `contentMediaType` and `contentSchema`, to describe and validate such fields with more detail than string-related validation keywords such as `pattern` can support.
1694+
Note that `contentSchema` is [not automatically validated by default](https://www.ietf.org/archive/id/draft-bhutton-json-schema-validation-01.html#name-implementation-requirements-2) (see also the [Non-validating constraint keywords](#non-validating-constraint-keywords) section of this specification).
1695+
1696+
The following Schema Object is a generic schema for the `text/event-stream` media type as documented by the HTML specification as of the time of this writing:
1697+
1698+
```YAML
1699+
type: array
1700+
items:
1701+
type: object
1702+
required:
1703+
- data
1704+
properties:
1705+
data:
1706+
type: string
1707+
event:
1708+
type: string
1709+
id:
1710+
type: string
1711+
retry:
1712+
type: integer
16571713
```
16581714

1715+
##### Media Type Examples
1716+
1717+
For form-related media type examples, see the [Encoding Object](#encoding-object).
1718+
1719+
###### JSON
1720+
1721+
Note that since this example is written in YAML, the Example Object `value` field can be formatted as YAML due to the trivial conversion to JSON.
1722+
This avoids needing to embed JSON as a string.
1723+
16591724
```yaml
16601725
application/json:
16611726
schema:
@@ -1681,6 +1746,214 @@ application/json:
16811746
$ref: '#/components/examples/frog-example'
16821747
```
16831748

1749+
Alternatively, since all JSON is valid YAML, the example value can use JSON syntax within a YAML document:
1750+
1751+
```yaml
1752+
application/json:
1753+
schema:
1754+
$ref: '#/components/schemas/Pet'
1755+
examples:
1756+
cat:
1757+
summary: An example of a cat
1758+
value: {
1759+
"name": "Fluffy",
1760+
"petType": "Cat",
1761+
"color": "White",
1762+
"gender": "male",
1763+
"breed": "Persian"
1764+
}
1765+
dog:
1766+
summary: An example of a dog with a cat's name
1767+
value: {
1768+
"name": "Puma",
1769+
"petType": "Dog",
1770+
"color": "Black",
1771+
"gender": "Female",
1772+
"breed": "Mixed"
1773+
}
1774+
frog:
1775+
$ref: '#/components/examples/frog-example'
1776+
```
1777+
1778+
###### Sequential JSON
1779+
1780+
For any [sequential media type](#sequential-media-types) where the items in the sequence are JSON values, no conversion of each value is required.
1781+
JSON Text Sequences ([[?RFC7464]] `application/json-seq` and [[?RFC8091]] the `+json-seq` structured suffix), [JSON Lines](https://jsonlines.org/) (`application/jsonl`), and [NDJSON](https://github.com/ndjson/ndjson-spec) (`application/x-ndjson`) are all in this category.
1782+
Note that the media types for JSON Lines and NDJSON are not registered with the IANA, but are in common use.
1783+
1784+
The following example shows Media Type Objects for both streaming log entries and returning a fixed-length set in response to a query.
1785+
This shows the relationship between `schema` and `itemSchema`, and when to use each even though the `examples` field is the same either way.
1786+
1787+
```YAML
1788+
components:
1789+
schemas:
1790+
LogEntry:
1791+
type: object
1792+
properties:
1793+
timestamp:
1794+
type: string
1795+
format: date-time
1796+
level:
1797+
type: integer
1798+
minimum: 0
1799+
message:
1800+
type: string
1801+
Log:
1802+
type: array
1803+
items:
1804+
$ref: "#/components/schemas/LogEntry"
1805+
maxItems: 100
1806+
examples:
1807+
LogJSONSeq:
1808+
summary: Log entries in application/json-seq
1809+
# JSON Text Sequences require an unprintable character
1810+
# that cannot be escaped in a YAML string, and therefore
1811+
# must be placed in an external document shown below
1812+
externalValue: examples/log.json-seq
1813+
LogJSONPerLine:
1814+
summary: Log entries in application/jsonl or application/x-ndjson
1815+
description: JSONL and NDJSON are identical for this example
1816+
# Note that the value must be written as a string with newlines,
1817+
# as JSONL and NDJSON are not valid YAML
1818+
value: |
1819+
{"timestamp": "1985-04-12T23:20:50.52Z", "level": 1, "message": "Hi!"}
1820+
{"timestamp": "1985-04-12T23:20:51.37Z", "level": 1, "message": "Bye!"}
1821+
responses:
1822+
LogStream:
1823+
description: |
1824+
A stream of JSON-format log messages that can be read
1825+
for as long as the application is running, and is available
1826+
in any of the sequential JSON media types.
1827+
content:
1828+
application/json-seq:
1829+
itemSchema:
1830+
$ref: "#/components/schemas/LogEntry"
1831+
examples:
1832+
JSON-SEQ:
1833+
$ref: "#/components/examples/LogJSONSeq"
1834+
application/jsonl:
1835+
itemSchema:
1836+
$ref: "#/components/schemas/LogEntry"
1837+
examples:
1838+
JSONL:
1839+
$ref: "#/components/examples/LogJSONPerLine"
1840+
application/x-ndjson:
1841+
itemSchema:
1842+
$ref: "#/components/schemas/LogEntry"
1843+
examples:
1844+
NDJSON:
1845+
$ref: "#/components/examples/LogJSONPerLine"
1846+
LogExcerpt:
1847+
description: |
1848+
A response consisting of no more than 100 log records,
1849+
generally as a result of a query of the historical log,
1850+
available in any of the sequential JSON media types.
1851+
content:
1852+
application/json-seq:
1853+
schema:
1854+
$ref: "#/components/schemas/Log"
1855+
examples:
1856+
JSON-SEQ:
1857+
$ref: "#/components/examples/LogJSONSeq"
1858+
application/jsonl:
1859+
schema:
1860+
$ref: "#/components/schemas/Log"
1861+
examples:
1862+
JSONL:
1863+
$ref: "#/components/examples/LogJSONPerLine"
1864+
application/x-ndjson:
1865+
schema:
1866+
$ref: "#/components/schemas/Log"
1867+
examples:
1868+
NDJSON:
1869+
$ref: "#/components/examples/LogJSONPerLine"
1870+
```
1871+
1872+
Our `application/json-seq` example has to be an external document because of the use of both newlines and of the unprintable Record Separator (`0x1E`) character, which cannot be escaped in YAML block literals:
1873+
1874+
```JSONSEQ
1875+
0x1E{
1876+
"timestamp": "1985-04-12T23:20:50.52Z",
1877+
"level": 1,
1878+
"message": "Hi!"
1879+
}
1880+
0x1E{
1881+
"timestamp": "1985-04-12T23:20:51.37Z",
1882+
"level": 1,
1883+
"message": "Bye!"
1884+
}
1885+
```
1886+
1887+
###### Server-Sent Event Streams
1888+
1889+
For this example, assume that the generic event schema provided in the "Special Considerations for `text/event-stream` Content" section is available at `#/components/schemas/Event`:
1890+
1891+
```YAML
1892+
description: A request body to add a stream of typed data.
1893+
required: true
1894+
content:
1895+
text/event-stream:
1896+
itemSchema:
1897+
$ref: "#/components/schemas/Event"
1898+
required: [event]
1899+
oneOf:
1900+
- properties:
1901+
event:
1902+
const: addString
1903+
- properties:
1904+
event:
1905+
const: addNumber
1906+
data:
1907+
$comment: |
1908+
Since the data field is a string,
1909+
we need a format to signal that
1910+
it should be handled as a number
1911+
format: double
1912+
- properties:
1913+
event:
1914+
const: addJson
1915+
data:
1916+
$comment: |
1917+
These content fields indicate
1918+
that the string value should
1919+
be parsed and validated as a
1920+
JSON document (since JSON is not
1921+
a binary format, contentEncoding
1922+
is not needed)
1923+
contentMediaType: application/json
1924+
contentSchema:
1925+
type: object
1926+
required: [foo]
1927+
properties:
1928+
foo:
1929+
type: integer
1930+
```
1931+
1932+
The following `text/event-stream` document is an example of a valid request body for the above example:
1933+
1934+
```EVENTSTREAM
1935+
event: addString
1936+
data: This data is formatted
1937+
data: across two lines
1938+
retry: 5
1939+
1940+
event: addNumber
1941+
data: 1234.5678
1942+
unknownField: this is ignored
1943+
1944+
: This is a comment
1945+
event: addJSON
1946+
data: {"foo": 42}
1947+
```
1948+
1949+
To more clearly see how this stream is handled, the following is the equivalent JSON Lines document, which shows how the numeric and JSON data are handled as strings, and how unknown fields and comments are ignored and not passed to schema validation:
1950+
1951+
```JSONL
1952+
{"event": "addString", "data": "This data is formatted\nacross two lines", "retry": 5}
1953+
{"event": "addNumber", "data": "1234.5678"}
1954+
{"event": "addJSON", "data": "{\"foo\": 42}"}
1955+
```
1956+
16841957
##### Considerations for File Uploads
16851958

16861959
In contrast to OpenAPI 2.0, `file` input/output content in OAS 3.x is described with the same semantics as any other schema type.

0 commit comments

Comments
 (0)