Skip to content

Commit 8ce49b8

Browse files
authored
HTTP binary binding (#15)
* Implements HTTP Binary binding * Improvements in field types implementation * Name formatter functions consistently * Refactor and improve tests * Promote stability to beta
1 parent d55e583 commit 8ce49b8

30 files changed

+1384
-577
lines changed

.github/workflows/python-code-style.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- name: Install dependencies
2525
run: |
2626
python -m pip install --upgrade pip
27-
python -m pip install poetry tox
27+
python -m pip install poetry poetry-plugin-export tox
2828
make poetry-export
2929
- name: Check code style with black
3030
run: |

.github/workflows/python-lint.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- name: Install dependencies
2525
run: |
2626
python -m pip install --upgrade pip
27-
python -m pip install poetry tox
27+
python -m pip install poetry poetry-plugin-export tox
2828
make poetry-export
2929
- name: Lint with ruff
3030
run: make lint

.github/workflows/python-typing.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- name: Install dependencies
2525
run: |
2626
python -m pip install --upgrade pip
27-
python -m pip install poetry tox
27+
python -m pip install poetry poetry-plugin-export tox
2828
make poetry-export
2929
- name: Check typing
3030
run: make typing

.idea/cloudevents-pydantic.iml

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.idea/misc.xml

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 41 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# cloudevents-pydantic
2+
23
![Static Badge](https://img.shields.io/badge/Python-3.9_%7C_3.10_%7C_3.11_%7C_3.12_%7C_3.13-blue?logo=python&logoColor=white)
34
[![Stable Version](https://img.shields.io/pypi/v/cloudevents-pydantic?color=blue)](https://pypi.org/project/cloudevents-pydantic/)
4-
[![stability-wip](https://img.shields.io/badge/stability-wip-lightgrey.svg)](https://github.com/mkenney/software-guides/blob/master/STABILITY-BADGES.md#work-in-progress)
5+
[![stability-beta](https://img.shields.io/badge/stability-beta-33bbff.svg)](https://github.com/mkenney/software-guides/blob/master/STABILITY-BADGES.md#beta)
56

67
[![Python tests](https://github.com/febus982/cloudevents-pydantic/actions/workflows/python-tests.yml/badge.svg?branch=main)](https://github.com/febus982/cloudevents-pydantic/actions/workflows/python-tests.yml)
78
[![Maintainability](https://api.codeclimate.com/v1/badges/c7fe3ebcadd850d7ed3f/maintainability)](https://codeclimate.com/github/febus982/cloudevents-pydantic/maintainability)
@@ -11,22 +12,14 @@
1112
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff)
1213
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
1314
[![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)
15+
[![Pydantic v2](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/pydantic/pydantic/main/docs/badge/v2.json)](https://pydantic.dev)
1416

1517
This is an implementation of the [CloudEvents spec](https://github.com/cloudevents/spec/tree/main) using
1618
[Pydantic V2](https://docs.pydantic.dev/latest/) for high performance during validation and serialization.
1719

1820
It is meant to support natively [FastAPI](https://fastapi.tiangolo.com/)
1921
and [FastStream](https://faststream.airt.ai/latest/) (WIP)
2022

21-
Currently supported bindings:
22-
23-
| Binding | Format | Single | Batch |
24-
|---------|:-------|:-------:|:-------:|
25-
| HTTP | JSON |||
26-
| HTTP | Binary | planned | planned |
27-
| KAFKA | JSON | planned | planned |
28-
| KAFKA | Binary | planned | planned |
29-
3023
## How to use
3124

3225
```shell
@@ -69,21 +62,52 @@ for details on how to create custom events.
6962
Using pydantic gives a great performance boost if compared to the official SDK. (there's obviously
7063
some performance issue in the official serialization using pydantic)
7164

72-
These results come from a Macbook Pro M3 Max on python 3.12. Feel free to run the `benchmark.py`
65+
These results come from a Macbook Pro M4 Pro on python 3.13. Feel free to run the `benchmark.py`
7366
script yourself.
7467

7568
```
69+
==== 1M iterations benchmark ====
7670
Timings for HTTP JSON deserialization:
77-
This package: 3.0855846670019673
78-
Official SDK with pydantic model: 15.35431600001175
79-
Official SDK with http model: 13.728038166998886
71+
This package: 2.3955996250006137
72+
Official SDK using pydantic model: 11.389213957998436
73+
Official SDK using http model: 10.174893917006557
8074
8175
Timings for HTTP JSON serialization:
82-
This package: 4.292417042001034
83-
Official SDK with pydantic model: 44.50933354199515
84-
Official SDK with http model: 8.929204874992138
76+
This package: 3.497491959002218
77+
Official SDK using pydantic model: 31.92037604199868
78+
Official SDK using http model: 6.780242209002608
8579
```
8680

81+
## Supported specification features
82+
83+
| Core Specification | [v1.0](https://github.com/cloudevents/spec/blob/v1.0.2/spec.md) |
84+
|--------------------|:---------------------------------------------------------------:|
85+
| CloudEvents Core ||
86+
87+
---
88+
89+
| Event Formats | [v1.0](https://github.com/cloudevents/spec/blob/v1.0.2/spec.md#event-format) |
90+
|-------------------|:----------------------------------------------------------------------------:|
91+
| AVRO Event Format ||
92+
| JSON Event Format ||
93+
94+
---
95+
96+
| Protocol Bindings | [v1.0](https://github.com/cloudevents/spec/blob/v1.0.2/spec.md#protocol-binding) |
97+
|------------------------|:--------------------------------------------------------------------------------:|
98+
| HTTP Protocol Binding ||
99+
| Kafka Protocol Binding ||
100+
101+
---
102+
103+
| Content Modes | [v1.0](https://github.com/cloudevents/spec/blob/v1.0.2/http-protocol-binding.md#13-content-modes) |
104+
|------------------|:-------------------------------------------------------------------------------------------------:|
105+
| HTTP Binary ||
106+
| HTTP Structured ||
107+
| HTTP Batch ||
108+
| Kafka Binary ||
109+
| Kafka Structured ||
110+
| Kafka Batch ||
87111

88112
## Commands for development
89113

benchmark.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -63,14 +63,15 @@ def json_deserialization_official_sdk_cloudevent():
6363
from_json_http(valid_json)
6464

6565

66+
print("==== 1M iterations benchmark ====")
6667
print("Timings for HTTP JSON deserialization:")
6768
print("This package: " + str(timeit(json_deserialization, number=test_iterations)))
6869
print(
69-
"Official SDK with pydantic model: "
70+
"Official SDK using pydantic model: "
7071
+ str(timeit(json_deserialization_official_sdk_pydantic, number=test_iterations))
7172
)
7273
print(
73-
"Official SDK with http model: "
74+
"Official SDK using http model: "
7475
+ str(timeit(json_deserialization_official_sdk_cloudevent, number=test_iterations))
7576
)
7677

@@ -101,10 +102,10 @@ def json_serialization_official_sdk_cloudevent():
101102
print("Timings for HTTP JSON serialization:")
102103
print("This package: " + str(timeit(json_serialization, number=test_iterations)))
103104
print(
104-
"Official SDK with pydantic model: "
105+
"Official SDK using pydantic model: "
105106
+ str(timeit(json_serialization_official_sdk_pydantic, number=test_iterations))
106107
)
107108
print(
108-
"Official SDK with http model: "
109+
"Official SDK using http model: "
109110
+ str(timeit(json_serialization_official_sdk_cloudevent, number=test_iterations))
110111
)

cloudevents_pydantic/bindings/http.py

Lines changed: 74 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,26 +21,42 @@
2121
# DEALINGS IN THE SOFTWARE. =
2222
# ==============================================================================
2323
from typing import (
24+
Any,
2425
Dict,
2526
Generic,
2627
List,
2728
NamedTuple,
29+
Optional,
2830
Type,
2931
TypeVar,
3032
cast,
3133
)
34+
from urllib.parse import quote, unquote
3235

3336
from pydantic import TypeAdapter
3437

3538
from cloudevents_pydantic.events import CloudEvent
36-
from cloudevents_pydantic.formats import json
39+
from cloudevents_pydantic.formats import canonical, json
3740

3841
_T = TypeVar("_T", bound=CloudEvent)
3942

4043

4144
class HTTPComponents(NamedTuple):
4245
headers: Dict[str, str]
43-
body: str
46+
body: Optional[str]
47+
48+
49+
_HTTP_safe_chars = "".join(
50+
[
51+
x
52+
for x in list(map(chr, range(ord("\u0021"), ord("\u007e") + 1)))
53+
if x not in [" ", '"', "%"]
54+
]
55+
)
56+
"""
57+
Characters NOT to be percent encoded in http headers.
58+
https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/http-protocol-binding.md#3132-http-header-values
59+
"""
4460

4561

4662
class HTTPHandler(Generic[_T]):
@@ -62,7 +78,7 @@ def to_json(self, event: _T) -> HTTPComponents:
6278
:rtype: HTTPComponents
6379
"""
6480
headers = {"content-type": "application/cloudevents+json; charset=UTF-8"}
65-
body = json.to_json(event)
81+
body = json.serialize(event)
6682
return HTTPComponents(headers, body)
6783

6884
def to_json_batch(self, events: List[_T]) -> HTTPComponents:
@@ -75,7 +91,7 @@ def to_json_batch(self, events: List[_T]) -> HTTPComponents:
7591
:rtype: HTTPComponents
7692
"""
7793
headers = {"content-type": "application/cloudevents-batch+json; charset=UTF-8"}
78-
body = json.to_json_batch(events, self.batch_adapter)
94+
body = json.serialize_batch(events, self.batch_adapter)
7995
return HTTPComponents(headers, body)
8096

8197
def from_json(
@@ -90,7 +106,7 @@ def from_json(
90106
:return: The deserialized event
91107
:rtype: CloudEvent
92108
"""
93-
return json.from_json(body, self.event_adapter)
109+
return json.deserialize(body, self.event_adapter)
94110

95111
def from_json_batch(
96112
self,
@@ -104,4 +120,56 @@ def from_json_batch(
104120
:return: The deserialized event batch
105121
:rtype: List[CloudEvent]
106122
"""
107-
return json.from_json_batch(body, self.batch_adapter)
123+
return json.deserialize_batch(body, self.batch_adapter)
124+
125+
def to_binary(self, event: _T) -> HTTPComponents:
126+
"""
127+
Serializes an event in HTTP binary format.
128+
129+
:param event: The event object to serialize
130+
:type event: CloudEvent
131+
:return: The headers and the body representation of the event
132+
:rtype: HTTPComponents
133+
"""
134+
if event.datacontenttype is None:
135+
raise ValueError("Can't serialize event without datacontenttype")
136+
137+
serialized = canonical.serialize(event)
138+
139+
body = serialized.get("data")
140+
headers = {
141+
f"ce-{k}": self._header_encode(v)
142+
for k, v in serialized.items()
143+
if k not in ["data", "datacontenttype"] and v is not None
144+
}
145+
headers["content-type"] = self._header_encode(serialized["datacontenttype"])
146+
147+
return HTTPComponents(headers, body)
148+
149+
def from_binary(self, headers: Dict[str, str], body: Any) -> CloudEvent:
150+
"""
151+
Deserializes an event from HTTP binary format.
152+
153+
:param headers: The request headers
154+
:type headers: Dict[str, str]
155+
:param body: The request body
156+
:type body: Any
157+
:return:
158+
"""
159+
if not headers.get("content-type"):
160+
raise ValueError("content-type not found in headers")
161+
162+
canonical_data = {
163+
k[3:]: self._header_decode(v)
164+
for k, v in headers.items()
165+
if k.startswith("ce-")
166+
}
167+
canonical_data["datacontenttype"] = self._header_decode(headers["content-type"])
168+
canonical_data["data"] = body
169+
return canonical.deserialize(canonical_data, self.event_adapter)
170+
171+
def _header_encode(self, value: str) -> str:
172+
return quote(value, safe=_HTTP_safe_chars)
173+
174+
def _header_decode(self, value: str) -> str:
175+
return unquote(value, errors="strict")

cloudevents_pydantic/events/_event.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,15 @@
4646
FieldTime,
4747
FieldType,
4848
)
49-
from .fields.types import URI, Binary, DateTime, SpecVersion, String, URIReference
49+
from .fields.types import (
50+
URI,
51+
Binary,
52+
MimeType,
53+
SpecVersion,
54+
String,
55+
Timestamp,
56+
URIReference,
57+
)
5058

5159
DEFAULT_SPECVERSION = SpecVersion.v1_0
5260

@@ -96,10 +104,10 @@ def event_factory(
96104
specversion: Annotated[SpecVersion, FieldSpecVersion]
97105

98106
# Optional fields
99-
time: Annotated[Optional[DateTime], Field(default=None), FieldTime]
107+
time: Annotated[Optional[Timestamp], Field(default=None), FieldTime]
100108
subject: Annotated[Optional[String], Field(default=None), FieldSubject]
101109
datacontenttype: Annotated[
102-
Optional[String], Field(default=None), FieldDataContentType
110+
Optional[MimeType], Field(default=None), FieldDataContentType
103111
]
104112
dataschema: Annotated[Optional[URI], Field(default=None), FieldDataSchema]
105113

cloudevents_pydantic/events/fields/types/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,10 @@
2525
URI,
2626
Binary,
2727
Boolean,
28-
DateTime,
2928
Integer,
29+
MimeType,
3030
SpecVersion,
3131
String,
32+
Timestamp,
3233
URIReference,
3334
)

0 commit comments

Comments
 (0)