Skip to content

Commit 962ef54

Browse files
authored
Merge pull request #23 from compas-dev/copilot/add-protobuf-support
Add extensible codec system with Protocol Buffers support and remove IronPython
2 parents 49615f8 + 187e7eb commit 962ef54

25 files changed

+756
-313
lines changed

.github/workflows/build.yml

Lines changed: 3 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -27,29 +27,6 @@ jobs:
2727
run: |
2828
pytest tests/unit
2929
30-
build-ironpython:
31-
name: windows-ironpython
32-
runs-on: windows-latest
33-
steps:
34-
- uses: actions/checkout@v2
35-
- name: Install dependencies
36-
shell: cmd
37-
run: |
38-
curl -o compas.tar.gz -LJO https://pypi.debian.net/compas/latest
39-
curl -o ironpython-pytest.tar.gz -LJO https://pypi.debian.net/ironpython-pytest/latest
40-
choco install ironpython --version=2.7.8.1
41-
ipy -X:Frames -m ensurepip
42-
ipy -X:Frames -m pip install --no-deps ironpython-pytest.tar.gz
43-
44-
rem untar and rename, these cannot be installed using ironpip because they not longer have a setup.py
45-
tar -xf compas.tar.gz && for /d %%i in (compas-*) do ren "%%i" compas
46-
47-
- name: Run tests
48-
env:
49-
IRONPYTHONPATH: ./src;./compas/src
50-
run: |
51-
ipy -m pytest tests/unit
52-
5330
integration_tests:
5431
if: "!contains(github.event.pull_request.labels.*.name, 'docs-only')"
5532
runs-on: 'ubuntu-latest'
@@ -63,6 +40,9 @@ jobs:
6340
python: '3.11'
6441
invoke_lint: false
6542
invoke_test: false
43+
- name: Install test dependencies
44+
run: |
45+
pip install -r tests/integration/requirements.txt
6646
- name: Run integration tests
6747
run: |
6848
pytest tests/integration

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1010
### Added
1111

1212
* Added support for MQTT-PAHO 2.0 versioned callbacks.
13+
* Added `MessageCodec` abstract base class for extensible message serialization.
14+
* Added `JsonMessageCodec` for JSON-based message serialization (default).
15+
* Added `ProtobufMessageCodec` for binary message serialization using Protocol Buffers (requires `compas_pb`).
16+
* Added `codec` parameter to `Transport`, `InMemoryTransport`, and `MqttTransport` classes.
1317

1418
### Changed
1519

1620
* Updated dependency on `paho-mqtt` to support `>=1, <3` to include version `2.x` with backward compatibility.
1721

1822
### Removed
1923

24+
* **BREAKING**: Removed IronPython support and `mqtt_cli` implementation.
25+
* **BREAKING**: Removed support for Rhino 7 (IronPython-based).
26+
2027

2128
## [1.0.0] 2024-05-27
2229

README.md

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Or using `conda`:
3535
* Publisher/subscriber communication model (N-to-N communication)
3636
* In-process events
3737
* MQTT support
38-
* CPython & IronPython support
38+
* Extensible codec system for message serialization (JSON, Protocol Buffers)
3939

4040
## Examples
4141

@@ -78,13 +78,28 @@ for i in range(10):
7878
This example shows how to send and receive from a single script, but
7979
running publishers and subscribers on different scripts, different processes, or even different computers will work the exact same way.
8080

81+
### Using different codecs
8182

82-
### Usage from Rhinoceros 3D
83+
By default, COMPAS EVE uses JSON for message serialization. However, you can use different codecs for more efficient serialization:
8384

84-
It is possible to use the same code from within Rhino/Grasshopper.
85+
```python
86+
import compas_eve as eve
87+
from compas_eve import JsonMessageCodec
88+
from compas_eve.codecs import ProtobufMessageCodec
89+
from compas_eve.mqtt import MqttTransport
8590

86-
Make sure you have installed it to Rhino using the COMPAS installation mechanism:
91+
# Use JSON codec (default)
92+
json_codec = JsonMessageCodec()
93+
tx = MqttTransport("broker.hivemq.com", codec=json_codec)
8794

88-
```bash
89-
python -m compas_rhino.install -v 7.0
95+
# Or use Protocol Buffers for binary serialization (requires compas_pb)
96+
pb_codec = ProtobufMessageCodec()
97+
tx = MqttTransport("broker.hivemq.com", codec=pb_codec)
9098
```
99+
100+
101+
### Usage from Rhinoceros 3D
102+
103+
It is possible to use the same code from within Rhino/Grasshopper.
104+
105+
To install `compas_eve`, use the the syntax `# r: compas_eve` at the top of any Python 3.x script in Rhino/Grasshopper.

docs/api.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ API Reference
66
:maxdepth: 1
77

88
api/compas_eve
9-
api/compas_eve.mqtt
9+
api/compas_eve.codecs
1010
api/compas_eve.memory
11+
api/compas_eve.mqtt
1112
api/compas_eve.ghpython
1213

docs/api/compas_eve.codecs.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
2+
.. automodule:: compas_eve.codecs

docs/api/compas_eve.rst

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1 @@
1-
********************************************************************************
2-
compas_eve
3-
********************************************************************************
4-
51
.. automodule:: compas_eve

examples/codec_usage.py

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
"""
2+
Example demonstrating custom codec usage with COMPAS EVE.
3+
4+
This example shows how to:
5+
1. Use the default JsonMessageCodec
6+
2. Create a custom codec
7+
3. Use ProtobufMessageCodec (if compas_pb is installed)
8+
"""
9+
10+
import json
11+
12+
import compas_eve as eve
13+
from compas_eve import MessageCodec
14+
from compas_eve.codecs import JsonMessageCodec
15+
16+
17+
# Example 1: Using the default JSON codec (implicit)
18+
print("=" * 60)
19+
print("Example 1: Default JSON Codec (implicit)")
20+
print("=" * 60)
21+
22+
pub = eve.Publisher("/example/default")
23+
sub = eve.EchoSubscriber("/example/default")
24+
sub.subscribe()
25+
26+
pub.publish(eve.Message(text="Hello with default JSON codec", count=1))
27+
print()
28+
29+
30+
# Example 2: Explicitly using JSON codec
31+
print("=" * 60)
32+
print("Example 2: Explicit JSON Codec")
33+
print("=" * 60)
34+
35+
json_codec = JsonMessageCodec()
36+
transport = eve.InMemoryTransport(codec=json_codec)
37+
38+
pub = eve.Publisher("/example/json", transport=transport)
39+
sub = eve.EchoSubscriber("/example/json", transport=transport)
40+
sub.subscribe()
41+
42+
pub.publish(eve.Message(text="Hello with explicit JSON codec", count=2))
43+
print()
44+
45+
46+
# Example 3: Custom codec
47+
print("=" * 60)
48+
print("Example 3: Custom Codec")
49+
print("=" * 60)
50+
51+
52+
class UpperCaseCodec(MessageCodec):
53+
"""A simple custom codec that converts all string values to uppercase."""
54+
55+
def encode(self, message):
56+
"""Encode message by converting all string values to uppercase."""
57+
# Assume message is a Message instance
58+
data = message.data
59+
60+
# Convert string values to uppercase
61+
encoded_data = {}
62+
for key, value in data.items():
63+
if isinstance(value, str):
64+
encoded_data[key] = value.upper()
65+
else:
66+
encoded_data[key] = value
67+
return json.dumps(encoded_data)
68+
69+
def decode(self, encoded_data, message_type):
70+
"""Decode data (strings remain uppercase)."""
71+
data = json.loads(encoded_data)
72+
return message_type.parse(data)
73+
74+
75+
custom_codec = UpperCaseCodec()
76+
custom_transport = eve.InMemoryTransport(codec=custom_codec)
77+
78+
pub = eve.Publisher("/example/custom", transport=custom_transport)
79+
80+
# Create a custom subscriber that prints the message
81+
class PrintSubscriber(eve.Subscriber):
82+
def message_received(self, message):
83+
print(f"Received: {message}")
84+
85+
sub = PrintSubscriber("/example/custom", transport=custom_transport)
86+
sub.subscribe()
87+
88+
pub.publish(eve.Message(text="hello world", count=3))
89+
print()
90+
91+
92+
# Example 4: Protocol Buffers codec (if available)
93+
print("=" * 60)
94+
print("Example 4: Protocol Buffers Codec (if available)")
95+
print("=" * 60)
96+
97+
try:
98+
from compas_eve.codecs import ProtobufMessageCodec
99+
100+
pb_codec = ProtobufMessageCodec()
101+
pb_transport = eve.InMemoryTransport(codec=pb_codec)
102+
103+
pub = eve.Publisher("/example/protobuf", transport=pb_transport)
104+
sub = eve.EchoSubscriber("/example/protobuf", transport=pb_transport)
105+
sub.subscribe()
106+
107+
pub.publish(eve.Message(text="Hello with Protocol Buffers", count=4))
108+
print("✓ Protocol Buffers codec is working!")
109+
110+
except ImportError as e:
111+
print(f"Protocol Buffers codec not available: {e}")
112+
print("Install with: pip install compas_pb")
113+
114+
print()
115+
print("=" * 60)
116+
print("All examples completed!")
117+
print("=" * 60)

pyproject.toml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,54 @@
1+
[build-system]
2+
requires = ["setuptools>=66.0"]
3+
build-backend = "setuptools.build_meta"
4+
5+
# ============================================================================
6+
# project info
7+
# ============================================================================
8+
9+
[project]
10+
name = "compas_eve"
11+
description = "COMPAS Event Extensions: adds event-based communication infrastructure to the COMPAS framework."
12+
keywords = ["events", "event-driven", "compas", "architecture", "distributed systems"]
13+
authors = [
14+
{ name = "Gonzalo Casas", email = "[email protected]" },
15+
{ name = "Chen Kasirer", email = "[email protected]" },
16+
]
17+
license = { file = "LICENSE" }
18+
readme = "README.md"
19+
requires-python = ">=3.9"
20+
dynamic = ['dependencies', 'optional-dependencies', 'version']
21+
classifiers = [
22+
"Development Status :: 4 - Beta",
23+
"Topic :: Scientific/Engineering",
24+
"Operating System :: Unix",
25+
"Operating System :: POSIX",
26+
"Operating System :: Microsoft :: Windows",
27+
"Programming Language :: Python",
28+
"Programming Language :: Python :: 3",
29+
"Programming Language :: Python :: 3.10",
30+
"Programming Language :: Python :: 3.11",
31+
"Programming Language :: Python :: 3.12",
32+
"Programming Language :: Python :: 3.13",
33+
"Programming Language :: Python :: 3.14",
34+
]
35+
36+
[project.entry-points.'compas_pb.plugins']
37+
serializers = 'compas_eve.codecs.conversions'
38+
39+
[project.urls]
40+
Homepage = "https://github.com/compas-dev/compas_eve"
41+
Documentation = "https://compas.dev/compas_eve"
42+
Repository = "https://github.com/compas-dev/compas_eve"
43+
Changelog = "https://github.com/compas-dev/compas_eve/blob/main/CHANGELOG.md"
44+
Issues = "https://github.com/compas-dev/compas_eve/issues"
45+
Forum = "https://forum.compas-framework.org/"
46+
47+
48+
# ============================================================================
49+
# setuptools config
50+
# ============================================================================
51+
152
[tool.black]
253
line-length = 120
354

requirements-dev.txt

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,12 @@
11
attrs >=17.4
2-
autopep8
3-
black
4-
bump2version >=1.0.1
5-
check-manifest >=0.36
6-
compas_invocations
7-
doc8
8-
flake8
2+
black >=22.12.0
3+
build
4+
bump-my-version
5+
compas_invocations2
6+
compas_pb >= 0.4.4
97
invoke >=0.14
10-
isort
11-
pydocstyle
12-
pytest >=3.2
8+
pytest-mock
9+
ruff
1310
sphinx_compas2_theme
1411
twine
15-
wheel
16-
-e .
12+
wheel

src/compas_eve/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
EchoSubscriber
2121
Transport
2222
InMemoryTransport
23+
MessageCodec
2324
get_default_transport
2425
set_default_transport
2526
@@ -47,6 +48,7 @@
4748
get_default_transport,
4849
set_default_transport,
4950
)
51+
from .codecs import MessageCodec
5052
from .memory import InMemoryTransport
5153

5254
HERE = os.path.dirname(__file__)
@@ -62,6 +64,7 @@
6264
"EchoSubscriber",
6365
"Topic",
6466
"Transport",
67+
"MessageCodec",
6568
"get_default_transport",
6669
"set_default_transport",
6770
"InMemoryTransport",

0 commit comments

Comments
 (0)