Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to Contribute

Thank you for taking the time to contribute to `msgspec`!
Thank you for taking the time to contribute to `msgspec-x`!

Here we document some contribution guidelines to help you ensure that your
contribution is at its best.
Expand All @@ -17,17 +17,17 @@ Once you have those installed, you're ready to:

- Clone the repository
- Install all development dependencies
- Build a development version of `msgspec`
- Build a development version of `msgspec-x`
- Install the `pre-commit` hooks

```bash
# Clone the repository
git clone https://github.com/jcrist/msgspec.git
git clone https://github.com/nightsailer/msgspec-x.git

# cd into the repo root directory
cd msgspec/
cd msgspec-x/

# Build and install msgspec & all dev dependencies
# Build and install msgspec-x & all dev dependencies
pip install -e ".[dev]"

# Install the pre-commit hooks
Expand All @@ -36,9 +36,9 @@ pre-commit install

## Editing and Rebuilding

You now have a "development" build of `msgspec` installed. This means that you
You now have a "development" build of `msgspec-x` installed. This means that you
can make changes to the `.py` files and test them without requiring a rebuild
of msgspec's C extension. Edit away!
of msgspec-x's C extension. Edit away!

If you do make changes to a `.c` file, you'll need to recompile. You can do
this by running
Expand All @@ -47,7 +47,7 @@ this by running
pip install -e .
```

By default `msgspec` is built in release mode, with optimizations enabled. To
By default `msgspec-x` is built in release mode, with optimizations enabled. To
build a debug build instead (for use with e.g. `gdb` or `lldb`) define the
`MSGSPEC_DEBUG` environment variable before building.

Expand Down Expand Up @@ -112,6 +112,6 @@ and fix any issues that come up.

## Code of Conduct

``msgspec`` has a code of conduct that must be followed by all contributors to
``msgspec-x`` has a code of conduct that must be followed by all contributors to
the project. You may read the code of conduct
[here](https://github.com/jcrist/msgspec/blob/main/CODE_OF_CONDUCT.md).
[here](https://github.com/nightsailer/msgspec-x/blob/main/CODE_OF_CONDUCT.md).
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 🪲 Bug Report
description: Report a bug or unexpected behavior in msgspec
description: Report a bug or unexpected behavior in msgspec-x
body:
- type: markdown
attributes:
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 🙌 Feature Request
description: Suggest a new feature or change to msgspec
description: Suggest a new feature or change to msgspec-x
body:
- type: markdown
attributes:
Expand Down
4 changes: 2 additions & 2 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Security Policy

If you believe you have found a security-related bug with `msgspec`, **do not
open a public GitHub issue**. Instead, please email jcristharif@gmail.com.
If you believe you have found a security-related bug with `msgspec-x`, **do not
open a public GitHub issue**. Instead, please email nightsailer@gmail.com.

Please include as much detail as you would for a normal issue in your report.
In particular, including a minimal reproducible example will help the
Expand Down
3 changes: 3 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ include msgspec/*.h
include msgspec/*.py
include msgspec/*.pyi
include msgspec/py.typed
include msgspec_x/*.py
include msgspec_x/*.pyi
include msgspec_x/py.typed
include setup.py
include versioneer.py
include README.md
Expand Down
186 changes: 116 additions & 70 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,121 +1,167 @@
# Msgspec-x


<p align="center">
<a href="https://jcristharif.com/msgspec/">
<a href="https://nightsailer.github.io/msgspec-x/">
<img src="https://raw.githubusercontent.com/jcrist/msgspec/main/docs/source/_static/msgspec-logo-light.svg" width="35%" alt="msgspec" />
</a>
</p>

<p align="center">
<a href="https://github.com/jcrist/msgspec/actions/workflows/ci.yml">
<img src="https://github.com/jcrist/msgspec/actions/workflows/ci.yml/badge.svg">
<a href="https://github.com/nightsailer/msgspec-x/actions/workflows/ci.yml">
<img src="https://github.com/nightsailer/msgspec-x/actions/workflows/ci.yml/badge.svg">
</a>
<a href="https://jcristharif.com/msgspec/">
<a href="https://nightsailer.github.io/msgspec-x/">
<img src="https://img.shields.io/badge/docs-latest-blue.svg">
</a>
<a href="https://github.com/jcrist/msgspec/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/jcrist/msgspec.svg">
<a href="https://github.com/nightsailer/msgspec-x/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/nightsailer/msgspec-x.svg">
</a>
<a href="https://pypi.org/project/msgspec/">
<img src="https://img.shields.io/pypi/v/msgspec.svg">
<a href="https://pypi.org/project/msgspec-x/">
<img src="https://img.shields.io/pypi/v/msgspec-x.svg">
</a>
<a href="https://anaconda.org/conda-forge/msgspec">
<img src="https://img.shields.io/conda/vn/conda-forge/msgspec.svg">
<a href="https://anaconda.org/conda-forge/msgspec-x">
<img src="https://img.shields.io/conda/vn/conda-forge/msgspec-x.svg">
</a>
<a href="https://codecov.io/gh/jcrist/msgspec">
<img src="https://codecov.io/gh/jcrist/msgspec/branch/main/graph/badge.svg">
<a href="https://codecov.io/gh/nightsailer/msgspec-x">
<img src="https://codecov.io/gh/nightsailer/msgspec-x/branch/main/graph/badge.svg">
</a>
</p>

## Overview

`msgspec` is a *fast* serialization and validation library, with builtin
support for [JSON](https://json.org), [MessagePack](https://msgpack.org),
[YAML](https://yaml.org), and [TOML](https://toml.io). It features:
`msgspec-x` is a community-driven fork of the [original msgspec library](https://jcristharif.com/msgspec/) by Jim Crist-Harif. This project was created to address the challenge of slow upstream maintenance and to provide a platform for community contributions that couldn't be timely integrated into the original project.

- 🚀 **High performance encoders/decoders** for common protocols. The JSON and
MessagePack implementations regularly
[benchmark](https://jcristharif.com/msgspec/benchmarks.html) as the fastest
options for Python.
### Why msgspec-x?

- 🎉 **Support for a wide variety of Python types**. Additional types may be
supported through
[extensions](https://jcristharif.com/msgspec/extending.html).
The original msgspec library is an excellent project, but the maintainer has limited time to review and merge community pull requests. This has resulted in valuable contributions and bug fixes being stuck in the review process. `msgspec-x` was created to:

- 🔍 **Zero-cost schema validation** using familiar Python type annotations. In
[benchmarks](https://jcristharif.com/msgspec/benchmarks.html) `msgspec`
decodes *and* validates JSON faster than
[orjson](https://github.com/ijl/orjson) can decode it alone.
- **Accelerate community contributions**: Provide a faster path for community PRs and enhancements
- **Enable rapid bug fixes**: Address issues without waiting for upstream review cycles
- **Extend functionality**: Add new features that complement the original design
- **Maintain compatibility**: Keep full backward compatibility with the original msgspec API

- ✨ **A speedy Struct type** for representing structured data. If you already
use [dataclasses](https://docs.python.org/3/library/dataclasses.html) or
[attrs](https://www.attrs.org),
[structs](https://jcristharif.com/msgspec/structs.html) should feel familiar.
However, they're
[5-60x faster](https://jcristharif.com/msgspec/benchmarks.html#benchmark-structs>)
for common operations.
### **⚠️ IMPORTANT: Installation Notice**

All of this is included in a
[lightweight library](https://jcristharif.com/msgspec/benchmarks.html#benchmark-library-size)
with no required dependencies.
**Do not install both `msgspec` and `msgspec-x` simultaneously!** They are conflicting packages that cannot coexist in the same environment. If you have the original `msgspec` installed, uninstall it first:

```bash
pip uninstall msgspec
pip install msgspec-x
```

---
## Dual Namespace Architecture

`msgspec` may be used for serialization alone, as a faster JSON or
MessagePack library. For the greatest benefit though, we recommend using
`msgspec` to handle the full serialization & validation workflow:
`msgspec-x` provides two distinct namespaces to serve different needs:

**Define** your message schemas using standard Python type annotations.
### 1. `msgspec` Namespace - Full Compatibility
The `msgspec` namespace maintains 100% API compatibility with the original library. All your existing code will work without any changes:

```python
>>> import msgspec
import msgspec # Drop-in replacement for original msgspec

>>> class User(msgspec.Struct):
... """A new type describing a User"""
... name: str
... groups: set[str] = set()
... email: str | None = None
class User(msgspec.Struct):
name: str
email: str

# All existing msgspec code works exactly the same
user = User("alice", "[email protected]")
data = msgspec.json.encode(user)
decoded = msgspec.json.decode(data, type=User)
```

**Encode** messages as JSON, or one of the many other supported protocols.
### 2. `msgspec_x` Namespace - Extended Features
The `msgspec_x` namespace provides additional functionality and enhancements not available in the original library:

```python
>>> alice = User("alice", groups={"admin", "engineering"})
import msgspec_x # Extended features and community contributions

# Extended features will be documented as they are added
# This namespace allows for innovative features without breaking compatibility
```

>>> alice
User(name='alice', groups={"admin", "engineering"}, email=None)
## Core Features

>>> msg = msgspec.json.encode(alice)
`msgspec-x` inherits all the powerful features from the original msgspec library:

>>> msg
b'{"name":"alice","groups":["admin","engineering"],"email":null}'
- 🚀 **High performance encoders/decoders** for JSON, MessagePack, YAML, and TOML
- 🎉 **Support for a wide variety of Python types** with extension capabilities
- 🔍 **Zero-cost schema validation** using Python type annotations
- ✨ **Fast Struct type** for structured data representation
- 📦 **Lightweight library** with no required dependencies

All protocols and performance characteristics are maintained from the original implementation.

## Quick Start

### Installation

```bash
pip install msgspec-x
```

**Decode** messages back into Python objects, with optional schema validation.
### Basic Usage

Define your message schemas using standard Python type annotations:

```python
>>> msgspec.json.decode(msg, type=User)
User(name='alice', groups={"admin", "engineering"}, email=None)
import msgspec

>>> msgspec.json.decode(b'{"name":"bob","groups":[123]}', type=User)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
msgspec.ValidationError: Expected `str`, got `int` - at `$.groups[0]`
class User(msgspec.Struct):
"""A new type describing a User"""
name: str
groups: set[str] = set()
email: str | None = None
```

`msgspec` is designed to be as performant as possible, while retaining some of
the nicities of validation libraries like
[pydantic](https://pydantic-docs.helpmanual.io/). For supported types,
encoding/decoding a message with `msgspec` can be
[~10-80x faster than alternative libraries](https://jcristharif.com/msgspec/benchmarks.html).
Encode messages as JSON or other supported protocols:

```python
alice = User("alice", groups={"admin", "engineering"})
msg = msgspec.json.encode(alice)
# Output: b'{"name":"alice","groups":["admin","engineering"],"email":null}'
```

Decode messages back into Python objects with schema validation:

```python
# Successful decoding
user = msgspec.json.decode(msg, type=User)

# Validation error example
msgspec.json.decode(b'{"name":"bob","groups":[123]}', type=User)
# Raises: ValidationError: Expected `str`, got `int` - at `$.groups[0]`
```

## Performance

`msgspec-x` maintains the same exceptional performance characteristics as the original msgspec library. In benchmarks, it can be 10-80x faster than alternative libraries for encoding/decoding with validation.

<p align="center">
<a href="https://jcristharif.com/msgspec/benchmarks.html">
<img src="https://raw.githubusercontent.com/jcrist/msgspec/main/docs/source/_static/bench-validation.svg">
</a>
</p>

See [the documentation](https://jcristharif.com/msgspec/) for more information.
## Community & Contributing

This project welcomes community contributions! Unlike the original project, we aim to provide faster review cycles and more responsive maintenance.

- 🐛 **Bug Reports**: Issues are addressed promptly
- 🚀 **Feature Requests**: Community-driven feature development
- 🔧 **Pull Requests**: Faster review and merge process
- 📚 **Documentation**: Community-maintained documentation improvements

## Documentation

For detailed documentation, examples, and API references, visit:
- **Project Documentation**: [https://nightsailer.github.io/msgspec-x/](https://nightsailer.github.io/msgspec-x/)
- **Original msgspec docs**: [https://jcristharif.com/msgspec/](https://jcristharif.com/msgspec/) (for reference)

## License

New BSD License. See the [License File](https://github.com/nightsailer/msgspec-x/blob/main/LICENSE).

## LICENSE
## Acknowledgments

New BSD. See the
[License File](https://github.com/jcrist/msgspec/blob/main/LICENSE).
Special thanks to Jim Crist-Harif for creating the original msgspec library. This fork exists to complement and extend his excellent work, not to replace it.
4 changes: 2 additions & 2 deletions docs/source/_templates/help.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<h3>Need help?</h3>

<p>
Open an issue in the <a href="https://github.com/jcrist/msgspec/issues">issue tracker</a>.
</p>
Open an issue in the <a href="https://github.com/nightsailer/msgspec-x/issues">issue tracker</a>.
</p>
4 changes: 4 additions & 0 deletions docs/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ Structs

.. autoclass:: Struct

**StructMeta**

The metaclass for Struct types. This class can be subclassed to create custom struct behaviors. See :ref:`struct-meta-subclasses` for detailed information on using StructMeta subclasses.

.. autofunction:: field

.. autofunction:: defstruct
Expand Down
1 change: 1 addition & 0 deletions docs/source/benchmarks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ Benchmarks
==========

.. note::
These benchmarks are for ``msgspec-x``, a community-driven fork of the original msgspec project.

Benchmarks are *hard*.

Expand Down
Loading