error-stack RFC: Enhancing Serialization and Deserialization for Reports

[indietyp]

This proposal introduces improvements for serializing and deserializing error-stack Reports. While error-stack currently supports serialization, the functionality is limited and rigid. This RFC aims to address these limitations, proposing a more flexible approach to handling Reports during serialization.

Motivation

The current serialization process for Reports loses critical information, making deserialization problematic. This RFC proposes a revised format that retains more information and is designed to be consistent with our existing debug output.

Proposed Format

The new format differentiates between two types of objects: Context and Attachment. While their serialized forms appear similar, they serve distinct roles. Below is the proposed structure:

Context

{
	"typeId": "e8007964-c86e-4550-8977-23f3603b1fe4", // further explained below; can be null
	"message": "Internal Server Error", // the `Display` output of the `Context`
	"details": {}, // result of serde::Serialize
	"attachments": [],
	"opaqueAttachments": 0, // count of attachments that could not be serialized
	"children": [] // child contexts
}

Attachment

{
	"typeId": "5c197aee-489d-4269-a3e5-0e02604a078d",
	"message": "", // optional `Display` output or `DebugHook` output
	"details": {}, // result of serde::Serialize
	"shared": false // indicates if the attachment is part of multiple contexts
}

If message or details is absent, it means those fields don't exist for that object. The typeId helps during deserialization, acting as a unique identifier for reconstructing objects.

For example:

let report = Report::new(A)
	.attach(B)
	.attach(C)
	.attach(D);

This would serialize to:

A = {
	...,
	attachments: [B, C, D]
}

Here, the attachment order is preserved. This mirrors the existing Debug implementation, where the order is maintained rather than reversed (which might be more intuitive from an implementation perspective).

typeId and Improving Serde Hooks

Past attempts to move Debug hook behavior to serde have encountered issues, especially with deserialization. Specifically, important metadata is lost during serialization, making accurate reconstruction impossible.

The proposed typeId addresses this. It's akin to AssetId::Uuid in Bevy. For each pair of serde operations, we assign a (constant or non-constant) UUID to identify and register the type. While TypeId could serve a similar purpose, it isn’t stable across compilations, rendering it unsuitable for storage. A static user UUID solves this problem, albeit with some additional friction, which can be mitigated via a derive macro.

The API might look like this:

pub trait StableTypeId {
	fn stable_type_id() -> u128; // Provides a stable identifier
}

impl Report<()> {
	pub fn register_serde<T>(hook: Hook<T>) {..} // Registers a serialization hook
}

pub struct Hook<T> {
	id: u128, // Type ID for identifying the type
	serialize: Option<Box<dyn Fn(HookContext) -> erased_serde::Serialize>>, // Serialization function
	_marker: PhantomData<fn() -> *const T> // Ensures type uniqueness without storing the type
}

impl<T> Hook<T> {
	pub fn auto() where T: serde::Serialize + serde::de::DeserializeOwned + StableTypeId {..} // Automatically generate hooks for types that implement `serde`
	pub fn manual(id: u128) where T: serde::Serialize + serde::de::DeserializeOwned + StableTypeId {..} // Manually create a hook using a specific ID
	pub fn with_id(self, id: u128) -> Self {..} // Add a custom ID to a hook
	pub fn from_parts<U>(id: u128, serialize: impl Fn(HookContext<T>) -> U) -> Self where U: serde::Serialize, T: serde::Deserialize {..} // Create a hook with custom serialization logic
}

In this example, the StableTypeId trait provides a stable u128 identifier that helps track the type during serialization and deserialization, this is just a Uuid, but removing the reliance on Uuid allows easier implementations for the end-user. The Hook<T> struct allows users to register custom serialization logic for specific types. The auto method automatically handles this for types implementing serde, while manual allows for more control over the process.

During deserialization, the system checks the typeId and attempts to deserialize using the registered deserializer. If deserialization fails (for example, if the type wasn’t registered), the system falls back to a default approach instead of throwing an error.

Deserialization Fallback

When deserialization fails due to e.g. a missing or unknown typeId, we introduce two fallback constructs: AnyAttachment and AnyContext (name TBD). These types capture the data that couldn't be deserialized:

pub struct AnyContext {
	details: Option<serde_value::Value>, // Holds the details of the context if available
	message: Option<Box<str>> // Holds the message if available
}

This ensures deserialization never fails entirely, and users can still retrieve the available data. We could then provide an API like AnyContext::downcast<T> where T: serde::Deserialize to attempt a downcast to the appropriate type.

In cases where the last Context is unknown an end-user could fall back to an AnyReport, which is a simply an alias to Report<AnyContext>. While this potentially loses some information, it allows to reconstruct the most essential parts.

Improving the API for Lossy Serialization

Currently, one major downside is the inflexibility of serializing attachments. For every new capability, such as serialization or printing, we would need to create new combinations of methods like attach_printable or attach_serializable. As a result, the number of possible combinations grows exponentially, leading to unsustainable API design, and confusion to the end-user.

This serialization is considered lossy, because unlike the hook implementation there is no requirement for a type id, making it impossible to deserialize it with the correct type. They would still be deserializable, but as a AnyAttachment instead of T.

To simplify this, we propose a new Attachment<T> type. This abstraction allows the user to attach data with flexible capabilities without needing multiple API combinations. The new Attachment type would work as follows:

impl<T> From<T> for Attachment<T> {
	// Blanket implementation to convert any type into an attachment
}

With this change, the Report::attach method would have the signature:

Report::attach<T>(attachment: impl Into<Attachment<T>>)

The Attachment struct would provide methods for adding capabilities like printing or serialization in a chainable manner:

Attachment::printable(self) -> Self
Attachment::serializable(self) -> Self

Here’s an example of how this change reflects itself in the API:

Report::new(..)
	.attach(Attachment::new(T).printable().serializable());

Furthermore, to maintain backward compatibility and ease of use attach_printable will remain available and will not be deprecated:

Report::new(..)
	.attach_printable(T);

Future Possibilities

While I am usually not a fan of life before main, one could imagine a derive macro like thiserror::Error, that also registers the context automatically via a serde hook. This also the pro of us being able to support things thiserror isn't planning any time soon, like proper generic_member_access support. This is definitely not needed in this version, but something to think about in potentially a future version, it could look like:

#[derive(error_stack::Error)]
#[error("unable to find field)]
#[register(serializable)] // register via ctor and mark it to be serializable
#[provide(expr = ...)] // a custom expression, for example to provide a constant
pub struct Error {
	#[provide]
	expected: ExpectedField
	#[provide]
	missing: MissingField,
}