A few ideas about JSON serializer/deserializer

[Iwan91]

1)I know that there is attribute "ComponentKey" for changing component name when serializing/deserialaizing.
I can't find in project attribute that do the same when serializing/deserialaizing variable.
Can you add support for something like "VariableKey".
Example:

 [ComponentKey("my-component")]
    public struct MyComponent : IComponent {
        [VariableKey("v")]
        public int value;
    }

2)I see that "EntitySerializer" only write output as "pretty". In code I can see that you have a parameter "pretty" in methods but it is set to always true in code. Can you add this parameter to constructor of EntitySerializer?
Example:
converter.EntityToDataEntity(entity, writeEntity, true);

3)Is planned support for serializing/deserializing in binary format or for example MessagePack or other format that take less space on disk and probably is faster to serialize/deserialize than JSON?
I ask because 1_000_000 entities with one component (above example 1) is around 85MB, after I zipped it it is around 5.5MB. It it still a lot. I know that parsing a large file of json is not very efficient on mobiles espacialy on android.

[friflo]

hi @Iwan91,
thx for your input!
Below my answers

1. Customize JSON serialization

I added some examples to the documentation
https://friflo.gitbook.io/friflo.engine.ecs/documentation/entity#json-serialization
Specific to your example it is:

    public struct MyComponent : IComponent {
        [Serialize("v")]
        public int value;
    }

2. pretty parameter to format JSON in EntitySerializer.EntityToDataEntity()

This parameter controls only the format of entire components - not their fields.
The JSON of a component is always written in a single line. This is by intention.
Imho - it makes it easier to read the data of an entity by humans.

If true each component is written in a separate line e.g.

{
    "name": {"value":"test"},
    "pos": {"x":1,"y":2,"z":3}
}

If false the generated JSON has no line feeds

{"name":{"value":"test"},"pos":{"x":1,"y":2,"z":3}}

3. Support of MessagePack or other binary formats.

Thought about this. But it requires significant effort.
It is currently not on my list but this can change fast if I need it myself.
I don't think there is much demand for this feature. It is simply too special.

If doing this it requirements are:

• Support MessagePack - its the most popular binary generic format in .NET
• Hard focus on optimization. I expect this format is mainly used for real time communication.
  • avoid any heap allocations if not necessary
  • No structural changes. An entity is created directly in its final Archetype.
    So no intermediate structural changes caused by adding components ony-by-one.

[friflo]

An additional note to JSON vs MessagePack serialization regarding the size.

The unzipped JSON version will be larger than unzipped MessagePack. JSON requires simply more control characters (bytes).
I would not expect a big difference in size if zipping both formats.

Btw:
How long does it take to write / read this 85 MB JSON on Windows?

[Iwan91]

On my computer:

Build mode: Release
OS : Windows 10
i7-4770K @ 4500Mhz (4 cores - 8 threads)
DDR3 2400 Mhz
SSD: Samsung 860 EVO 500GB

85MB file, 1_000_000 entities with one component that i posted in first example (1):
write about 1247ms
read about 1074ms

[Iwan91]

4)I looked at "EntitySerializer" and how it write/read data to JSON.
Maybe I am mistaken but if someone want to implement different serialization in other project I think it would be difficult, because some methods/members are internal. Maybe you can read all components in entity in different way than it shows in "EntitySerializer".
I don't know why you wrote code that way, maybe it is for performance reasons.

I think that JSON serializer/deserializer should be a separate project that depends on ECS framework.
This way you can be sure that API is available for others to use in future.

[friflo]

The intention of EntitySerializer is to enable JSON serialization out of the box without the need of writing any serialization code. Its implementation goals are performance and minimizing structural changes.

When creation a custom serializer EntitySerializer and all its internals should be completely ignored.
Making its internals public would not be useful for a custom serializer.

A custom generic Writer can use Entity.Components to get entity component data. These components can be written as text or binary.

A custom generic Reader read binary / text data.
Than create an entity with store.CreateEntity(). The deserialized data can be added using Entity
or EntityUtils methods.