Skip to content

Rewrite binary serialization#3804

Open
clementleclercRTE wants to merge 14 commits intomainfrom
cl-binary-named-attributes
Open

Rewrite binary serialization#3804
clementleclercRTE wants to merge 14 commits intomainfrom
cl-binary-named-attributes

Conversation

@clementleclercRTE
Copy link
Contributor

Please check if the PR fulfills these requirements

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • A PR or issue has been opened in all impacted repositories (if any)

Does this PR already have an issue describing the problem?

What kind of change does this PR introduce?

Feature — redesign of the binary IIDM serialization format (BinReader / BinWriter).

What is the current behavior?

The binary format uses a positional (sequential) attribute encoding: attributes are written in a fixed order and read back in the same order. The name parameter passed to read*Attribute("name") is
completely ignored — the reader blindly consumes bytes from the stream. As a result:

  • Reading attributes out of order silently produces corrupted values with no exception.
  • skipNode() is not implemented and throws PowsyblException, making it impossible to skip unknown or private extensions.
  • Absent values (NaN, absentValue, null) are still written to the stream, wasting space.

What is the new behavior (if this is a feature change)?

Attributes are now named: a global dictionary maps attribute names to compact 1-byte indices. Each attribute occurrence in the body is encoded as [byte: index][byte: type][value]. Before a node's children are processed, all its attributes are pre-read into a Map<String, Object> keyed by name.

Does this PR introduce a breaking change or deprecate an API?

  • Yes
  • No

If yes, please check if the following requirements are fulfilled

  • The Breaking Change or Deprecated label has been added
  • The migration steps are described in the following section

What changes might users need to make in their application due to this PR? (migration steps)

The binary file format is not backward compatible. Binary files produced before this PR cannot be read by the new code, and vice versa. Since the binary format was already marked as BETA with no stability guarantee across releases, no migration path is provided. Users should re-export any existing .biidm files using the new version.

Other information:

Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
…n XML, JSON, and binary serialization

Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
…ort for skipping extensions

Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
Signed-off-by: Leclerc Clement <clement.leclerc@rte-france.com>
private String[] attrNames;
private byte[] attrTypes;

private Map<String, Object> currentAttrs = Collections.emptyMap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't buffer attributes, just read them on the fly

@sonarqubecloud
Copy link

@rolnico rolnico self-requested a review March 19, 2026 15:32
Comment on lines +115 to +119
if (index > 255) {
throw new PowsyblException("Binary format: too many distinct attribute (name, type) pairs (max 255)");
}
try {
attrDos.writeByte(index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are checking that the index is not more than 255, but in java Byte is signed, shouldn't it be 127 instead ? How about using Byte.MAX_VALUE instead ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants