Skip to content

Conversation

@loitly
Copy link
Contributor

@loitly loitly commented Jan 10, 2026

Ticket: https://jira.ipac.caltech.edu/browse/FIREFLY-1925

This PR replaces Redis values currently stored as JSON strings and Java-serialized objects with MessagePack.

  • New values are serialized using MessagePack.
  • Reads remain backward compatible and can still handle existing Java-serialized and JSON values.

Motivation:

  • Smaller payloads: Reduce Redis memory usage and network traffic by using a compact binary format.
  • Future extensibility: Leverage Jackson as a single serialization framework (shared configuration, annotations, modules) across formats. e.g. JSON and XML.

Implementation Details:

  • Added Jackson + MessagePack dependencies.
  • Implemented a new Redis serializer.
    • Primary format: MessagePack for all new writes.
    • Backward-compatible reads.
      • Can still deserialize existing Java-serialized objects and JSON values.
  • Existing API for Redis access is unchanged. Only different internal serialization mechanism has been updated.

Unit Tests:

  • See edu.caltech.ipac.util.serialization.SerializerTest
    • It should cover all current use cases.

Test Instances:
https://firefly-1925-redis-value-messagepack.irsakubedev.ipac.caltech.edu/irsaviewer/
https://fireflydev.ipac.caltech.edu/firefly-1925-redis-value-messagepack/firefly/

All behavior should remain the same as before the change.

@loitly loitly added this to the 2026.1 milestone Jan 10, 2026
@loitly loitly requested a review from robyww January 10, 2026 00:37
@loitly loitly self-assigned this Jan 10, 2026
@loitly loitly force-pushed the FIREFLY-1925-redis-value-messagepack branch from 21f682d to 8f7915c Compare January 12, 2026 18:13
Copy link
Contributor

@robyww robyww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. See my question about JsonAutoDetect

Comment on lines +19 to +23
@JsonAutoDetect(
fieldVisibility = JsonAutoDetect.Visibility.ANY,
getterVisibility = JsonAutoDetect.Visibility.NONE,
setterVisibility = JsonAutoDetect.Visibility.NONE
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty verbose. would

@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY)

since it all defaults to NONE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not sure whether it defaults to NONE, but when using auto-detect I explicitly want to exclude all getter and setter methods and rely only on the declared fields.

@loitly loitly merged commit 8a8e06c into dev Jan 14, 2026
@loitly loitly deleted the FIREFLY-1925-redis-value-messagepack branch January 14, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants