Skip to content

Data Retention Regulator Change Log

Jump to bottom
Daniel Shearer edited this page Dec 4, 2025 · 4 revisions

This page has moved. See:

1.3.2 (2025-11-24)

1.3.1 (2025-11-01)

1.3.0 - 2025-10-03

  • [Doc] AsciiDoc changelog.

  • Upgrade to Aeron 1.49.0.

  • Upgrade to Agrona 2.3.0.

  • Upgrade to Gradle 9.1.0.

  • Upgrade to Checkstyle 11.1.0.

  • Upgrade to JUnit 6.0.0.

  • Upgrade to Mockito 5.20.0.

  • Upgrade to Shadow 9.2.2.

  • Upgrade to Versions 0.53.0.

1.2.6 - 2025-08-08

1.2.5 - 2025-07-21

1.2.4 - 2025-06-27

  • Upgrade to Aeron 1.48.4.

  • Upgrade to Agrona 2.2.4.

  • Upgrade to Shadow 8.3.7.

  • Upgrade to JUnit 5.13.2.

  • Upgrade to Checkstyle 10.26.0.

1.2.3 - 2025-06-20

1.2.2 - 2025-06-12

1.2.1 - 2025-06-06

1.2.0 - 2025-06-03

1.1.0 - 2025-05-09

  • [Breaking (behaviour)] Changed RetainEnoughDataForRecoveryPolicy and MatchingLogRecordingPolicy to fall back to Aeron’s system properties for clusterId, clusterDirName, and standbyDirName if not set programmatically or via explicit policy properties.

  • [Breaking (behaviour)] Changed the DataRetentionRegulator#main entrypoint to make property value precedence more sane. Now, system properties take precedence over property values defined in properties files, and property values defined in "later" files (w.r.t. the command line arguments) take precedence over property values defined in "earlier files.

  • [Breaking (API)] Changed the signature of DataRetentionRegulator#execute to return a boolean indicating whether the data retention regulation was successfully completed or not. It returns false, e.g., if backing up a segment file was not complete or if an archive operation failed. This allows the caller to retry the operation.

  • [Breaking (API, behaviour)] DataRetentionRegulator.Context has had some significant changes:

  • #archiveContext now defaults to null. It is initialised to use the local control channels of the archive using the specified archiveDirName (which are read from its mark file) in the DataRetentionRegulator.Context#conclude method.

  • #aeronDirectoryName has been removed, as it is now discovered from the archive’s mark file.

  • it no longer extends CommonContext.

  • Added support for automatic retries in the DataRetentionRegulator#main entrypoint (until successful execution) when the aeron.data.retention.execute.timeout system property is set to a positive value. The default is 0, which will behave as before (i.e., no retries). When set to a positive value, the regulator will retry the operation every aeron.data.retention.execute.retry.period (which defaults to 10 seconds) until the operation is successful or the timeout is reached. Both system properties accept time unit suffixes (ns, us, ms, and s).

  • Fixed a bug where MatchingLogRecordingPolicy did propagate an evaluation context to the inner policy; therefore, the inner policy might fail due to missing dependencies, e.g., the epochClock in DeleteSegmentsAfterPeriodPolicy.

  • Added a retentionPeriod overload to both DeleteSegmentsAfterPeriodPolicy.Context and DeleteRecordingsAfterPeriodPolicy.Context that accepts amount and unit simultaneously, to avoid the need to call two methods to set the retention period.

  • Improved javadoc for DeleteSegmentsAfterPeriodPolicy.Context

  • Made the retention behaviour of DeleteSegmentsAfterPeriodPolicy configurable via a mode. Opting into retentionMode(RETENTION_UNTIL_DELETION) makes it possible to compose the policy with others using RetainMaxOfPolicy in a way that will keep at least the retention period of data. Previously, and when using mode NO_RETENTION, the policy would only "cover" recordings when they had segments due for deletion, i.e., segments last modified before the retention period. When using the new RETENTION_UNTIL_DELETION mode, the policy will "cover" all recordings, i.e., the policy will ensure data is kept for the retention period by proposing a recording start position based on the earliest segment that was modified within the retention period.

  • Support providing a backing up strategy via the aeron.data.retention.backing.up.strategy system property, where the value should be the name of a class with a no*arg constructor that implements SegmentFileBackingUpStrategy.

  • Upgrade to Aeron 1.47.5.

  • Upgrade to Agrona 2.1.0.

  • Upgrade to Aeron Cluster Standby 1.47.5.

1.0.0 (2025-03-14)

  • [Breaking] groupId changed to io.aeron.premium.drr:

    <groupId>io.aeron.premium.drr</groupId>
<artifactId>aeron-data-retention-regulator</artifactId>

  • Upgrade to Aeron 1.47.4.

  • Upgrade to Aeron Cluster Standby 1.47.4.

0.11.0 (2025-01-31)

  • [Breaking] The aeron-data-retention-regulator-x.y.z.jar artifact is now a regular thin jar, the fat jar now lives in a new artifact aeron-data-retention-regulator-x.y.z-all.jar.

  • [Breaking] Upgrade to Agrona 2.0.1 - requires additional command line argument.

  • Upgrade to Aeron 1.47.2.

  • The jars now contain the README and the CHANGELOG: aeron-data-retention-regulator/README.md aeron-data-retention-regulator/CHANGELOG.md

0.10.0 (2024-11-01)

  • Fix a bug where RecordingMetadata would be incorrectly populated (introduced in 0.7.0):

    • maxRecordedPosition would be correct only for stopped recordings, -1 for active ones

    • startTimestamp would be the maxRecordedPosition

    • stopTimestamp would be the startTimestamp. Which would have the following effects:

      • RetainLengthPolicy would always retain all active recordings

      • DeleteSegmentsAfterPeriodPolicy would not delete any segments for active recordings

      • DeleteRecordingsAfterPeriodPolicy would delete after period calculated from recording start instead of stop, unless recording is active, then it would fail to delete

  • Upgrade to Aeron 1.46.7.

  • Upgrade to Agrona 1.23.1.

0.9.0 (2024-08-30)

  • [Breaking] Java 17 or newer is now required.

  • JavaDoc jars are now published.

  • Upgrade to Aeron 1.46.0.

  • Upgrade to Agrona 1.23.0.

0.8.0 (2024-08-30)

0.7.0 (2024-08-29)

  • Make testing policy trees easier in downstream projects without extensive mocking:

    • Allow creation of SegmentFileBoundary

    • Allow creation of RecordingMetadata

  • Make dry run output more precise by avoiding the calculation of new start positions before the existing start position in the built-in policies.

0.6.0 (2024-08-14)

  • Introduced a new composite policy:

    • RetainMinOfPolicy: combines sub-policies by retaining the minimum of its inner policies.

  • Fixed a bug where DeleteRecordingsAfterPeriodPolicy gave the wrong retention instruction for live recordings. Thankfully, the purge would always fail, as the recording was live. A test has been added to prevent regressions.

0.5.0 (2024-08-01)

  • Introduce a ``dry run'' mode, where the tool does not delete any data, but logs what it would have done. This can be useful for testing policies.

  • Change the default controlResponseStreamId we use from 20 to 20240712 to reduce the chances of collisions with other AeronArchive clients.

0.4.0 (2024-07-24)

  • Make it easier to configure the controlReponseStreamId on the AeronArchive.Context that the DRR uses.

  • Avoid attempts to detach partial (i.e., not complete) segment files in DeleteSegmentsAfterPeriodPolicy if they have ``expired''.

0.3.0 (2024-07-19)

  • [Breaking] Renamed DeleteAfterPeriodPolicy to the more-specific DeleteRecordingsAfterPeriodPolicy

  • Introduced new policies:

    • DeleteSegmentsAfterPeriodPolicy: detaches and deletes recording segment files that have not been modified for some period of time.

    • MatchingLogRecordingPolicy: applies an inner policy to the Consensus Module’s log recording.

0.2.0 (2024-07-19)

Initial release with the following built-in data retention policies:

  • Leaf policies:

    • RetainEnoughDataForRecoveryPolicy: keeps the last N snapshots and the log following the earliest of these snapshots available for cluster node recovery.

    • RetainLengthPolicy: keeps at least the last N bytes of data. It will detach and delete recording segments before the last N bytes.

    • DeleteAfterPeriodPolicy: deletes recordings that stopped some period of time ago.

  • Composite policies:

    • MatchingStreamsPolicy: applies an inner policy to recordings that match a channel URI and/or stream identifier.

    • RetainMaxOfPolicy: safely combines policies, e.g., to retain X snapshots worth of cluster data and Y bytes of data per recording, by retaining the maximum of its inner policies.

    • CascadePolicy: accepts a sequence of policies. It will try each policy in turn and apply the first policy that ``covers'' a recording, i.e., that has an opinion on what the extent of the recording should be.

Clone this wiki locally