Skip to content

Disk buffering api changes #2084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

LikeTheSalad
Copy link
Contributor

These changes are meant to address the feedback received since the creation of this module, as well as prepare it to get promoted from alpha status to beta, as mentioned here.

The key takeaways for this new API are the following:

  • Abstracts the storage mechanism to allow for custom implementations to store signals using a different approach (unlike the current API that forces storing them into protobuf files as the only possible approach).
  • Provide more control over the writing process from disk buffering exporter instances (unlike the current API, which makes opinionated decisions around what to do when a write operation fails and what happens when an exporter's shutdown method is called).
  • Provide a cleaner reader API. (The current implementation consists of passing around export functions, with multiple overlapping enum states and making opinionated decisions for non-happy paths).
  • Provide a way to clear stored data on-demand. (The current API doesn't have a way to clear data other than reading and exporting it).

The plan is to create a default implementation for the new API that covers the existing API's behavior. Since migrating the existing behavior to the new API will require changing many files, I've decided to leave it for a follow-up PR, so that we can focus on the new API design on this one.

@LikeTheSalad LikeTheSalad requested a review from a team as a code owner August 8, 2025 08:58
@github-actions github-actions bot requested a review from zeitlinger August 8, 2025 08:58
@trask trask added this to the v1.49.0 milestone Aug 12, 2025
Copy link

@fractalwrench fractalwrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines +18 to +33
void onExportSuccess(SignalType type);

/**
* Called when an export to disk operation failed.
*
* @param type The type of signal associated to the exporter.
* @param error Optional - provides more information of why the operation failed.
*/
void onExportError(SignalType type, @Nullable Throwable error);

/**
* Called when the exporter is closed.
*
* @param type The type of signal associated to the exporter.
*/
void onShutdown(SignalType type);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge deal, but I often find it convenient to provide default empty methods for implementations that may not necessarily care about all of the interface methods. That way, implementers can choose to implement the ones that they are interested in, and it doesn't require many implementations to repeat empty bodies.

Copy link
Contributor Author

@LikeTheSalad LikeTheSalad Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I think I see what you mean, though considering that this interface is only for callback purposes, do you think that also applies here? And if so, how can we know what callbacks to set a default body to and which ones to enforce? Given that none of them are required to make the exporter work? - Usually, my concern with default impls, is that people might miss available options as the compiler won't complain if they're not implemented, unless users read the docs and take the time to scan through the lib's source code. It's not a big deal if we can provide defaults that work for everyone in case an implementation is used as part of the business logic, but for merely informative ones, such as this one, I'm not sure it applies.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO default method bodies should be avoided in interfaces that form public APIs. It makes discoverability harder, doesn't cleanly separate the interface from the implementation, and the behavior can become part of the public API's contract which can constrain changes in future.

Having said that, an empty body is less of a big deal to me than a body containing actual business logic would be

if (operation.isSuccessful()) {
callback.onExportSuccess(type);
return CompletableResultCode.ofSuccess();
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer removing the redundant else and unindenting the part below it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've added the changes.

private final Duration writeTimeout;
private final SignalType type;

public SignalStorageExporter(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have unit test coverage for this new class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just added unit tests for it.

import javax.annotation.Nonnull;

/** Default storage implementation where items are stored in multiple protobuf files. */
public final class FileSpanStorage implements SignalStorage.Span {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably there will be analogs for the other signal types? Why not include those in this PR as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably there will be analogs for the other signal types?

Yes, that's the idea.

Why not include those in this PR as well?

I just wanted to put the new API design to test with this PR, not to fully implement it yet; this is to avoid wasting time in case some design changes were requested during the review. If there are no issues with the design itself, I'm planning to create a follow-up PR with all the implementations and refactorings needed to use the new approach.

Copy link
Contributor

@breedx-splk breedx-splk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, just a few small comments. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants