Skip to content

Conversation

@antiguru
Copy link
Member

@antiguru antiguru commented Jan 17, 2025

Convert the progress and reachability logs to typed logs, which do not require casting at runtime.

The loggers are accessible using new names:

  • timely/progress -> timely/progress/ with the type name appended.
  • timely/reachability -> timely/reachability/ with the type name appended.

It's using std::any::type_name() to determine a unique string for the timestamp type.

Analyzing logging-send for allocations, this roughly cuts down total allocations in half.

target/release/examples/logging-send 1000 1000 -w4 produces roughly 850k allocations in current master, and 445k on this branch.

Convert the progress and reachability logs to typed logs, which do not
require casting at runtime.

Signed-off-by: Moritz Hoffmann <[email protected]>
Copy link
Member

@frankmcsherry frankmcsherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good to me; no notes.

Comment on lines +137 to +138
let mut messages = Vec::with_capacity(changes.len());
let mut internal = Vec::with_capacity(changes.len());
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we should be using recv_changes.len() here, because changes doesn't have any relation to what we're receiving. But, then we'd over-allocate :/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, could you explain that more? I agree that recv_changes.len() looks more appropriate, but what is the downside (seems like we fill these up with exactly that many elements, and then ship them in the TimelyProgressEvent)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, because we have to demux the two types of events, probably?

@frankmcsherry frankmcsherry merged commit 09994a8 into TimelyDataflow:master Jan 17, 2025
7 checks passed
@antiguru antiguru deleted the typed_logging branch January 17, 2025 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants