V2 Log Encoding

#V2 HdrHistogram Log format The log format encodes into a single file, multiple histograms with optional shared meta data.

File format

([MetaData]\n)*
[ColumnHeaders]\n
([Histogram]\n)*

MetaData format

#[StartTime|BaseTime|Comment]

Meta data rows include comments, the StartTime and the BaseTime for the set of Histograms being logged

StartTime value format

[StartTime:\d*.\d{1,3}															

Start time is represented as seconds since epoch with up to 3 decimal places. Line starts with the leading text '#[StartTime:'

BaseTime value format

[BaseTime:\d*.\d

Base time is represented as seconds since epoch with up to 3 decimal places. Line starts with the leading text '#[BaseTime:'

Comment value format

.*

A comment is any line that leads with '#' that is not matched by the BaseTime or StartTime formats. Comments are ignored when parsed.

ColumnHeaders format

StartTimestamp.*

Human readable column headers. Ignored when parsed.

Histogram format

[LogTimeStamp],[IntervalLengthSeconds],[MaxTime],[CompressedPayload]

LogTimeStamp value format

\d*.\d*

The LogTimeStamp is the start time of the Histogram in seconds since the BaseTime measured as a double. If the BaseTime is not provided, then this is an absolute time in seconds.

IntervalLengthSeconds value format

\d*.\d*

The interval length is the total time in seconds that the Histogram has measured.

MaxTime value format

\d*.\d*

The MaxTime is the finsih time in seconds since the BaseTime measured as a double. This is ignored as it can be inferred from the LogTimeStamp and the IntervalLengthSeconds

CompressedPayload format

BASE64(
	DEFLATE(
		[cookie][lengthOfCompressedContents][CompressedHeader]
	)
)

The compressed payload is a Base64Encoded, ZLIB compressed encoding (RFC-1950) of an HdrHistogram.

CompressedHeader format

DEFALTE(
	[Cookie][payloadLengthInBytes][normalizingIndexOffset][numberOfSignificantValueDigits][lowestTrackableUnitValue][highestTrackableValue][integerToDoubleValueConversionRatio][CompressedContents]		
)

The Zlib compressed payload of an HdrHistogram. The uncompressed payload is stored in binary in the following format

01234567890123456789012345678901
[A ][B ][C ][D ][E     ][F     ][G....>

• A. Cookie value. 32bit integer (4bytes) that stores a code for the type of Histogram.
• B. Payload length as a 32bit integer (4bytes).
• C. NomalizingIndex offset as an 32bit integer (4bytes). Currently not implemented in HdrHistogram.NET
• D. Lowest Trackable Unit value as a 64bit integer (8bytes).
• E. Highest Trackable Unit value as a 64bit integer (8bytes).
• F. Integer to double conversion ratio as a double floating point number (8bytes)
• G. Compressed contents

CompressedContents

ZigZaggedEncoding(
	[count]*
)

The ZigZag encoded (LEB128 format) compression of the counts for an HdrHistogram. Consecutive 0 counts will be compressed into a single negative number indicating the span of the zero count sequence. Positive counts will be represented as the actual number. Counts are recorded as 64bit integers.

See also

• ZLIB RCF-1950 compression - [https://en.wikipedia.org/wiki/Zlib], [https://www.ietf.org/rfc/rfc1950.txt]
• Base64 Encoding - [https://en.wikipedia.org/wiki/Base64], [https://www.base64encode.org/]
• ZigZag LEB128 compression - [https://en.wikipedia.org/wiki/LEB128]