Skip to content

[ETW] Add configuration to export 64-bit integer as timestamp #3286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 22, 2025
Merged

[ETW] Add configuration to export 64-bit integer as timestamp #3286

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
71e3240
[ETW] Add configuration to export 64-bit integer as timestamp
ThomsonTan Feb 20, 2025
9b679ee
Fix test value
ThomsonTan Feb 21, 2025
7d6ecd6
Merge branch 'main' into etw_export_timestamp
lalitb Feb 21, 2025
70d0366
Format
ThomsonTan Feb 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions exporters/etw/include/opentelemetry/exporters/etw/etw_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@
#pragma once
#include <map>

#if defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)
#include <set>
#endif // defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

#include "opentelemetry/nostd/shared_ptr.h"
#include "opentelemetry/nostd/string_view.h"
#include "opentelemetry/nostd/unique_ptr.h"
Expand All @@ -23,10 +27,19 @@ namespace etw
/**
* @brief TelemetryProvider Options passed via SDK API.
*/

#if defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)
using TelemetryProviderOptions = std::map<
std::string,
nostd::variant<std::string, uint64_t, float, bool, std::map<std::string, std::string>, std::set<std::string>>>;

#else
using TelemetryProviderOptions = std::map<
std::string,
nostd::variant<std::string, uint64_t, float, bool, std::map<std::string, std::string>>>;

#endif // defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

/**
* @brief TelemetryProvider runtime configuration class. Internal representation
* of TelemetryProviderOptions used by various components of SDK.
Expand All @@ -45,6 +58,13 @@ typedef struct
bool enableTableNameMappings; // Map instrumentation scope name to table name with
// `tableNameMappings`
std::map<std::string, std::string> tableNameMappings;

#if defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

std::set<std::string> timestampAttributes; // Attributes to use as timestamp

#endif // defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

} TelemetryProviderConfiguration;

/**
Expand Down
33 changes: 33 additions & 0 deletions exporters/etw/include/opentelemetry/exporters/etw/etw_logger.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,33 @@ class Logger : public opentelemetry::logs::Logger
}
evt[ETW_FIELD_LOG_SEVERITY_NUM] = static_cast<uint32_t>(severity);
evt[ETW_FIELD_LOG_BODY] = std::string(body.data(), body.length());

#if defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

for (const auto &attr : cfg.timestampAttributes)
{
auto it = evt.find(attr);
if (it != evt.end())
{
auto value_index = it->second.index();
if (value_index != exporter_etw::PropertyType::kTypeInt64 &&
value_index != exporter_etw::PropertyType::kTypeUInt64)
{
continue;
}
int64_t filetime = value_index == exporter_etw::PropertyType::kTypeUInt64
? nostd::get<uint64_t>(it->second)
: nostd::get<int64_t>(it->second);
constexpr int64_t FILETIME_EPOCH_DIFF = 11644473600LL; // Seconds from 1601 to 1970
constexpr int64_t HUNDRED_NANOSECONDS_PER_SECOND = 10000000LL;
int64_t unix_time_seconds = (filetime / HUNDRED_NANOSECONDS_PER_SECOND) - FILETIME_EPOCH_DIFF;
int64_t unix_time_nanos = unix_time_seconds * 1'000'000'000 + (filetime % HUNDRED_NANOSECONDS_PER_SECOND) * 100;
Copy link
Member

@lalitb lalitb Feb 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check for the overflow, as unix_time_seconds can be large value in uint64_t, and we are multiplying with 1'000'000'000 , followed by addition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how should overflow be handled here, but it seems it is converted into time64_t which doesn't take the signing of int64_t into account (see here, probably passing overflowed value is fine here, which will just be treated as sort of invalid timestamp and returns some new or old timestamp.

Please feel free to correct me.

Copy link
Member

@lalitb lalitb Feb 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some test here - https://godbolt.org/z/dGoen89bh, and can see the overflow happening for the filetime > year 2280. The existing code is used for conversion of the current timestamp (span start time, log timestamp etc), which should be fine as it won't overflow. However, with timestamp coming as attribute can have any value past year 2280, and this can overflow.
This should be fine for now, but good to add some code comment so it can be revisited.

it->second = utils::formatUtcTimestampNsAsISO8601(unix_time_nanos);
}
}

#endif // defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

etwProvider().write(provHandle, evt, nullptr, nullptr, 0, encoding);
}

Expand Down Expand Up @@ -354,6 +381,12 @@ class LoggerProvider : public opentelemetry::logs::LoggerProvider
GetOption(options, "enableTableNameMappings", config_.enableTableNameMappings, false);
GetOption(options, "tableNameMappings", config_.tableNameMappings, {});

#if defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

GetOption(options, "timestampAttributes", config_.timestampAttributes, {});

#endif // defined(OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW)

// Determines what encoding to use for ETW events: TraceLogging Dynamic, MsgPack, XML, etc.
config_.encoding = GetEncoding(options);
}
Expand Down
49 changes: 49 additions & 0 deletions exporters/etw/test/etw_logger_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@

# include <gtest/gtest.h>
# include <map>
# include <set>
# include <string>

# define OPENTELEMETRY_ATTRIBUTE_TIMESTAMP_PREVIEW

# include "opentelemetry/exporters/etw/etw_logger_exporter.h"
# include "opentelemetry/sdk/trace/simple_processor.h"

Expand Down Expand Up @@ -146,4 +149,50 @@ TEST(ETWLogger, LoggerCheckWithTableNameMappings)
logger->Debug("This is a debug log body", opentelemetry::common::MakeAttributes(attribs)));
}

/**
* @brief Logger Test with structured attributes
*
* Example Event for below test:
* {
* "Timestamp": "2024-06-02T15:04:15.4227815-07:00",
* "ProviderName": "OpenTelemetry-ETW-TLD",
* "Id": 1,
* "Message": null,
* "ProcessId": 37696,
* "Level": "Always",
* "Keywords": "0x0000000000000000",
* "EventName": "table1",
* "ActivityID": null,
* "RelatedActivityID": null,
* "Payload": {
* "SpanId": "0000000000000000",
* "Timestamp": "2021-09-30T22:04:15.066411500Z",
* "TraceId": "00000000000000000000000000000000",
* "_name": "table1",
* "tiemstamp1": "2025-02-20T19:18:11.048166700Z",
* "attrib2": "value2",
* "body": "This is a debug log body",
* "severityNumber": 5,
* "severityText": "DEBUG"
* }
* }
*
*/

TEST(ETWLogger, LoggerCheckWithTimestampAttributes)
{
std::string providerName = kGlobalProviderName; // supply unique instrumentation name here
std::set<std::string> timestampAttributes = {{"timestamp1"}};
exporter::etw::TelemetryProviderOptions options = {{"timestampAttributes", timestampAttributes}};
exporter::etw::LoggerProvider lp{options};

auto logger = lp.GetLogger(providerName, "name1");

// Log attributes
Properties attribs = {{"timestamp1", 133845526910481667ULL}, {"attrib2", "value2"}};

EXPECT_NO_THROW(
logger->Debug("This is a debug log body", opentelemetry::common::MakeAttributes(attribs)));
}

#endif // _WIN32
Loading