Skip to content

Have Opentelemetry error handler to use tracing::error #852

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuandrew merged 8 commits into from
Dec 17, 2024
Merged

Have Opentelemetry error handler to use tracing::error #852

Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
efc31e6
Set global opentelemetry error handler to use tracing::error
yuandrew Dec 10, 2024
513c533
Add OTel HTTP support
yuandrew Dec 10, 2024
f390cfc
Revert "Add OTel HTTP support"
yuandrew Dec 10, 2024
d4eb94d
Merge branch 'master' into otel-tracing-error
yuandrew Dec 10, 2024
289145a
Merge branch 'master' into otel-tracing-error
yuandrew Dec 13, 2024
6d582af
attempting to write a new test to validate this new setting
yuandrew Dec 13, 2024
5386ced
Added test
yuandrew Dec 16, 2024
a55969a
Merge branch 'master' into otel-tracing-error
yuandrew Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion core/src/telemetry/otel.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use super::{
};
use crate::{abstractions::dbg_panic, telemetry::metrics::DEFAULT_S_BUCKETS};
use opentelemetry::{
self,
self, global,
metrics::{Meter, MeterProvider as MeterProviderT},
Key, KeyValue, Value,
};
Expand Down Expand Up @@ -120,6 +120,9 @@ pub(super) fn augment_meter_provider_with_defaults(
pub fn build_otlp_metric_exporter(
opts: OtelCollectorOptions,
) -> Result<CoreOtelMeter, anyhow::Error> {
global::set_error_handler(|err| {
Copy link
Member

@cretz cretz Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this was removed recently in 0.27 (ref open-telemetry/opentelemetry-rust#2260), so we either need to say in the dep that we are 0.26 (not sure if you can say < 0.27) or look at that migration guide and see if just upgrading to 0.27 is enough to get proper logs here.

Copy link
Contributor Author

@yuandrew yuandrew Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the change only changes the behavior on how you can override their default logging behavior, we would still need to override their behavior, just in a different way now. I'm not sure it's worth upgrading to 0.27 over this.

Copy link
Member

@cretz cretz Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my concern is whether other people want to upgrade to 0.27+. I am not sure what Cargo's auto-upgrade logic is for zero-ver. Obviously one day we will need to as well even if that day is not today (can't stay on 0.26 forever).

Copy link
Contributor Author

@yuandrew yuandrew Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who are the "other" people mentioned here? customers?

I am not sure what Cargo's auto-upgrade logic is for zero-ver.

I assume cargo wouldn't auto-upgrade from 0.26 to 0.27, since there are a number of breaking changes with that update.

Obviously one day we will need to as well even if that day is not today (can't stay on 0.26 forever).

Agreed, I'm just reluctant to upgrade today 😅 but now that I've written a test, that future upgrade should be easier

Copy link
Member

@cretz cretz Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who are the "other" people mentioned here? customers?

Nope just us internally

Agreed, I'm just reluctant to upgrade today

Works for me

tracing::error!("{}", err);
})?;
Comment on lines +124 to +126
Copy link
Member

@cretz cretz Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is a better more global place to set this instead of on each build_otlp_metric_exporter call? Unsure if we have global metric initialization code.

let mut exporter =
opentelemetry_otlp::TonicExporterBuilder::default().with_endpoint(opts.url.to_string());
if opts.url.scheme() == "https" || opts.url.scheme() == "grpcs" {
Expand Down
Loading