Skip to content

Conversation

@andborja
Copy link
Contributor

@andborja andborja commented Dec 19, 2025

Add internal logs sdk exporter sending telemetry to an internal logs receiver.

Opening as draft initially to discuss the general model and decide on the proper objects to send through the channel.

This PR presents the bridge to receive the internal telemetry. It also demonstrate how to implement the receiver to be included in a pipeline.

@github-actions github-actions bot added the rust Pull requests that update Rust code label Dec 19, 2025
@codecov
Copy link

codecov bot commented Dec 19, 2025

Codecov Report

❌ Patch coverage is 75.30562% with 101 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.04%. Comparing base (2584e4b) to head (dc8c801).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1663      +/-   ##
==========================================
- Coverage   84.06%   84.04%   -0.03%     
==========================================
  Files         469      471       +2     
  Lines      137651   138042     +391     
==========================================
+ Hits       115715   116014     +299     
- Misses      21402    21494      +92     
  Partials      534      534              
Components Coverage Δ
otap-dataflow 85.28% <75.30%> (-0.04%) ⬇️
query_abstraction 80.61% <ø> (ø)
query_engine 90.39% <ø> (ø)
syslog_cef_receivers ∅ <ø> (∅)
otel-arrow-go 53.50% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andborja andborja changed the title feat: Add internal telemetry receiver feat: Add internal logs receiver Dec 20, 2025
@jmacd
Copy link
Contributor

jmacd commented Jan 5, 2026

Thank you @andborja.

Comparing and contrasting with the draft I'm working on, #1672, your approach configures the OTel SDK with its various processor and exporter options from the configuration. I agree that we want an option to configure the OTel SDK in all of the ways implied by the declarative configuration model, and at the same time I'm worried about relying on the SDK's data representation in this code base.

In this code base, I think it makes more sense to focus on the OTLP bytes approach outlined in #1672 which is to optimize for the self-hosted internal telemetry code path at the expense of making the use
OpenTelemetry SDK processors/exporters somewhat more costly. The argument is that OTAP Dataflow has a batch processor and OTLP exporter for OTLP bytes that bypasses the use of OTLP protobuf objects. The OTAP Dataflow code path will be less expensive than the OTel SDK code path because it avoids OTLP protobuf objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants