Skip to content

Conversation

birschick-bq
Copy link
Contributor

@birschick-bq birschick-bq commented Jun 10, 2025

Adds a new project and tests to support adding OpenTelemetry exporters (TracerProvider).

  • Adds an ADBC FileExporter implementation.
  • Adds a ExportersBuilder that
    • Builds a list of supported exporters
    • Activates and returns the appropriate exporter based on the passed option or environment variable setting.

Note to reviewer:
ExportersBuilder is a convenience class - let me know if you think it is useful.

@birschick-bq birschick-bq changed the title feat(csharp/src/Telemetry): Add support for adding and configuring OTel exporters feat(csharp/src): Add support for adding and configuring OTel exporters Jun 10, 2025
@birschick-bq birschick-bq marked this pull request as ready for review June 10, 2025 16:07
@github-actions github-actions bot modified the milestone: ADBC Libraries 19 Jun 10, 2025
@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher - Just a reminder to see if you can review this separate PR to handle exporting trace activity.

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've left some initial comments but will return to reviewing this later.

private static IOrderedEnumerable<FileInfo> GetTracingFiles(DirectoryInfo tracingDirectory, string searchPattern)
{
return tracingDirectory
.EnumerateFiles(searchPattern, SearchOption.TopDirectoryOnly)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there no way to enumerate the files in an async fashion?

{
/// <summary>
/// Provides access to writing trace files, limiting the
/// individual files size and ensuring unique file names.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would happen if two instances of this were pointed at the same directory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CurtHagenlocher
I've added a test (and reworked the retry parameters) to support reasonably heavy concurrency. However, I don't think that would be normal, because this class is called by the FileExporter, that ensures there is only once instance for the folder/basefile name combination. So contention would not be likely.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is this part of the contract, then, that two processes which each instantiate this will have to point to different directories? If so, consider adding this detail to readme.md.

@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher - Ready for a new review

@birschick-bq
Copy link
Contributor Author

@jduo Let me know if you have any comments or suggestions. Thanks.

@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher Checking to see if you have some time to review this PR. Thanks.

@davidhcoe
Copy link
Contributor

@CurtHagenlocher - anything else needed on this? It would be nice to have support for outputting the BigQuery logs.

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! What do you think about making this be a standalone executable? It could still be referenced as an assembly and loaded into another process, but that would also be let it used directly.

{
/// <summary>
/// Provides access to writing trace files, limiting the
/// individual files size and ensuring unique file names.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is this part of the contract, then, that two processes which each instantiate this will have to point to different directories? If so, consider adding this detail to readme.md.

@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher

... What do you think about making this be a standalone executable?

I'm afraid I don't understand how a standalone executable would work. Would it be a sort of CLI tool to load and arbitrary driver and run queries?

Could you clarify this question? Thanks.

@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher

"Provides access to writing trace files, limiting the individual files size and ensuring unique file names."
So is this part of the contract, then, that two processes which each instantiate this will have to point to different directories? If so, consider adding this detail to readme.md.

I've update the file naming to include the process ID. This should allow multiple processes to create files independently of each other without having to compete for the same trace file in the same folder.

Let me know if you still have concerns about the concurrency or efficiency of the file generation/re-use design. Thanks.

@CurtHagenlocher
Copy link
Contributor

I'm afraid I don't understand how a standalone executable would work. Would it be a sort of CLI tool to load and arbitrary driver and run queries?

I'm assuming that the collector doesn't need to run in the same process as the producer. If that's not true, then this suggestion is pointless. But if it is true, then a collector could be started manually while something was happening and just record traces during that time before being stopped manually again.

I may be improperly projecting an internal EventTracer app I wrote onto a circumstance that isn't really equivalent.

@CurtHagenlocher CurtHagenlocher merged commit 13dac00 into apache:main Aug 15, 2025
7 checks passed
@birschick-bq birschick-bq deleted the dev/birschick-bq/csharp-otel-exporters branch August 15, 2025 21:36
@birschick-bq
Copy link
Contributor Author

I'm afraid I don't understand how a standalone executable would work. Would it be a sort of CLI tool to load and arbitrary driver and run queries?

I'm assuming that the collector doesn't need to run in the same process as the producer. If that's not true, then this suggestion is pointless. But if it is true, then a collector could be started manually while something was happening and just record traces during that time before being stopped manually again.

I may be improperly projecting an internal EventTracer app I wrote onto a circumstance that isn't really equivalent.

To the best of my knowledge, The ActivityListener needs to be in-process. The ActivityListener does not have an RPC to send to an external process, by itself. Enabling the OTLP exporter becomes the in-process collector that is then responsible for forwarding/sending to an OTLP endpoint like the OTel Collector.

@birschick-bq
Copy link
Contributor Author

@CurtHagenlocher P.S.: Thanks for taking the time to review this PR. Much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants