Add Configuration SDK

[ysolomchenko]

What

Add Configuration SDK as a new project.

Declarative configuration provides a mechanism for configuring OpenTelemetry which is more expressive and full-featured than the environment variable based scheme, and language agnostic in a way not possible with programmatic configuration. Notably, declarative configuration defines tooling allowing users to load OpenTelemetry components according to a file-based representation of a standardized configuration data model.

Why

Many users want more advanced and flexible configuration options for auto-instrumentation than what environment variables can provide. At the same time, they don’t want or aren’t able to modify application code. This is often the case in production environments, legacy systems, or where the source code isn't accessible.

This work is essential to enable initiatives such as OpAmp, NoCode, and the Ansible Runbook for OTel .NET Auto-Instrumentation (or other deployment solutions).

Even without technologies like OpAmp, this feature alone is extremely valuable.

Proposal

• We want to gather support from approvers and maintainers who can assist with reviewing PRs, providing feedback, and possibly contributing to the implementation.
• If experienced contributors such as @alanwest can join the effort, it would greatly increase our capacity to deliver on time.
• Without enough review support, there is a risk that PRs related to this feature will become stale and delay progress.

[pellared]

It is worth mentioning that currently there is an ongoing initiative to stabilize some parts of Declarative Configuration:

• Mark declarative config as stable opentelemetry-specification#4568

Thanks to working on it now we can:

• have impact on features before they are stabilized (e.g. we can report issues that may be problematic from our perspective)
• collaborate with other contributors working on the same features in different languages
• help with the stabilization effort
• discuss any doubts during Configuration SIG meetings

CC @open-telemetry/configuration-maintainers

Personally, I am not an approve in this repository, but I can try helping as a specification sponsor.
I am also a code owner of OTel Go Configuration SDK.

[alanwest]

Some notes from our SIG discussion today

• Old prototype from CodeBlanch
• Today it probably makes sense to build this on top of the OpenTelemetryBuilder within the OpenTelemetry.Api.ProviderBuilderExtensions package.
• Interoperability with IConfiguration would be nice
• File based configuration probably belongs in its own package.
• Consider dependencies. Would the new package depend on all SDK packages in order to configure them? Or can we design a plugin model where file-based configuration does not require any dependencies and discovers things at runtime? Discovering things at runtime could be problematic in AOT environments - maybe this is ok.
• The file-based configuration solution probably belongs in the opentelemetry-dotnet repository because it is a feature driven by the specification.

[thompson-tomo]

It would be great to see the solution built around IConfiguration that way we can handle the value subsitution easily etc. What I think would need the most discussion is how to get the config into the libraries/module.