Add user agent customization

[meijeran]

Fixes #6496

Design discussion issue #

Changes

Please provide a brief description of the changes here.

This pull request enhances the configuration of the User-Agent HTTP header in the OtlpExporterOptions class, allowing users to prepend custom product identifiers to the default exporter information. It introduces a new property for customization, updates header construction logic, and adds comprehensive tests to ensure correct behavior and compliance with HTTP standards.

User-Agent customization improvements:

• Added a public UserAgentProductIdentifier property to OtlpExporterOptions that allows users to specify a custom product identifier to be prepended to the default User-Agent header. The header updates dynamically when this property changes.
• Modified the logic for building the User-Agent string to include the custom identifier, ensuring proper formatting and RFC 7231 compliance (space-separated tokens).
• Changed StandardHeaders from a static readonly array to a static property backed by a private field, so it reflects the current user agent identifier. [1] [2]

Testing enhancements:

• Added multiple unit tests in OtlpExporterOptionsTests.cs to verify the default and custom behaviors of the UserAgentProductIdentifier property, including handling of empty/whitespace values, correct header formatting, and compliance with HTTP standards.

Merge requirement checklist

• CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
• [ X] Unit tests added/updated
• Appropriate CHANGELOG.md files updated for non-trivial changes
• Changes in public API reviewed (if applicable)

[martincostello]

Looks OK to me now - the maintainers just need to be happy with how it presents as a new public API.

[cijothomas]

nit: make it clear that this will be prepended to the existing default string.

also, please do add a changelog entry.

[meijeran]

Updated the documentation