feat: generate JSON schemas for internal extensions

[SoumyaRaikwar]

Which problem is this PR solving?

part of #6186

Description of the changes

This PR implements automated JSON schema generation for Jaeger v2's internal extensions using the schemagen tool.

• Added a root-level .schemagen.yaml configuration to define Jaeger-specific type mappings (e.g., time.Duration to JSON Schema duration format).
• Added a generate-schemas target to the root Makefile to automate the process.
• Generated config.schema.yaml files for the 6 core internal extensions:
  • expvar
  • jaegermcp
  • jaegerquery
  • jaegerstorage
  • remotesampling
  • remotestorage

The generated schemas provide machine-readable definitions for the configuration surface of these extensions, facilitating better validation and documentation.

How was this change tested?

• Successfully executed make generate-schemas locally.
• Manually verified that the generated schemas correctly handle internal references and external OTel types.
• Ran make build-jaeger and make lint to ensure no regressions.

Checklist

• I have read https://github.com/jaegertracing/jaeger/blob/main/CONTRIBUTING_GUIDELINES.md
• I have signed all commits
• I have added unit tests for the new functionality (N/A - Schema generation)
• I have run lint and test steps successfully: make lint test

AI Usage in this PR

• Light: AI provided minor assistance (formatting, simple suggestions)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.69%. Comparing base (aa4193d) to head (55901aa).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8089      +/-   ##
==========================================
+ Coverage   95.66%   95.69%   +0.02%     
==========================================
  Files         317      317              
  Lines       16668    16668              
==========================================
+ Hits        15946    15950       +4     
+ Misses        570      567       -3     
+ Partials      152      151       -1     

Flag	Coverage Δ
badger_v1	9.00% <ø> (ø)
badger_v2	0.93% <ø> (ø)
cassandra-4.x-v1-manual	13.23% <ø> (ø)
cassandra-4.x-v2-auto	0.92% <ø> (ø)
cassandra-4.x-v2-manual	0.92% <ø> (ø)
cassandra-5.x-v1-manual	13.23% <ø> (ø)
cassandra-5.x-v2-auto	0.92% <ø> (ø)
cassandra-5.x-v2-manual	0.92% <ø> (ø)
clickhouse	1.01% <ø> (ø)
elasticsearch-6.x-v1	16.61% <ø> (ø)
elasticsearch-7.x-v1	16.64% <ø> (ø)
elasticsearch-8.x-v1	16.79% <ø> (ø)
elasticsearch-8.x-v2	0.93% <ø> (ø)
elasticsearch-9.x-v2	0.93% <ø> (ø)
grpc_v1	7.72% <ø> (ø)
grpc_v2	0.93% <ø> (ø)
kafka-3.x-v2	0.93% <ø> (ø)
memory_v2	0.93% <ø> (ø)
opensearch-1.x-v1	16.68% <ø> (ø)
opensearch-2.x-v1	16.68% <ø> (ø)
opensearch-2.x-v2	0.93% <ø> (ø)
opensearch-3.x-v2	0.93% <ø> (ø)
query	0.93% <ø> (ø)
tailsampling-processor	0.53% <ø> (ø)
unittests	94.37% <ø> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yurishkuro]

does this align with open-telemetry/opentelemetry-collector#14548 ?

[yurishkuro]

could you please create an ADR that outlines the overall plan? For example, how would this external ref be used?

[SoumyaRaikwar]

Done! added in the latest commit.

[SoumyaRaikwar]

@yurishkuro

i have added the ADR And proposing that as per Alignment with OTel #14548 (mdatagen): We are using schemagen instead of mdatagen because schemagen automatically reflects our complex Go structs, whereas mdatagen currently requires manually redefining schemas in metadata.yaml. We will evaluate migrating to mdatagen in the future if it natively supports Go reflection.,
How external references ($ref) will be used: We are leaving external $ref pointers intact as logical URIs ( pointing to OTel modules). In a future phase, we will introduce a Schema Bundler script that will download and resolve these URIs at build-time to create a fully self-contained JSON schema for documentation.

[yurishkuro]

@SoumyaRaikwar it is not clear to me from the ADR what is the goal and the end state. As the main issue states, the goal is to be able to generate documentation that accurately reflects the config schema expected by the Jaeger binary. Go struct -> YAML may be an implementation detail but it does not achieve this stated goal. And while it is easy to imagine a second step of using the YAML schema to generate the documentation, it does not explain how foreign references to OTEL types would help with that, nor how to make the schemas functionally rich (so that they can be used for config authoring and validation) and not just limited to type/description.

The typical schema-first solution would be to use schemas as the source of truth and use them to generate Go structs and validation code, as well as documentation.