OTel Blueprints project proposal

[danielgblanco]

This project aims to deliver a set of architecture blueprints, with the goal of facilitating and guiding adoption of best practices when deploying OpenTelemetry on a defined set of common environments. We'd like these blueprints to be backed by evidence in the form of reference architectures shared by end users.

The end-goal is to provide holistic, incremental, high-level guidance that any adopter can apply across their environments, resulting in mature architectures ready for production use, at scale.

Note: this PR was created as a draft to gather industry feedback on the initial approach.

[jaronoff97]

@danielgblanco i'd love to help out 🫡

[luke6Lh43]

I'd love to help here, @danielgblanco! Count me in!

[luke6Lh43]

Hi @danielgblanco

As we discussed at KubeCon and during the last SIG call, I believe it would be beneficial to organize OTel blueprints by vertical/industry. Many organizations within the same vertical share similar needs and challenges, so a more generalized framework for each industry makes sense. We could create verticals such as Manufacturing, Healthcare, Financial Services, Media/Tech, and Retail. For each, we could identify a few typical use cases and challenges, and then develop a reference architecture accordingly. If we have customers willing to provide a statement or testimonial, we could include those as well.

Please let me know your thoughts.

Also, do I understand it correctly that blueprints will be published on https://opentelemetry.io/docs/ ?

[tsloughter]

I'm surprised to hear the Otel blueprint would differ between types of industry. Is it due to organizational structures or types of systems they have to instrument or what?

[danielgblanco]

IMO I think we should keep in mind the difference (at least in my mind) between blueprints and reference architectures. I think there's value in tagging/categorising end-user reference architectures with the vertical/industry they're part of (as done in https://architecture.cncf.io/architectures). People in industry X will want to know how other companies in industry X are approaching OTel adoption, and there will be some common regulatory frameworks or challenges to solve. This is if by "reference architectures" we mean a single document at a point in time (i.e. end-user X approached it this way in Jan 2025).

However, I think OTel blueprints should be categorised depending on scenarios/environments/challenges, not verticals/industries, and then link to specific end-user ref architectures that exemplify recommendations.

After speaking to some end-users at KubeCon (and outside of KubeCon) I think that OTel blueprints should be aimed at giving guidance on specific environments/use cases. For instance, some that would come to mind:

• Centralised OpenTelemetry platform in Kubernetes environments (i.e. one team responsible for E2E journey)
  • Challenges to solve: providing consolidated SDK config, integration with CI, Kubernetes instrumentation, efficient Collector deployment patterns, trace sampling at scale, governance, etc
• Decentralised OpenTelemetry configuration (i.e. each team owning their own area)
  • Challenges to solve: providing consolidated Collector pipeline config and distributing that config, separation between central OTel gateway vs individual Collector deployments, traffic routing, etc.
• Instrumenting infrastructure and processes on VMs outside (not in Kubernetes)
  • Challenges to solve: Autodiscovery and collector config, receiver configuration, OpAMP, etc
• Client-side instrumentation integrated with publicly routed Collector gateway
• Serverless environments within a centralised observability platform
• IoT instrumentation
• etc

This is a non-exhaustive list and I think we should focus on a 3 blueprints to start, building a framework for future blueprints written by others in the End-User SIG (end-users or not). I was intending to discuss which three to cover after the project starts, but if folks think it's better we can do this as part of the project proposal.

[luke6Lh43]

Thank you, @danielgblanco, for clarifying what the OTel Blueprints project is. I apologize for any confusion I may have caused regarding verticals. I agree that it makes sense to focus on specific environments or use cases and then align appropriate reference architectures to them.

I'm happy to begin work on Instrumenting infrastructure and processes on VMs outside of Kubernetes.

Do we have a template or format that I should follow for this?

[danielgblanco]

Do we have a template or format that I should follow for this?

Nope, I think that's one thing I wanted to discuss first. However, this project has not been approved yet by the GC. @open-telemetry/governance-committee do you think we need more staffing listed down to start this project?

After this project is approved, my thinking is:

1. Create a new GitHub Project and starting putting in some draft issues to structure the work, not needing to go into too much detail.
2. Agree on a format for reference architectures (influenced by existing ones) and a format for blueprints.
3. Split tasks so we can all work in parallel and start to put down words into docs to be reviewed by the community. I think the easiest collaboration form will be Google Docs to start, and then move those into GitHub under the End-User part of our website (templates potentially remaining in the sig-end-user repo, but that can be discussed).

[alainpham]

Hi, i'm happy to contribute to this. I have been battle testing OTel for the past 2 years in engagements where I'm on boarding prospects and customers that go from 0 to full OTel . For most people out there it is still pretty much the jungle to get started, so an initiative like this really beneficial to drive adoption and gain new users.

+1 on the fact that it should be generic regardless of industry vertical, though it is always interesting to document the context when possible about real use cases in their vertical. From my experience the challenges are similar like

• what are the best practices in instrumentation to get good correlation between different assets (front end, backend, infra ..)
• best practices to deal with certain types of workloads (request response, pub sub, request response)
• how to deal with scale and get to a good cost/visibility ratio
• automate instrumentation

[ChaosKyle]

I would love to contribute what our oa team is building on this topic specifically to best practices/well architected frameworks. Count me in

[alainpham]

Happy to be able to contribute.

[trask]

@danielgblanco can you rebase? Looks like there are some new spell check failures when I tried to merge it.

[danielgblanco]

Thanks @trask. I just rebased and added names to cSpell config.