library-based alternative to opentelemetry-instrument for easy integration in Jupyter notebooks

[codefromthecrypt]

What problem do you want to solve?

Right now, we have a zero code solution: opentelemetry-instrument which is great as it removes the cruft of setting up various components, especially if your instrumentation uses multiple signals like logs, metrics and tracing.

However, as this wraps python itself, it presents integration problems which show up in certain scenarios like debuggers and Jupyter notebooks.

Describe the solution you'd like

I would like to see such a "quick start" that works the same as the CLI, but doesn't require bootstrapping python.

from opentelemetry.sdk import AutoInstrument
AutoInstrument.init()

Describe alternatives you've considered

I see some code on github doing this:

from opentelemetry.instrumentation.auto_instrumentation.sitecustomize import initialize
initialize()

which is incorrect as it redundantly initializing, and should be like this instead.

import opentelemetry.instrumentation.auto_instrumentation.sitecustomize

However, for this to work you must set PYTHONPATH (e.g. PYTHONPATH=$PWD dotenv run -- python chat.py), or you get an error until PR 2886 is resolved.

As the import name is neither intuitive, nor documented and has a pending PR to make it usable by default, it isn't yet an ideal solution. If it were (possibly renamed or at least) documented, and the PYTHONPATH glitch cleared, it could close this issue.

Additional Context

3rd party otel SDKs such as openllmetry and langtrace have a library alternative where you include a single initialization comment in your main.py, and that reads ENV variables such as opentelemetry-instrument does.

langtrace:

from langtrace_python_sdk import langtrace
langtrace.init(api_key='<your_api_key>') # Get your API key at langtrace.ai

openllmetry:

from traceloop.sdk import Traceloop
Traceloop.init()

This solves the problem, and I can verify that with the correct settings it works for any OTLP endpoint. However, as you notice these are branded and vendor specific. Also, there are configuration differences as well instrumentation choices.

Would you like to implement a fix?

None

[codefromthecrypt]

added to "alternatives you've considered" opentelemetry.instrumentation.auto_instrumentation.sitecustomize which @xrmx mentioned to me and seems several else are using.

We could improve this and document it in the zero-code docs, although technically it is 2 lines of code. Before we do that, it probably should be scrubbed a bit to avoid so many console messages.

[bettiah]

To avoid the warnings, a simple
import opentelemetry.instrumentation.auto_instrumentation.sitecustomize
without a subsequent call to initialize() works too

It does seem to require source root in PYTHONPATH (PYTHONPATH=pwd or equivalent)

[aabmass]

This sounds awesome, I'd love to see this.

For JS, there is a NodeSDK that lets you bundle everything into one call too https://opentelemetry.io/docs/languages/js/getting-started/nodejs/#setup. That might be a little lower level than this proposal though.

[codefromthecrypt]

re-jigged the description as yeah if you don't follow other examples in github and do these two things, you can get the intended behavior (at least in a basic example)

• import opentelemetry.instrumentation.auto_instrumentation.sitecustomize
• ensure PYTHONPATH is explicitly set until PR 2886 is resolved

[codefromthecrypt]

FYI I'm not sure everyone here noticed #3273 solves this including the PYTHONPATH concern

[codefromthecrypt]

can't wait for the next release to try this! thanks!