feat: add gem for auto-instrumentation in otel-operator #1384

xuan-cao-swi · 2025-01-30T18:55:02Z

Description

This PR aims to contribute a script/gem that can load the necessary OpenTelemetry-related gems and initialize the SDK for the opentelemetry-operator. The script/gem also includes a resource detector as part of its features.

The main idea was inspired by new_relic, which prepends the bundler.require function and places the loading script at the end of the loading procedure. For apps or scripts that don't use Bundler or a Gemfile, more details on variations of app loading can be found in the README. Currently, the goal is to ensure that Rails apps can work, with future improvements in mind.

The reason behind creating this gem is to facilitate easy installation and dependency enforcement. Although for the OpenTelemetry-Operator Node.js, it puts the loading script directly in the src folder (and then copy over to binding volume), it still relies on the auto-instrumentation-node package to install all required packages (e.g., API/SDK and instrumentation packages).

The main purpose of the gem is to serve the OpenTelemetry-Operator, which requires more configuration changes (see the configuration section in the README). Users don't need to modify their Ruby app codebase while using the instrumentation from OpenTelemetry-Ruby, but anyone can still use it in any environment.

The gem name, file structure, and implementation need more suggestions to make them more reasonable. I have tested it with rails and sinatra (through rackup), and I currently still in the process of testing against the opentelemetry operator cluster locally. Once the operator testing done, I will mark the pr as ready.

xuan-cao-swi · 2025-02-26T19:14:43Z

We don't need to push to make the gem ready/release, but the script file need to get some review first.

github-actions · 2025-03-29T02:02:41Z

👋 This pull request has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this pull request will be closed eventually by the stale bot

github-actions · 2025-05-03T02:05:46Z

👋 This pull request has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this pull request will be closed eventually by the stale bot

.github/workflows/ci-contrib.yml

instrumentation/base/lib/opentelemetry/instrumentation/base.rb

opentelemetry-auto-instrumentation/README.md

kaylareopelle · 2025-07-16T23:13:56Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+  end
+
+  # this is used for case when user particularly want to enable single instrumentation
+  OTEL_INSTRUMENTATION_MAP = {


Do we need to update this list every time we add a new gem? If so, is this something we could add to the instrumentation generator?

This option is for convenience of only required certain libraries for instrumentation (idea is borrowed from nodejs). We don't necessarily need to have it if it's too much.

Other option is having something like

OTEL_INSTRUMENTATION_MAP = {} OpenTelemetry::Instrumentation.constants.each do |const| OTEL_INSTRUMENTATION_MAP[const.to_s] = "OpenTelemetry::Instrumentation::#{const}" end OTEL_INSTRUMENTATION_MAP['NET_HTTP'] = "OpenTelemetry::Instrumentation::Net::HTTP" OTEL_INSTRUMENTATION_MAP.delete(:Base) OTEL_INSTRUMENTATION_MAP.delete(:All) output: {"HTTP"=>"OpenTelemetry::Instrumentation::HTTP", "DelayedJob"=>"OpenTelemetry::Instrumentation::DelayedJob", "Ethon"=>"OpenTelemetry::Instrumentation::Ethon", ...

I am not sure if user prefer having the capitcal naming or all downcase (e.g. DelayedJob vs delayed_job)
e.g.

export OTEL_RUBY_ENABLED_INSTRUMENTATIONS=delayed_job,action_mailer # or export OTEL_RUBY_ENABLED_INSTRUMENTATIONS=DelayedJob,ActionMailer

I do really like this and wished that the registry would support it. We only need a fully qualified name if the gem does not match the existing module pattern.

opentelemetry-auto-instrumentation/README.md

Co-authored-by: Kayla Reopelle <[email protected]>

…lemetry-ruby-contrib into auto-instrumentation

arielvalentin

I have a few observations and thoughts here for you. I do not want to block progress on this PR due to limited availability so I am not going to "request changes" but I would like you to strongly consider making changes to isolate the methods defined in the OTelBundlerPatch.

Nit: The upstream docs uses mixed language but the headings at least say "Zero-Code instrumentation" as opposed to auto-instrumentation. How would you feel about naming this gem zero-code as opposed to auto?

I would also like to ask @simi their opinion here to confirm any bundler changes or see if there is another means of hooking this into Bundler.

arielvalentin · 2025-06-04T17:37:15Z

instrumentation/base/lib/opentelemetry/instrumentation/base.rb

@@ -214,6 +214,7 @@ def initialize(name, version, install_blk, present_blk,
      #
      # @param [Hash] config The config for this instrumentation
      def install(config = {})
+        puts "name: #{@name}"


Do you intend for this change to be present?

arielvalentin · 2025-07-27T13:45:26Z

.github/workflows/ci-contrib.yml

+          bundle exec rake test
+          rm -f Gemfile.lock
+        working-directory: './opentelemetry-auto-instrumentation'
+      - name: "Test Ruby 3.1"


Ruby 3.1 was EoL 2025-03-31? Want to still support it?

arielvalentin · 2025-07-27T13:47:03Z

opentelemetry-auto-instrumentation/example/app.rb

+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+


Nit: Consider using bundler inline like the other ones.

If use autoinstrumentation, then there is no need to do Bundler.require here since the autoinstrumentation script will do it.

I think though that what we want is to see what "real world" applications would do. They require bundler and we are hooking into that flow. So event though we are requiring bundler as part of the Ruby run, wouldn't we want our examples to reflect how people would use it?

opentelemetry-auto-instrumentation/example/app.rb

arielvalentin · 2025-07-27T13:49:47Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+      end
+      OpenTelemetry.logger.info { 'Auto-instrumentation initialized' }
+    rescue StandardError => e
+      puts "Auto-instrumentation failed to initialize. Error: #{e.message}"


can you send this to $stderr since it is a diagnostic message?

Suggested change

puts "Auto-instrumentation failed to initialize. Error: #{e.message}"

$stderr.puts "Auto-instrumentation failed to initialize. Error: #{e.message}"

The rubocop seems prefer warn; but I can disable that warning and use $stderr.puts.

arielvalentin · 2025-07-27T14:03:12Z

opentelemetry-auto-instrumentation/test/opentelemetry-auto-instrumentation_test.rb

+  end
+
+  after do
+    OTelBundlerPatch.send(:remove_const, :OTEL_INSTRUMENTATION_MAP)


A side effect of localizing the methods. will mean only having to remove a single method from Bundler patch.

Could you elaborate on this (I updated the test case based on nested module update)?

arielvalentin · 2025-07-27T14:04:10Z

opentelemetry-auto-instrumentation/opentelemetry-auto-instrumentation.gemspec

+               ['LICENSE']
+
+  spec.require_paths = ['lib']
+  spec.required_ruby_version = '>= 3.1'


simiilar question mentioned above regarding support for Ruby 3.1

arielvalentin · 2025-07-27T14:05:12Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+# /otel-auto-instrumentation-ruby is set in opentelemetry-operator ruby.go
+operator_gem_path = ENV['OTEL_RUBY_OPERATOR'].nil? || ENV['OTEL_RUBY_OPERATOR'] == 'true' ? '/otel-auto-instrumentation-ruby' : nil
+additional_gem_path = operator_gem_path || ENV['OTEL_RUBY_ADDITIONAL_GEM_PATH'] || Gem.dir
+puts "Loading the additional gem path from #{additional_gem_path}" if ENV['OTEL_RUBY_AUTO_INSTRUMENTATION_DEBUG'] == 'true'


Similar comment regarding diagnostic messages. Consider using $stderr for these so they can be siphoned by users that rely on structured logs coming from $stdout

Updated to use $stdout.puts

arielvalentin · 2025-07-27T14:06:57Z

opentelemetry-auto-instrumentation/Gemfile

+
+source 'https://rubygems.org'
+
+# avoid use gemspec to avoid circular load


I am confused about this. Aren't these already declared in the gemspec?

Simplified the Gemfile

arielvalentin · 2025-07-27T14:08:06Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+  end
+
+  # this is used for case when user particularly want to enable single instrumentation
+  OTEL_INSTRUMENTATION_MAP = {


I do really like this and wished that the registry would support it. We only need a fully qualified name if the gem does not match the existing module pattern.

arielvalentin · 2025-07-27T14:18:41Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+# OTelBundlerPatch
+module OTelBundlerPatch
+  # ref: https://github.com/newrelic/newrelic-ruby-agent/blob/dev/lib/boot/strap.rb
+  def require(*_groups)


Suggested change

def require(*_groups)

def require(...)

kaylareopelle · 2025-07-29T00:55:22Z

Thank you for re-reviewing, @arielvalentin!

Nit: The upstream docs uses mixed language but the headings at least say "Zero-Code instrumentation" as opposed to auto-instrumentation. How would you feel about naming this gem zero-code as opposed to auto?

There's an open PR to change the naming to replace "zero-code" in favor of "automatic" on the public docs: open-telemetry/opentelemetry.io#7157

They're having some CI issues, so the changes haven't been deployed yet.

arielvalentin · 2025-07-30T03:43:52Z

opentelemetry-auto-instrumentation/example/app.rb

+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+


I think though that what we want is to see what "real world" applications would do. They require bundler and we are hooking into that flow. So event though we are requiring bundler as part of the Ruby run, wouldn't we want our examples to reflect how people would use it?

arielvalentin · 2025-07-30T03:46:27Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+module OTelBundlerPatch
+  # Nested module to handle OpenTelemetry initialization logic
+  module OTelInitializer
+    OTEL_INSTRUMENTATION_MAP = {


to be clear. This will add the constant to Bundler::Runtime as well right?

Yes, it will be part of Bundler::Runtime.
I also added another rails example for validation (with separated readme).

arielvalentin · 2025-07-30T03:48:39Z

opentelemetry-auto-instrumentation/lib/opentelemetry-auto-instrumentation.rb

+
+Bundler::Runtime.prepend(OTelBundlerPatch)
+
+Bundler.require if ENV['OTEL_RUBY_REQUIRE_BUNDLER'] == 'true'


what is the relationshiop between using Bundler.require and if someone uses bundler/setup?

Will this work as expected?

Do you mean the user’s Ruby file looks like this?

require 'bundler/setup' Bundler.require

If OTEL_RUBY_OPERATOR is set to true, the gem path from /otel-auto-instrumentation-ruby will be added to $LOAD_PATH.
Ideally, /otel-auto-instrumentation-ruby should only contain OpenTelemetry-related gems. If it includes dependencies that the OpenTelemetry SDK relies on, and the user’s gem path contains the same gems but with different versions, version conflicts may occur. To avoid this, the user can define OTEL_RUBY_UNLOAD_LIBRARY to exclude specific gems (e.g., google-protobuf).

If OTEL_RUBY_OPERATOR is set to false, the default Gem.dir will be loaded into $LOAD_PATH.

require 'bundler/setup' only adds the gems specified in the Gemfile to $LOAD_PATH, so if everything is configured correctly, it should not cause conflicts.

Will this work as expected?

Do you mean: will the script still inject the auto-instrumentation code if the user has both bundler/setup and Bundler.require?
Yes—it should work as expected. In fact, a default Rails app includes both, and it worked in the demo we tested last time.

…trib into auto-instrumentation

xuan-cao-swi added 8 commits January 29, 2025 14:59

feat: init commit for zero-code instrumentation

9d13c5a

lint

444959a

rubocop

4c0c97b

enforce the dependencies version

8a2fadd

assume use bundle exec all the time

f96a637

merge

ef5f348

update readme

60a3ab2

cspell

3512225

xuan-cao-swi marked this pull request as ready for review February 26, 2025 19:13

xuan-cao-swi requested review from fbogsany, mwear, robertlaurin, dazuma, ericmustin, arielvalentin, ahayworth, plantfansam, robbkidd, simi, kaylareopelle and a team as code owners February 26, 2025 19:13

xuan-cao-swi mentioned this pull request Mar 3, 2025

Autoinstrumentation for opentelemetry ruby open-telemetry/opentelemetry-operator#3762

Open

github-actions bot added the stale Marks an issue/PR stale label Mar 29, 2025

kaylareopelle removed the stale Marks an issue/PR stale label Apr 2, 2025

update the script

2c880da

github-actions bot added the stale Marks an issue/PR stale label May 3, 2025

xuan-cao-swi requested a review from arielvalentin June 17, 2025 17:56

xuan-cao-swi mentioned this pull request Jun 17, 2025

[autoinstrumentation] add ruby autoinstrumentation open-telemetry/opentelemetry-operator#3756

Open

Merge branch 'main' into auto-instrumentation

7f285d2

kaylareopelle reviewed Jul 17, 2025

View reviewed changes

kaylareopelle changed the title ~~feat: add gem for zero-code-instrumentation in otel-operator~~ feat: add gem for auto-instrumentation in otel-operator Jul 18, 2025

xuan-cao-swi and others added 13 commits July 21, 2025 14:12

Update opentelemetry-auto-instrumentation/README.md

1080edc

Co-authored-by: Kayla Reopelle <[email protected]>

Update opentelemetry-auto-instrumentation/README.md

74c73f7

Co-authored-by: Kayla Reopelle <[email protected]>

Update .github/workflows/ci-contrib.yml

f024787

Co-authored-by: Kayla Reopelle <[email protected]>

Update .github/workflows/ci-contrib.yml

395ce32

Co-authored-by: Kayla Reopelle <[email protected]>

Update opentelemetry-auto-instrumentation/README.md

5fe8419

Co-authored-by: Kayla Reopelle <[email protected]>

Update opentelemetry-auto-instrumentation/README.md

56d7ba5

Co-authored-by: Kayla Reopelle <[email protected]>

Merge branch 'main' into auto-instrumentation

5475748

update

cd42f0b

Merge branch 'main' into auto-instrumentation

d7f3452

update workflow file

c03917f

Merge branch 'auto-instrumentation' of github.com:xuan-cao-swi/opente…

c5b0c8e

…lemetry-ruby-contrib into auto-instrumentation

update ci

501918b

use custom ci for auto-instrumentation

5bd69ee

arielvalentin reviewed Jul 27, 2025

View reviewed changes

xuan-cao-swi and others added 5 commits July 29, 2025 11:28

Merge branch 'main' into auto-instrumentation

e6cb868

revision and fix ci

e625a80

lint

f2781df

reverse ci to original

c478025

moved to a nested module

a462533

arielvalentin reviewed Jul 30, 2025

View reviewed changes

xuan-cao-swi added 3 commits August 8, 2025 10:38

Merge branch 'main' of github.com:xuan-cao-swi/opentelemetry-ruby-con…

e680d0a

…trib into auto-instrumentation

doc: add more test example

f811cbe

typo

5c032ca

	puts "Auto-instrumentation failed to initialize. Error: #{e.message}"
	$stderr.puts "Auto-instrumentation failed to initialize. Error: #{e.message}"


		source 'https://rubygems.org'

		# avoid use gemspec to avoid circular load


		Bundler::Runtime.prepend(OTelBundlerPatch)

		Bundler.require if ENV['OTEL_RUBY_REQUIRE_BUNDLER'] == 'true'

feat: add gem for auto-instrumentation in otel-operator #1384

Are you sure you want to change the base?

feat: add gem for auto-instrumentation in otel-operator #1384

Conversation

xuan-cao-swi commented Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

xuan-cao-swi commented Feb 26, 2025

Uh oh!

github-actions bot commented Mar 29, 2025

Uh oh!

github-actions bot commented May 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arielvalentin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kaylareopelle commented Jul 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuan-cao-swi Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

xuan-cao-swi commented Jan 30, 2025 •

edited

Loading

xuan-cao-swi Aug 11, 2025 •

edited

Loading