feat: http semconv opt in files (open-telemetry#1547)

* feat: WIP semcon file re-structure

* feat: semconv testing

* Clean up appraisal file

* Appease Rubocop

* Add README and consider database migration env vars

* safe nav stability_opt_in

* Add tests for environment variable

* Appease rubocop

* Cleanup

* Test refactor

* update determine_semconv test

---------

Co-authored-by: Kayla Reopelle <[email protected]>

Author: hannahramadan

instrumentation/http/Appraisals

@@ -4,10 +4,19 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-appraise 'http-4.4' do
-  gem 'http', '~> 4.4.0'
-end
+# To faclitate HTTP semantic convention stability migration, we are using
+# appraisal to test the different semantic convention modes along with different
+# HTTP gem versions. For more information on the semantic convention modes, see:
+# https://opentelemetry.io/docs/specs/semconv/non-normative/http-migration/
+
+semconv_stability = %w[dup stable old]
+
+semconv_stability.each do |mode|
+  appraise "http-4.4.0-#{mode}" do
+    gem 'http', '~> 4.4.0'
+  end
 
-appraise 'http-3.3.0' do
-  gem 'http', '~> 3.3.0'
+  appraise "http-3.3.0-#{mode}" do
+    gem 'http', '~> 3.3.0'
+  end
 end

instrumentation/http/README.md

@@ -64,3 +64,19 @@ The `opentelemetry-instrumentation-http` gem is distributed under the Apache 2.0
 [community-meetings]: https://github.com/open-telemetry/community#community-meetings
 [slack-channel]: https://cloud-native.slack.com/archives/C01NWKKMKMY
 [discussions-url]: https://github.com/open-telemetry/opentelemetry-ruby/discussions
+
+## HTTP semantic convention stability
+
+In the OpenTelemetry ecosystem, HTTP semantic conventions have now reached a stable state. However, the initial HTTP instrumentation was introduced before this stability was achieved, which resulted in HTTP attributes being based on an older version of the semantic conventions.
+
+To facilitate the migration to stable semantic conventions, you can use the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable. This variable allows you to opt-in to the new stable conventions, ensuring compatibility and future-proofing your instrumentation.
+
+When setting the value for `OTEL_SEMCONV_STABILITY_OPT_IN`, you can specify which conventions you wish to adopt:
+
+- `http` - Emits the stable HTTP and networking conventions and ceases emitting the old conventions previously emitted by the instrumentation.
+- `http/dup` - Emits both the old and stable HTTP and networking conventions, enabling a phased rollout of the stable semantic conventions.
+- Default behavior (in the absence of either value) is to continue emitting the old HTTP and networking conventions the instrumentation previously emitted.
+
+During the transition from old to stable conventions, HTTP instrumentation code comes in three patch versions: `dup`, `old`, and `stable`. These versions are identical except for the attributes they send. Any changes to HTTP instrumentation should consider all three patches.
+
+For additional information on migration, please refer to our [documentation](https://opentelemetry.io/docs/specs/semconv/non-normative/http-migration/).

instrumentation/http/lib/opentelemetry/instrumentation/http/instrumentation.rb

@@ -10,8 +10,9 @@ module HTTP
       # The Instrumentation class contains logic to detect and install the Http instrumentation
       class Instrumentation < OpenTelemetry::Instrumentation::Base
         install do |_config|
-          require_dependencies
-          patch
+          patch_type = determine_semconv
+          send(:"require_dependencies_#{patch_type}")
+          send(:"patch_#{patch_type}")
         end
 
         present do
@@ -20,14 +21,47 @@ class Instrumentation < OpenTelemetry::Instrumentation::Base
 
         option :span_name_formatter, default: nil, validate: :callable
 
-        def patch
-          ::HTTP::Client.prepend(Patches::Client)
-          ::HTTP::Connection.prepend(Patches::Connection)
+        def determine_semconv
+          stability_opt_in = ENV.fetch('OTEL_SEMCONV_STABILITY_OPT_IN', '')
+          values = stability_opt_in.split(',').map(&:strip)
+
+          if values.include?('http/dup')
+            'dup'
+          elsif values.include?('http')
+            'stable'
+          else
+            'old'
+          end
+        end
+
+        def patch_old
+          ::HTTP::Client.prepend(Patches::Old::Client)
+          ::HTTP::Connection.prepend(Patches::Old::Connection)
+        end
+
+        def patch_dup
+          ::HTTP::Client.prepend(Patches::Dup::Client)
+          ::HTTP::Connection.prepend(Patches::Dup::Connection)
+        end
+
+        def patch_stable
+          ::HTTP::Client.prepend(Patches::Stable::Client)
+          ::HTTP::Connection.prepend(Patches::Stable::Connection)
+        end
+
+        def require_dependencies_dup
+          require_relative 'patches/dup/client'
+          require_relative 'patches/dup/connection'
+        end
+
+        def require_dependencies_old
+          require_relative 'patches/old/client'
+          require_relative 'patches/old/connection'
         end
 
-        def require_dependencies
-          require_relative 'patches/client'
-          require_relative 'patches/connection'
+        def require_dependencies_stable
+          require_relative 'patches/stable/client'
+          require_relative 'patches/stable/connection'
         end
       end
     end

instrumentation/http/lib/opentelemetry/instrumentation/http/patches/client.rb

@@ -0,0 +1,84 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module OpenTelemetry
+  module Instrumentation
+    module HTTP
+      module Patches
+        # Module using old and stable HTTP semantic conventions
+        module Dup
+          # Module to prepend to HTTP::Client for instrumentation
+          module Client
+            # Constant for the HTTP status range
+            HTTP_STATUS_SUCCESS_RANGE = (100..399)
+
+            def perform(req, options)
+              uri = req.uri
+              request_method = req.verb.to_s.upcase
+              span_name = create_request_span_name(request_method, uri.path)
+
+              attributes = {
+                # old semconv
+                'http.method' => request_method,
+                'http.scheme' => uri.scheme,
+                'http.target' => uri.path,
+                'http.url' => "#{uri.scheme}://#{uri.host}",
+                'net.peer.name' => uri.host,
+                'net.peer.port' => uri.port,
+                # stable semconv
+                'http.request.method' => request_method,
+                'url.scheme' => uri.scheme,
+                'url.path' => uri.path,
+                'url.full' => "#{uri.scheme}://#{uri.host}",
+                'server.address' => uri.host,
+                'server.port' => uri.port
+              }
+              attributes['url.query'] = uri.query unless uri.query.nil?
+              attributes.merge!(OpenTelemetry::Common::HTTP::ClientContext.attributes)
+
+              tracer.in_span(span_name, attributes: attributes, kind: :client) do |span|
+                OpenTelemetry.propagation.inject(req.headers)
+                super.tap do |response|
+                  annotate_span_with_response!(span, response)
+                end
+              end
+            end
+
+            private
+
+            def config
+              OpenTelemetry::Instrumentation::HTTP::Instrumentation.instance.config
+            end
+
+            def annotate_span_with_response!(span, response)
+              return unless response&.status
+
+              status_code = response.status.to_i
+              span.set_attribute('http.status_code', status_code) # old semconv
+              span.set_attribute('http.response.status_code', status_code) # stable semconv
+              span.status = OpenTelemetry::Trace::Status.error unless HTTP_STATUS_SUCCESS_RANGE.cover?(status_code)
+            end
+
+            def create_request_span_name(request_method, request_path)
+              if (implementation = config[:span_name_formatter])
+                updated_span_name = implementation.call(request_method, request_path)
+                updated_span_name.is_a?(String) ? updated_span_name : "HTTP #{request_method}"
+              else
+                "HTTP #{request_method}"
+              end
+            rescue StandardError
+              "HTTP #{request_method}"
+            end
+
+            def tracer
+              HTTP::Instrumentation.instance.tracer
+            end
+          end
+        end
+      end
+    end
+  end
+end

instrumentation/http/lib/opentelemetry/instrumentation/http/patches/connection.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+# Copyright The OpenTelemetry Authors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module OpenTelemetry
+  module Instrumentation
+    module HTTP
+      module Patches
+        # Module using old and stable HTTP semantic conventions
+        module Dup
+          # Module to prepend to HTTP::Connection for instrumentation
+          module Connection
+            def initialize(req, options)
+              attributes = {
+                # old semconv
+                'net.peer.name' => req.uri.host,
+                'net.peer.port' => req.uri.port,
+                # stable semconv
+                'server.address' => req.uri.host,
+                'server.port' => req.uri.port
+              }.merge!(OpenTelemetry::Common::HTTP::ClientContext.attributes)
+
+              tracer.in_span('HTTP CONNECT', attributes: attributes) do
+                super
+              end
+            end
+
+            private
+
+            def tracer
+              HTTP::Instrumentation.instance.tracer
+            end
+          end
+        end
+      end
+    end
+  end
+end