feat: auto instrumentation for ruby lambda layer and add ruby 3.4.0 support (#1805)

Author: xuan-cao-swi

ruby/README.md

@@ -3,29 +3,21 @@
 Scripts and files used to build AWS Lambda Layers for running OpenTelemetry on AWS Lambda for Ruby.
 
 **Requirement**
-* [Ruby 3.2.0](https://www.ruby-lang.org/en/news/2022/12/25/ruby-3-2-0-released/) (only supported version)
+* Ruby 3.2.0/3.3.0/3.4.0
 * [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
 * [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
 * [Go](https://go.dev/doc/install)
 * [Docker](https://docs.docker.com/get-docker)
 
 **Building Lambda Ruby Layer With OpenTelemetry Ruby Dependencies**
 
-1. Pull and install all the gem dependencies in to `.aws-sam` folder
+1. Run build script
 
 ```bash
-sam build -u -t template.yml
+./build.sh
 ```
 
-2. Zip all the gems file, wrapper and handler into single zip file
-
-```bash
-(cd .aws-sam/build/OTelLayer/ && zip -qr ../<your_layer_name>.zip .)
-mv .aws-sam/build/<your_layer_name>.zip .
-
-# Or run the script
-zip_ruby_layer.sh -n <your_layer_name>
-```
+Layer is stored in `src/build` folder
 
 **Default GEM_PATH**
 
@@ -70,21 +62,28 @@ For more information about aws lambda wrapper and wrapper layer, check [aws lamb
 
 ### Sample App
 
-1. Make sure the requirements are met (e.g. sam, aws, docker, ruby version.)
-2. Navigate to the path `cd ruby/sample-apps`
-3. Build the layer and function based on template.yml. You will see .aws-sam folder after executed the command
+1. Make sure the requirements are met (e.g. sam, aws, docker, ruby version.). Current sample app only support testing Ruby 3.2.0. If you wish to play with other ruby version, please modify ruby version from Runtime in sample-apps/template.yml and src/otel/layer/Makefile.
+
+2. Navigate to the path `cd ruby/src` to build layer
+
+```bash
+sam build -u -t template.yml
+```
+
+3. Navigate to the path `cd ruby/sample-apps`
+4. Build the layer and function based on template.yml. You will see .aws-sam folder after executed the command
 ```bash
 sam build -u -t template.yml
 # for different arch, define it in properties from template.yml
    # Architectures:
     #   - arm64
 ```
-4. Test with local simulation
+5. Test with local simulation
 ```bash
 sam local start-api --skip-pull-image
 ```
 
-5. curl the lambda function
+6. curl the lambda function
 ```bash
 curl http://127.0.0.1:3000
 # you should expect: Hello 1.4.1

ruby/src/otel/Dockerfile

@@ -21,28 +21,44 @@ RUN echo 'alias be="bundle exec"' >> ~/.profile
 RUN . ~/.profile \
    && cd /root/.rbenv/plugins/ruby-build && git pull && cd - \
    && rbenv install 3.2.0 \
-   && rbenv install 3.3.0
+   && rbenv install 3.3.0 \
+   && rbenv install 3.4.0
 
 WORKDIR /build/layer
 
 RUN . ~/.profile && rbenv local 3.2.0 && bundle install
 RUN . ~/.profile && rbenv local 3.3.0 && bundle install
+RUN . ~/.profile && rbenv local 3.4.0 && bundle install
 
 WORKDIR /root/.rbenv/versions/3.2.0/lib/ruby/gems/
 RUN zip -r gems-3.2.0.zip 3.2.0/
 
 WORKDIR /root/.rbenv/versions/3.3.0/lib/ruby/gems/
 RUN zip -r gems-3.3.0.zip 3.3.0/
 
-RUN ls -al /root/.rbenv/versions/3.2.0/lib/ruby/gems && ls -al /root/.rbenv/versions/3.3.0/lib/ruby/gems
+WORKDIR /root/.rbenv/versions/3.4.0/lib/ruby/gems/
+
+# rbenv install 3.4.0 get 3.4.0+1/, so need to change back to 3.4.0+1
+RUN mv 3.4.0+1/ 3.4.0/
+RUN set -e && \
+  dir=$(find /root/.rbenv/versions/3.4.0/lib/ruby/gems/ -type d -name '3.4.0+1' | head -n 1) && \
+  target=$(echo "$dir" | sed 's/3\.4\.0+1/3.4.0/') && \
+  mv "$dir" "$target"
+RUN zip -r gems-3.4.0.zip 3.4.0/
 
 # copy gems to /build/ruby/gems for zipping 
 RUN mkdir /build/ruby && mkdir /build/ruby/gems
 WORKDIR /build/ruby/gems
 RUN cp /root/.rbenv/versions/3.2.0/lib/ruby/gems/gems-3.2.0.zip . && unzip gems-3.2.0.zip && rm gems-3.2.0.zip
 RUN cp /root/.rbenv/versions/3.3.0/lib/ruby/gems/gems-3.3.0.zip . && unzip gems-3.3.0.zip && rm gems-3.3.0.zip
+RUN cp /root/.rbenv/versions/3.4.0/lib/ruby/gems/gems-3.4.0.zip . && unzip gems-3.4.0.zip && rm gems-3.4.0.zip
 RUN ls -al /build/ruby/gems
 
+# rm gem cache
+RUN rm /root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/cache/* \
+    && rm /root/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/cache/* \
+    && rm /root/.rbenv/versions/3.4.0/lib/ruby/gems/3.4.0/cache/*
+
 # zip all the gems
 WORKDIR /build
 RUN cp layer/otel-handler . && cp layer/wrapper.rb .

ruby/src/otel/layer/Gemfile

@@ -2,4 +2,4 @@ source 'https://rubygems.org'
 
 gem 'opentelemetry-sdk', '~> 1.8.0'
 gem 'opentelemetry-exporter-otlp', '~> 0.30.0'
-gem 'opentelemetry-instrumentation-aws_lambda', '~> 0.3.0'
+gem 'opentelemetry-instrumentation-all', '~> 0.76.0'

ruby/src/otel/layer/wrapper.rb

@@ -1,12 +1,40 @@
-require 'opentelemetry/sdk'
-require 'opentelemetry/exporter/otlp'
-require 'opentelemetry/instrumentation/aws_lambda'
+require 'opentelemetry-sdk'
+require 'opentelemetry-exporter-otlp'
+require 'opentelemetry-instrumentation-all'
+
+# We need to load the function code's dependencies, and _before_ any dependencies might
+# be initialized outside of the function handler, bootstrap instrumentation.
+def preload_function_dependencies
+  default_task_location = '/var/task'
+
+  handler_file = ENV.values_at('ORIG_HANDLER', '_HANDLER').compact.first&.split('.')&.first
+
+  unless handler_file && File.exist?("#{default_task_location}/#{handler_file}.rb")
+    OpenTelemetry.logger.warn { 'Could not find the original handler file to preload libraries.' }
+    return nil
+  end
+
+  libraries = File.read("#{default_task_location}/#{handler_file}.rb")
+                  .scan(/^\s*require\s+['"]([^'"]+)['"]/)
+                  .flatten
+
+  libraries.each do |lib|
+    require lib
+  rescue StandardError => e
+    OpenTelemetry.logger.warn { "Could not load library #{lib}: #{e.message}" }
+  end
+  handler_file
+end
+
+handler_file = preload_function_dependencies
+
+OpenTelemetry.logger.info { "Libraries in #{handler_file} have been preloaded." } if handler_file
 
 OpenTelemetry::SDK.configure do |c|
-  c.use 'OpenTelemetry::Instrumentation::AwsLambda'
+  c.use_all()
 end
 
 def otel_wrapper(event:, context:)
-  otel_wrapper = OpenTelemetry::Instrumentation::AwsLambda::Handler.new()
+  otel_wrapper = OpenTelemetry::Instrumentation::AwsLambda::Handler.new
   otel_wrapper.call_wrapped(event: event, context: context)
-end
+end