feat: Implement the AIClient and AITracker classes (#1)

Author: jsonbailey

.gitignore

@@ -7,7 +7,7 @@ doc/
 /pkg/
 /spec/reports/
 /tmp/
-
+.DS_Store
 # rspec failure tracking
 .rspec_status
 

.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

CONTRIBUTING.md

@@ -43,12 +43,12 @@ The output will appear in `docs/build/html`.
 
 ## Code organization
 
-A special case is the namespace `LaunchDarkly::AI::Impl`, and any namespaces within it. Everything under `Impl` is considered a private implementation detail: all files there are excluded from the generated documentation, and are considered subject to change at any time and not supported for direct use by application developers. We do this because Ruby's scope/visibility system is somewhat limited compared to other languages: a method can be `private` or `protected` within a class, but there is no way to make it visible to other classes in the library yet invisible to code outside of the library, and there is similarly no way to hide a class.
+A special case is the namespace `LaunchDarkly::Server::AI::Impl`, and any namespaces within it. Everything under `Impl` is considered a private implementation detail: all files there are excluded from the generated documentation, and are considered subject to change at any time and not supported for direct use by application developers. We do this because Ruby's scope/visibility system is somewhat limited compared to other languages: a method can be `private` or `protected` within a class, but there is no way to make it visible to other classes in the library yet invisible to code outside of the library, and there is similarly no way to hide a class.
 
 So, if there is a class whose existence is entirely an implementation detail, it should be in `Impl`. Similarly, classes that are _not_ in `Impl` must not expose any public members that are not meant to be part of the supported public API. This is important because of our guarantee of backward compatibility for all public APIs within a major version: we want to be able to change our implementation details to suit the needs of the code, without worrying about breaking a customer's code. Due to how the language works, we can't actually prevent an application developer from referencing those classes in their code, but this convention makes it clear that such use is discouraged and unsupported.
 
 ## Documenting types and methods
 
-All classes and public methods outside of `LaunchDarkly::AI::Impl` should have documentation comments. These are used to build the API documentation that is published at https://launchdarkly.github.io/ruby-server-sdk-ai/ and https://www.rubydoc.info/gems/launchdarkly-server-sdk-ai. The documentation generator is YARD; see https://yardoc.org/ for the comment format it uses.
+All classes and public methods outside of `LaunchDarkly::Server::AI::Impl` should have documentation comments. These are used to build the API documentation that is published at https://launchdarkly.github.io/ruby-server-sdk-ai/ and https://www.rubydoc.info/gems/launchdarkly-server-sdk-ai. The documentation generator is YARD; see https://yardoc.org/ for the comment format it uses.
 
 Please try to make the style and terminology in documentation comments consistent with other documentation comments in the library. Also, if a class or method is being added that has an equivalent in other libraries, and if we have described it in a consistent away in those other libraries, please reuse the text whenever possible (with adjustments for anything language-specific) rather than writing new text.

Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in launchdarkly-server-sdk-ai.gemspec
+gemspec

bin/console

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'launchdarkly_server_sdk_ai'
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+require 'irb'
+IRB.start(__FILE__)

bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

examples/hello-bedrock/Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gem 'aws-sdk-bedrockruntime'
+gem 'launchdarkly-server-sdk-ai', path: '../../..'

launchdarkly-server-sdk-ai.gemspec

@@ -1,23 +1,34 @@
 # frozen_string_literal: true
 
-require_relative "lib/ldclient-ai/version"
+require_relative 'lib/server/ai/version'
 
 Gem::Specification.new do |spec|
-  spec.name = "launchdarkly-server-sdk-ai"
-  spec.version = LaunchDarkly::AI::VERSION
-  spec.authors = ["LaunchDarkly"]
-  spec.email = ["[email protected]"]
-  spec.summary = "LaunchDarkly AI SDK for Ruby"
-  spec.description = "LaunchDarkly SDK AI Configs integration for the Ruby server side SDK"
-  spec.license = "Apache-2.0"
-  spec.homepage = "https://github.com/launchdarkly/ruby-server-sdk-ai"
-  spec.metadata["source_code_uri"] = "https://github.com/launchdarkly/ruby-server-sdk-ai"
-  spec.metadata["changelog_uri"] = "https://github.com/launchdarkly/ruby-server-sdk-ai/blob/main/CHANGELOG.md"
+  spec.name = 'launchdarkly-server-sdk-ai'
+  spec.version = LaunchDarkly::Server::AI::VERSION
+  spec.authors = ['LaunchDarkly']
+  spec.email = ['[email protected]']
+  spec.summary = 'LaunchDarkly AI SDK for Ruby'
+  spec.description = 'LaunchDarkly SDK AI Configs integration for the Ruby server side SDK'
+  spec.license = 'Apache-2.0'
+  spec.homepage = 'https://github.com/launchdarkly/ruby-server-sdk-ai'
+  spec.metadata['source_code_uri'] = 'https://github.com/launchdarkly/ruby-server-sdk-ai'
+  spec.metadata['changelog_uri'] = 'https://github.com/launchdarkly/ruby-server-sdk-ai/blob/main/CHANGELOG.md'
 
-  spec.files = Dir["{lib}/**/*.rb", "bin/*", "LICENSE", "*.md"]
+  spec.files = Dir['{lib}/**/*.rb', 'bin/*', 'LICENSE', '*.md']
   spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
-  spec.require_paths = ["lib"]
-  spec.required_ruby_version = ">= 3.0.0"
+  spec.require_paths = ['lib']
+  spec.required_ruby_version = '>= 3.0.0'
 
-  spec.add_runtime_dependency "launchdarkly-server-sdk", "~> 8.4.0"
-end
+  spec.add_dependency 'launchdarkly-server-sdk', '~> 8.5'
+  spec.add_dependency 'logger'
+  spec.add_dependency 'mustache', '~> 1.1'
+
+  spec.add_development_dependency 'bundler', '~> 2.0'
+  spec.add_development_dependency 'debug', '~> 1.0'
+  spec.add_development_dependency 'rake', '~> 13.0'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'rubocop', '~> 1.21'
+  spec.add_development_dependency 'rubocop-performance', '~> 1.15'
+  spec.add_development_dependency 'rubocop-rake', '~> 0.6'
+  spec.add_development_dependency 'rubocop-rspec', '~> 3.6'
+end

lib/launchdarkly-server-sdk-ai.rb

@@ -1,3 +1,30 @@
 # frozen_string_literal: true
 
-raise "Reserved for LaunchDarkly"
+require 'logger'
+require 'mustache'
+
+require 'server/ai/version'
+require 'server/ai/client'
+require 'server/ai/ai_config_tracker'
+
+module LaunchDarkly
+  module Server
+    #
+    # Namespace for the LaunchDarkly AI SDK.
+    #
+    module AI
+      #
+      # @return [Logger] the Rails logger if in Rails, or a default Logger at WARN level otherwise
+      #
+      def self.default_logger
+        if defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
+          Rails.logger
+        else
+          log = ::Logger.new($stdout)
+          log.level = ::Logger::WARN
+          log
+        end
+      end
+    end
+  end
+end

lib/server/ai/ai_config_tracker.rb

@@ -0,0 +1,245 @@
+# frozen_string_literal: true
+
+require 'ldclient-rb'
+
+module LaunchDarkly
+  module Server
+    module AI
+      #
+      # Tracks token usage for AI operations.
+      #
+      class TokenUsage
+        attr_reader :total, :input, :output
+
+        #
+        # @param total [Integer] Total number of tokens used.
+        # @param input [Integer] Number of tokens in the prompt.
+        # @param output [Integer] Number of tokens in the completion.
+        #
+        def initialize(total: nil, input: nil, output: nil)
+          @total = total
+          @input = input
+          @output = output
+        end
+      end
+
+      #
+      # Summary of metrics which have been tracked.
+      #
+      class MetricSummary
+        attr_accessor :duration, :success, :feedback, :usage, :time_to_first_token
+
+        def initialize
+          @duration = nil
+          @success = nil
+          @feedback = nil
+          @usage = nil
+          @time_to_first_token = nil
+        end
+      end
+
+      #
+      # The AIConfigTracker class is used to track AI configuration usage.
+      #
+      class AIConfigTracker
+        attr_reader :ld_client, :config_key, :context, :variation_key, :version, :summary
+
+        def initialize(ld_client:, variation_key:, config_key:, version:, context:)
+          @ld_client = ld_client
+          @variation_key = variation_key
+          @config_key = config_key
+          @version = version
+          @context = context
+          @summary = MetricSummary.new
+        end
+
+        #
+        # Track the duration of an AI operation
+        #
+        # @param duration [Integer] The duration in milliseconds
+        #
+        def track_duration(duration)
+          @summary.duration = duration
+          @ld_client.track(
+            '$ld:ai:duration:total',
+            @context,
+            flag_data,
+            duration
+          )
+        end
+
+        #
+        # Track the duration of a block of code
+        #
+        # @yield The block to measure
+        # @return The result of the block
+        #
+        def track_duration_of(&block)
+          start_time = Time.now
+          yield
+        ensure
+          duration = ((Time.now - start_time) * 1000).to_i
+          track_duration(duration)
+        end
+
+        #
+        # Track time to first token
+        #
+        # @param duration [Integer] The duration in milliseconds
+        #
+        def track_time_to_first_token(time_to_first_token)
+          @summary.time_to_first_token = time_to_first_token
+          @ld_client.track(
+            '$ld:ai:tokens:ttf',
+            @context,
+            flag_data,
+            time_to_first_token
+          )
+        end
+
+        #
+        # Track user feedback
+        #
+        # @param kind [Symbol] The kind of feedback (:positive or :negative)
+        #
+        def track_feedback(kind:)
+          @summary.feedback = kind
+          event_name = kind == :positive ? '$ld:ai:feedback:user:positive' : '$ld:ai:feedback:user:negative'
+          @ld_client.track(
+            event_name,
+            @context,
+            flag_data,
+            1
+          )
+        end
+
+        #
+        # Track a successful AI generation
+        #
+        def track_success
+          @summary.success = true
+          @ld_client.track(
+            '$ld:ai:generation',
+            @context,
+            flag_data,
+            1
+          )
+          @ld_client.track(
+            '$ld:ai:generation:success',
+            @context,
+            flag_data,
+            1
+          )
+        end
+
+        #
+        # Track an error in AI generation
+        #
+        def track_error
+          @summary.success = false
+          @ld_client.track(
+            '$ld:ai:generation',
+            @context,
+            flag_data,
+            1
+          )
+          @ld_client.track(
+            '$ld:ai:generation:error',
+            @context,
+            flag_data,
+            1
+          )
+        end
+
+        #
+        # Track token usage
+        #
+        # @param token_usage [TokenUsage] An object containing token usage details
+        #
+        def track_tokens(token_usage)
+          @summary.usage = token_usage
+          if token_usage.total.positive?
+            @ld_client.track(
+              '$ld:ai:tokens:total',
+              @context,
+              flag_data,
+              token_usage.total
+            )
+          end
+          if token_usage.input.positive?
+            @ld_client.track(
+              '$ld:ai:tokens:input',
+              @context,
+              flag_data,
+              token_usage.input
+            )
+          end
+          return unless token_usage.output.positive?
+
+          @ld_client.track(
+            '$ld:ai:tokens:output',
+            @context,
+            flag_data,
+            token_usage.output
+          )
+        end
+
+        #
+        # Track OpenAI-specific operations.
+        # This method tracks the duration, token usage, and success/error status.
+        # If the provided block raises, this method will also raise.
+        # A failed operation will not have any token usage data.
+        #
+        # @yield The block to track.
+        # @return The result of the tracked block.
+        #
+        def track_openai_metrics(&block)
+          result = track_duration_of(&block)
+          track_success
+          track_tokens(openai_to_token_usage(result[:usage])) if result[:usage]
+          result
+        rescue StandardError
+          track_error
+          raise
+        end
+
+        #
+        # Track AWS Bedrock conversation operations.
+        # This method tracks the duration, token usage, and success/error status.
+        #
+        # @yield The block to track.
+        # @return [Hash] The original response hash.
+        #
+        def track_bedrock_converse_metrics(&block)
+          result = track_duration_of(&block)
+          track_success
+          track_tokens(bedrock_to_token_usage(result[:usage])) if result[:usage]
+          result
+        rescue StandardError
+          track_error
+          raise
+        end
+
+        private def flag_data
+          { variationKey: @variation_key, configKey: @config_key, version: @version }
+        end
+
+        private def openai_to_token_usage(usage)
+          TokenUsage.new(
+            total: usage[:total_tokens] || usage['total_tokens'],
+            input: usage[:prompt_tokens] || usage['prompt_tokens'],
+            output: usage[:completion_tokens] || usage['completion_tokens']
+          )
+        end
+
+        private def bedrock_to_token_usage(usage)
+          TokenUsage.new(
+            total: usage[:total_tokens] || usage['total_tokens'],
+            input: usage[:input_tokens] || usage['input_tokens'],
+            output: usage[:output_tokens] || usage['output_tokens']
+          )
+        end
+      end
+    end
+  end
+end