supports collecting metrics and logs (#7)

supports collecting metrics and logs

Author: CodePrometheus

CHANGELOG.md

@@ -2,6 +2,15 @@ Changes by Version
 ==================
 Release Notes.
 
+0.2.0
+------------------
+#### Features
+- Supports collecting metrics and logs.
+
+#### Plugins
+
+#### Documentation
+
 0.1.0
 ------------------
 #### Features

Gemfile

@@ -17,4 +17,8 @@ source 'https://rubygems.org'
 
 gemspec name: 'skywalking'
 
-ruby ">= 3.0.0"
+ruby ">= 3.0.0"
+
+# Optional dependencies for enhanced performance
+# Uncomment the following line to enable FFI for better system call performance
+# gem 'ffi', '~> 1.17'

docs/en/agent/meter-log-reporter.md

@@ -0,0 +1,59 @@
+# Ruby Agent Meter and Log Reporter
+
+The meter reporter feature enables collection and reporting of runtime metrics to the SkyWalking OAP backend.
+
+### Runtime Metrics (Enabled by Default)
+
+The agent automatically collects Ruby runtime metrics when `meter_reporter_active` is enabled (default: true).
+
+#### Collected Runtime Metrics
+
+**CPU Metrics:**
+
+- `instance_ruby_cpu_usage_percent` - Ruby process CPU usage percentage
+
+**Memory Metrics:**
+
+- `instance_ruby_memory_rss_mb` - Ruby process RSS memory usage in MB
+- `instance_ruby_memory_usage_percent` - Ruby process memory usage percentage
+
+**Garbage Collection Metrics:**
+
+- `instance_ruby_gc_count_total` - Total GC execution count
+- `instance_ruby_gc_minor_count_total` - Minor GC count (if available)
+- `instance_ruby_gc_major_count_total` - Major GC count (if available)
+- `instance_ruby_gc_time_total` - Total GC time in milliseconds (cumulative)
+- `instance_ruby_heap_usage_percent` - Heap memory usage percentage
+- `instance_ruby_heap_live_slots_count` - Number of live heap slots
+- `instance_ruby_heap_available_slots_count` - Number of available heap slots
+
+**Thread Metrics:**
+
+- `instance_ruby_thread_count_active` - Number of active threads (all alive threads)
+- `instance_ruby_thread_count_running` - Number of threads in running state
+
+### Log Integration
+
+When log reporter is enabled, the agent automatically:
+
+- **Intercepts Ruby Standard Logger**: Automatically patches the `Logger` class to collect logs
+- **Integrates Trace Context**: Adds trace ID, segment ID, and span ID to log messages when available
+
+#### Supported Logger
+
+Currently, the agent supports:
+
+- **Ruby Standard Logger** (`Logger` class) - Automatically intercepted and collected
+
+#### Log Configuration Options
+
+```ruby
+# Configure log reporter level (default: Logger::INFO)
+config[:log_reporter_level] = Logger::INFO
+
+# Configure log report period in seconds (default: 5)
+config[:log_report_period] = 5
+
+# Configure maximum log queue size (default: 1000)
+config[:max_log_queue_size] = 1000
+```

docs/en/setup/quick-start.md

@@ -33,7 +33,7 @@ gem build skywalking.gemspec
 
 If successful, the following will be displayed:
 
-```ruby
+```shell
   Successfully built RubyGem
   Name: skywalking
   Version: <version>
@@ -84,27 +84,39 @@ The following is an example of configuration at start:
 Skywalking.start(
   service_name: 'sw-srv',
   instance_name: 'sw-inst',
-  collector_backend_services: 'oap:11800'
+  collector_backend_services: 'oap:11800',
+  meter_reporter_active: true,
+  log_reporter_active: true,
+  meter_report_period: 30,
+  log_report_period: 10
 )
 ~~~
 
 The following is an example of a configuration file:
+
 ~~~yaml
 common: &defaults
   service_name: Ruby-Agent-Common
   log_level: debug
+  meter_reporter_active: true
+  log_reporter_active: true
+  meter_report_period: 20
+  log_report_period: 5
 
 development:
   <<: *defaults
   service_name: Ruby-Agent-Development
+  log_reporter_level: 0  # DEBUG
 
 test:
   <<: *defaults
   service_name: Ruby-Agent-Test
+  log_reporter_level: 1  # INFO
 
 production:
   <<: *defaults
   service_name: Ruby-Agent-Production
+  log_reporter_level: 2  # WARN
 ~~~
 
 The following lists all the configuration options:
@@ -127,3 +139,10 @@ The following lists all the configuration options:
 | collector_heartbeat_period      | SW_AGENT_COLLECTOR_HEARTBEAT_PERIOD      | 30                   | he agent will send heartbeat to OAP every `collector_heartbeat_period` seconds.                                                           |
 | properties_report_period_factor | SW_AGENT_PROPERTIES_REPORT_PERIOD_FACTOR | 10                   | The agent will report service instance properties every `collector_heartbeat_period * properties_report_period_factor` seconds.           |
 | max_queue_size                  | SW_AGENT_MAX_QUEUE_SIZE                  | 10000                | The maximum queue size for reporting data.                                                                                                |
+| meter_reporter_active           | SW_AGENT_METER_REPORTER_ACTIVE           | true                 | Enable/disable meter reporter for runtime metrics collection.                                                                             |
+| meter_report_period             | SW_AGENT_METER_REPORT_PERIOD             | 60                   | Meter report period in seconds.                                                                                                           |
+| max_meter_queue_size            | SW_AGENT_MAX_METER_QUEUE_SIZE            | 1000                 | Maximum meter queue size for buffering metrics data.                                                                                      |
+| log_reporter_active             | SW_AGENT_LOG_REPORTER_ACTIVE             | true                 | Enable/disable log reporter for log collection.                                                                                           |
+| log_reporter_level              | SW_AGENT_LOG_REPORTER_LEVEL              | 1 (INFO)             | Minimum log level to report (Logger::DEBUG=0, INFO=1, WARN=2, ERROR=3, FATAL=4).                                                          |
+| log_report_period               | SW_AGENT_LOG_REPORT_PERIOD               | 5                    | Log report period in seconds.                                                                                                             |
+| max_log_queue_size              | SW_AGENT_MAX_LOG_QUEUE_SIZE              | 1000                 | Maximum log queue size for buffering log data.                                                                                            |

docs/menu.yml

@@ -23,6 +23,8 @@ catalog:
     catalog:
       - name: Supported Plugins
         path: /en/agent/plugins
+      - name: Meter and Log Report
+        path: /en/agent/meter-and-log-report
   - name: Development and Contribution
     catalog:
       - name: How to Release

lib/skywalking/agent.rb

@@ -17,6 +17,7 @@
 require_relative 'environment'
 require_relative 'plugins_manager'
 require_relative 'reporter/report'
+require_relative 'reporter/log_buffer_trigger'
 require_relative 'tracing/span_context'
 require_relative 'tracing/carrier_item'
 require_relative 'tracing/segment'
@@ -62,13 +63,21 @@ def started?
       end
 
       attr_reader :logger, :agent_config
+
+      # Get the singleton instance
+      # @return [Agent, nil] the agent instance or nil if not started
+      def instance
+        @agent
+      end
     end
 
-    attr_reader :plugins, :reporter
+    attr_reader :plugins, :reporter, :log_buffer, :config
 
     def initialize(config)
+      @config = config
       @plugins = Plugins::PluginsManager.new(config)
       @reporter = Reporter::Report.new(config)
+      @log_buffer = Reporter::LogBufferTrigger.new(config)
 
       add_shutdown_hook
     end
@@ -80,6 +89,9 @@ def environment
     def start!
       @plugins.init_plugins
       @reporter.init_reporter
+      # Start log reporting thread
+      start_log_reporting_thread if @config[:log_reporter_active]
+
       self
     end
 
@@ -94,5 +106,46 @@ def add_shutdown_hook
         shutdown
       end
     end
+
+    # Check if log reporter is active
+    # @return [Boolean] true if log reporter is active
+    def log_reporter_active?
+      @config[:log_reporter_active]
+    end
+
+    private
+
+    # Start the log reporting thread
+    def start_log_reporting_thread
+      Thread.new do
+        loop do
+          break unless log_reporter_active?
+
+          process_log_queue
+          sleep @config[:log_report_period]
+        end
+      end
+    end
+
+    # Process the log queue and send data to the server
+    def process_log_queue
+      log_count = 0
+      enumerator = Enumerator.new do |yielder|
+        while (log_data = log_buffer.stream_data)
+          log_data.each do |data|
+            log_count += 1
+            yielder << data
+          end
+        end
+      end
+
+      enumerator.each_slice(10) do |batch|
+        begin
+          reporter.report_log(batch)
+        rescue => e
+          Agent.logger.warn "Failed to report log data: #{e.message}"
+        end
+      end
+    end
   end
 end

lib/skywalking/configuration.rb

@@ -100,6 +100,46 @@ class Configuration
         default: 10000,
         desc: 'The maximum queue size for reporting data'
       },
+      :meter_reporter_active => {
+        type: :bool,
+        default: true,
+        desc: 'Enable meter reporter'
+      },
+      :runtime_meter_reporter_active => {
+        type: :bool,
+        default: true,
+        desc: 'Enable Ruby runtime meter reporter'
+      },
+      :meter_report_period => {
+        type: :int,
+        default: 60,
+        desc: 'Meter report period in seconds'
+      },
+      :max_meter_queue_size => {
+        type: :int,
+        default: 1000,
+        desc: 'Maximum meter queue size'
+      },
+      :log_reporter_active => {
+        type: :bool,
+        default: true,
+        desc: 'Enable log reporter'
+      },
+      :log_reporter_level => {
+        type: :int,
+        default: Logger::INFO,
+        desc: 'Minimum log level to report (Logger::DEBUG, Logger::INFO, Logger::WARN, Logger::ERROR, Logger::FATAL)'
+      },
+      :log_report_period => {
+        type: :int,
+        default: 5,
+        desc: 'Log report period in seconds'
+      },
+      :max_log_queue_size => {
+        type: :int,
+        default: 1000,
+        desc: 'Maximum log queue size'
+      }
     }.freeze
 
     # @api private
@@ -173,18 +213,18 @@ def override_config_by_env
         next if env_value.nil?
 
         type = env_schema[:type]
-        case type
-        when :string
-          new_config[env_key] = env_value.to_s
-        when :bool
-          # rubocop:disable Performance/CollectionLiteralInLoop
-          new_config[env_key] = !%w[0 false].include?(env_value.strip.downcase)
-          # rubocop:enable Performance/CollectionLiteralInLoop
-        when :int
-          new_config[env_key] = env_value.to_s
-        else
-          env_value
-        end
+        new_config[env_key] = case type
+                              when :string
+                                env_value.to_s
+                              when :bool
+                                # rubocop:disable Performance/CollectionLiteralInLoop
+                                !%w[0 false].include?(env_value.strip.downcase)
+                              # rubocop:enable Performance/CollectionLiteralInLoop
+                              when :int
+                                env_value.to_i
+                              else
+                                env_value
+                              end
       end
 
       new_config

lib/skywalking/environment.rb

@@ -37,7 +37,7 @@ def app_name
       rescue
         nil
       end
-      
+
       def env
         ::Rails.env
       end
@@ -63,7 +63,7 @@ def app_name
       rescue
         "Sinatra"
       end
-      
+
       def env
         ENV['RACK_ENV'] || ENV['RAILS_ENV'] || 'development'
       end
@@ -81,7 +81,7 @@ def present?
       def app_name
         "Ruby"
       end
-      
+
       def env
         ENV['RACK_ENV'] || ENV['RAILS_ENV'] || 'development'
       end
@@ -111,7 +111,7 @@ def framework_root
                             "."
                           end
     end
-    
+
     def framework_env
       @framework_env ||= framework_info.env
     end

lib/skywalking/meter.rb

@@ -0,0 +1,29 @@
+#  Licensed to the Apache Software Foundation (ASF) under one or more
+#  contributor license agreements.  See the NOTICE file distributed with
+#  this work for additional information regarding copyright ownership.
+#  The ASF licenses this file to You under the Apache License, Version 2.0
+#  (the "License"); you may not use this file except in compliance with
+#  the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+require_relative 'meter/base'
+require_relative 'meter/meter_service'
+require_relative 'meter/runtime/cpu_data_source'
+require_relative 'meter/runtime/mem_data_source'
+require_relative 'meter/runtime/gc_data_source'
+require_relative 'meter/runtime/thread_data_source'
+
+module Skywalking
+  # Main module for meter functionality
+  module Meter
+    # Classes are already defined through require statements above
+    # No need to reassign them to themselves
+  end
+end