Add go agent so11y dashboard

docs/en/changes/changes-10.1.0.md

@@ -83,6 +83,7 @@
 * Optimize `network_address_alias` table to reduce the number of the index.
 * PromQL service: support round brackets operator.
 * Support query Alarm message Tag for auto-complete.
+* Add SkyWalking Go Agent self observability dashboard.
 
 #### UI
 

docs/en/setup/backend/dashboards-so11y-go-agent.md

@@ -0,0 +1,32 @@
+# Go Agent self observability dashboard
+
+SkyWalking go agent reports its metrics using Meter APIS to measure tracing performance.
+it also provides a dashboard to visualize the agent metrics.
+
+## Data flow
+1. SkyWalking go agent reports metrics data internally and automatically.
+2. SkyWalking OAP receives these meters through native protocols.
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+
+## Set up
+Go Agent so11y is a build-in feature, it reports meters automatically after boot.
+
+## Self observability monitoring
+Self observability monitoring provides monitoring of the runtime performance of the go agent itself. `agent.service_name` is a `Service` in Agent so11y, and land on the `Layer: SO11Y_GO_AGENT`.
+
+### Self observability metrics
+
+| Unit             | Metric Name                                           | Description                                 | Data Source         |
+|------------------|-------------------------------------------------------|---------------------------------------------|---------------------|
+| Count Per Minute | meter_sw_go_created_tracing_context_count             | Created Tracing Context Count (Per Minute)  | SkyWalking Go Agent |
+| Count Per Minute | meter_sw_go_finished_tracing_context_count            | Finished Tracing Context Count (Per Minute) | SkyWalking Go Agent |
+| Count Per Minute | meter_sw_go_created_ignored_context_count             | Created Ignored Context Count (Per Minute)  | SkyWalking Go Agent |
+| Count Per Minute | meter_sw_go_finished_ignored_context_count            | Finished Ignored Context Count (Per Minute) | SkyWalking Go Agent |
+| Count Per Minute | meter_sw_go_possible_leaked_context_count             | Possible Leak Context Count (Per Minute)    | SkyWalking Go Agent |
+| Count Per Minute | meter_sw_go_interceptor_error_count                   | Interceptor Error Count (Per Minute)        | SkyWalking Go Agent |
+| ns               | meter_sw_go_tracing_context_execution_time_percentile | Tracing Context Execution Time (ns)         | SkyWalking Go Agent |
+
+## Customizations
+You can customize your own metrics/expression/dashboard panel.
+The metrics definition and expression rules are found in `/meter-analyzer-config/go-agent.yaml`
+The self observability dashboard panel configurations are found in `/config/ui-initialized-templates/so11y_go_agent`.

docs/menu.yml

@@ -150,6 +150,8 @@ catalog:
                 path: "/en/setup/backend/dashboards-so11y-satellite"
               - name: "SkyWalking Java Agent self telemetry"
                 path: "/en/setup/backend/dashboards-so11y-java-agent"
+              - name: "SkyWalking Go Agent self telemetry"
+                path: "/en/setup/backend/dashboards-so11y-go-agent"
       - name: "Configuration Vocabulary"
         path: "/en/setup/backend/configuration-vocabulary"
       - name: "Advanced Setup"

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java

@@ -245,7 +245,13 @@ public enum Layer {
     /**
      * Kong is Cloud-Native API Gateway and AI Gateway.
      */
-    KONG(40, true);
+    KONG(40, true),
+
+    /**
+     * The self observability of SkyWalking Go Agent,
+     * which provides the abilities to measure the tracing performance and error statistics of plugins.
+     */
+    SO11Y_GO_AGENT(41, true);
 
     private final int value;
     /**

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java

@@ -78,6 +78,7 @@ public class UITemplateInitializer {
         Layer.CILIUM_SERVICE.name(),
         Layer.SO11Y_JAVA_AGENT.name(),
         Layer.KONG.name(),
+        Layer.SO11Y_GO_AGENT.name(),
         "custom"
     };
     private final UITemplateManagementService uiTemplateManagementService;

oap-server/server-starter/src/main/resources/application.yml

@@ -293,7 +293,7 @@ agent-analyzer:
     # Nginx and Envoy agents can't get the real remote address.
     # Exit spans with the component in the list would not generate the client-side instance relation metrics.
     noUpstreamRealAddressAgents: ${SW_NO_UPSTREAM_REAL_ADDRESS:6000,9000}
-    meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent} # Which files could be meter analyzed, files split by ","
+    meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent} # Which files could be meter analyzed, files split by ","
     slowCacheReadThreshold: ${SW_SLOW_CACHE_SLOW_READ_THRESHOLD:default:20,redis:10} # The slow cache read operation thresholds. Unit ms.
     slowCacheWriteThreshold: ${SW_SLOW_CACHE_SLOW_WRITE_THRESHOLD:default:20,redis:10} # The slow cache write operation thresholds. Unit ms.
 

oap-server/server-starter/src/main/resources/meter-analyzer-config/go-agent.yaml

@@ -0,0 +1,32 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: instance(['service'], ['instance'], Layer.SO11Y_GO_AGENT)
+metricPrefix: meter
+metricsRules:
+  - name: sw_go_created_tracing_context_count
+    exp: sw_go_created_tracing_context_counter.sum(['created_by', 'service', 'instance']).increase('PT1M')
+  - name: sw_go_finished_tracing_context_count
+    exp: sw_go_finished_tracing_context_counter.sum(['service', 'instance']).increase('PT1M')
+  - name: sw_go_created_ignored_context_count
+    exp: sw_go_created_ignored_context_counter.sum(['created_by', 'service', 'instance']).increase('PT1M')
+  - name: sw_go_finished_ignored_context_count
+    exp: sw_go_finished_ignored_context_counter.sum(['service', 'instance']).increase('PT1M')
+  - name: sw_go_possible_leaked_context_count
+    exp: sw_go_possible_leaked_context_counter.sum(['source', 'service', 'instance']).increase('PT1M')
+  - name: sw_go_interceptor_error_count
+    exp: sw_go_interceptor_error_counter.sum(['plugin_name', 'service', 'instance']).increase('PT1M')
+  - name: sw_go_tracing_context_execution_time_percentile
+    exp: sw_go_tracing_context_performance.sum(['le', 'service', 'instance']).histogram().histogram_percentile([50,70,90,99])

oap-server/server-starter/src/main/resources/ui-initialized-templates/menu.yaml

@@ -252,3 +252,8 @@ menus:
         description: The Java Agent for Apache SkyWalking, which provides the native tracing/metrics/logging/event/profiling abilities for Java projects.
         documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-java-agent/
         i18nKey: self_observability_java_agent
+      - title: SkyWalking Go Agent
+        layer: SO11Y_GO_AGENT
+        description: The Go Agent for Apache SkyWalking, which provides the native tracing/metrics/logging abilities for Golang projects.
+        documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-go-agent/
+        i18nKey: self_observability_go_agent

oap-server/server-starter/src/main/resources/ui-initialized-templates/so11y_go_agent/so11y-instance.json

@@ -0,0 +1,202 @@
+[
+  {
+    "id": "Self-Observability-Go-Agent-Instance",
+    "configuration": {
+      "children": [
+        {
+          "x": 0,
+          "y": 0,
+          "w": 6,
+          "h": 13,
+          "i": "14",
+          "type": "Widget",
+          "widget": {
+            "title": "Tracing Context Creation (Per Minute)",
+            "tips": "The number of created tracing contexts, including a label created_by(value=sampler,propagated)."
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "expressions": [
+            "meter_sw_go_created_tracing_context_count"
+          ]
+        },
+        {
+          "x": 6,
+          "y": 0,
+          "w": 6,
+          "h": 13,
+          "i": "6",
+          "type": "Widget",
+          "widget": {
+            "title": "Tracing Context Creation and Completion (Per Minute)",
+            "tips": "The number of created tracing contexts and finished tracing contexts."
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metricConfig": [
+            {
+              "label": "Creation"
+            },
+            {
+              "label": "Completion"
+            }
+          ],
+          "expressions": [
+            "aggregate_labels(meter_sw_go_created_tracing_context_count,sum)",
+            "meter_sw_go_finished_tracing_context_count"
+          ]
+        },
+        {
+          "x": 12,
+          "y": 0,
+          "w": 6,
+          "h": 13,
+          "i": "1",
+          "type": "Widget",
+          "widget": {
+            "title": "Ignored Context Creation (Per Minute)",
+            "tips": "The number of created ignored contexts, including a label created_by(value=sampler,propagated)."
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "expressions": [
+            "meter_sw_go_created_ignored_context_count"
+          ]
+        },
+        {
+          "x": 18,
+          "y": 0,
+          "w": 6,
+          "h": 13,
+          "i": "2",
+          "type": "Widget",
+          "widget": {
+            "title": "Ignored Context Creation and Completion (Per Minute)",
+            "tips": "The number of created ignored contexts and finished ignored contexts."
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "expressions": [
+            "aggregate_labels(meter_sw_go_created_ignored_context_count,sum)",
+            "meter_sw_go_finished_ignored_context_count"
+          ],
+          "metricConfig": [
+            {
+              "label": "Creation"
+            },
+            {
+              "label": "Completion"
+            }
+          ]
+        },
+        {
+          "x": 0,
+          "y": 13,
+          "w": 6,
+          "h": 13,
+          "i": "11",
+          "type": "Widget",
+          "widget": {
+            "title": "Possible Leaked Context (Per Minute)",
+            "tips": "The number of detected leaked contexts, including a label source(value=tracing, ignore)."
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "expressions": [
+            "meter_sw_go_possible_leaked_context_count"
+          ],
+          "metricConfig": [
+            {
+              "label": "count"
+            }
+          ]
+        },
+        {
+          "x": 12,
+          "y": 13,
+          "w": 12,
+          "h": 13,
+          "i": "8",
+          "type": "Widget",
+          "widget": {
+            "title": "Interceptor Error Count (Per Minute)",
+            "tips": "The number of errors happened in the interceptor logic, including the label plugin_name"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "expressions": [
+            "meter_sw_go_interceptor_error_count"
+          ],
+          "metricConfig": [
+            {
+              "label": "count"
+            }
+          ]
+        },
+        {
+          "x": 6,
+          "y": 13,
+          "w": 6,
+          "h": 13,
+          "i": "15",
+          "type": "Widget",
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": true,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "widget": {
+            "title": "Tracing Context Execution time (ms)",
+            "tips": "For successfully finished tracing context, it measures every interceptor's time cost."
+          },
+          "expressions": [
+            "relabels(meter_sw_go_tracing_context_execution_time_percentile,p='50,75,90,95,99',p='50,75,90,95,99')/1000000"
+          ]
+        }
+      ],
+      "layer": "SO11Y_GO_AGENT",
+      "entity": "ServiceInstance",
+      "name": "Self-Observability-Go-Agent-Instance",
+      "isRoot": false
+    }
+  }
+]

oap-server/server-starter/src/main/resources/ui-initialized-templates/so11y_go_agent/so11y-service.json

@@ -0,0 +1,62 @@
+[
+  {
+    "id": "Self-Observability-Go-Agent-Service",
+    "configuration": {
+      "children": [
+        {
+          "x": 0,
+          "y": 2,
+          "w": 24,
+          "h": 38,
+          "i": "0",
+          "type": "Widget",
+          "graph": {
+            "type": "InstanceList",
+            "dashboardName": "Self-Observability-Go-Agent-Instance",
+            "fontSize": 12
+          },
+          "metricConfig": [
+            {
+              "label": "Context Creation",
+              "detailLabel": "context_creation",
+              "unit": "Per Minute"
+            },
+            {
+              "label": "Context Completion",
+              "unit": "Per Minute",
+              "detailLabel": "context_completion"
+            }
+          ],
+          "expressions": [
+            "avg(aggregate_labels(meter_sw_go_created_tracing_context_count,sum)+aggregate_labels(meter_sw_go_created_ignored_context_count,sum))",
+            "avg(meter_sw_go_finished_tracing_context_count+meter_sw_go_finished_ignored_context_count)"
+          ],
+          "subExpressions": [
+            "aggregate_labels(meter_sw_go_created_tracing_context_count,sum)+aggregate_labels(meter_sw_go_created_ignored_context_count,sum)",
+            "meter_sw_go_finished_tracing_context_count+meter_sw_go_finished_ignored_context_count"
+          ]
+        },
+        {
+          "x": 0,
+          "y": 0,
+          "w": 24,
+          "h": 2,
+          "i": "100",
+          "type": "Text",
+          "graph": {
+            "fontColor": "theme",
+            "backgroundColor": "theme",
+            "content": "The self observability of SkyWalking Go Agent, which provides the abilities to measure the tracing performance and error statistics of plugins.",
+            "fontSize": 14,
+            "textAlign": "left",
+            "url": "https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-go-agent/"
+          }
+        }
+      ],
+      "layer": "SO11Y_GO_AGENT",
+      "entity": "Service",
+      "name": "Self-Observability-Go-Agent-Service",
+      "isRoot": true
+    }
+  }
+]