feat(instrumentations-mysql): support net.* and database semconv migration (open-telemetry#3288)

This adds support for using `OTEL_SEMCONV_STABILITY_OPT_IN` for controlled migration to stable `net.*` and `db.*` semconv. The `net.*` attributes are controlled by the `http[/dup]` token in `OTEL_SEMCONV_STABILITY_OPT_IN` (as [discussed here](open-telemetry/opentelemetry-js#5663 (comment))) and `db.*` with the `database[/dup]` token.

Refs: open-telemetry/opentelemetry-js#5663
Refs: open-telemetry#2953

Author: trentm

package-lock.json

@@ -44,27 +44,43 @@ See [examples/mysql](https://github.com/open-telemetry/opentelemetry-js-contrib/
 
 ### MySQL instrumentation Options
 
-| Options | Type | Default | Description |
-| ------- | ---- | ------- | ----------- |
-| [`enhancedDatabaseReporting`](./src/types.ts#L24) | `boolean` | `false` | If true, an attribute containing the query's parameters will be attached the spans generated to represent the query |
-
+| Options                                           | Type      | Default | Description |
+| ------------------------------------------------- | --------- | ------- | ----------- |
+| [`enhancedDatabaseReporting`](./src/types.ts#L24) | `boolean` | `false` | If true, a `db.mysql.values` attribute containing the query's parameters will be add to database spans. Note that this is not an attribute defined in [Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/database/mysql/). |
 
 ## Semantic Conventions
 
-This package uses `@opentelemetry/semantic-conventions` version `1.22+`, which implements Semantic Convention [Version 1.7.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.7.0/semantic_conventions/README.md)
+This instrumentation implements Semantic Conventions (semconv) v1.7.0. Since then, networking (in semconv v1.23.1) and database (in semconv v1.33.0) semantic conventions were stabilized. As of `@opentelemetry/instrumentation-mysql@0.55.0` support has been added for migrating to the stable semantic conventions using the `OTEL_SEMCONV_STABILITY_OPT_IN` environment variable as follows:
+
+1. Upgrade to the latest version of this instrumentation package.
+2. Set `OTEL_SEMCONV_STABILITY_OPT_IN=http/dup,database/dup` to emit both old and stable semantic conventions. (The `http` token is used to control the `net.*` attributes, the `database` token to control to `db.*` attributes.)
+3. Modify alerts, dashboards, metrics, and other processes in your Observability system to use the stable semantic conventions.
+4. Set `OTEL_SEMCONV_STABILITY_OPT_IN=http,database` to emit only the stable semantic conventions.
+
+By default, if `OTEL_SEMCONV_STABILITY_OPT_IN` includes neither of the above tokens, the old v1.7.0 semconv is used.
+The intent is to provide an approximate 6 month time window for users of this instrumentation to migrate to the new database and networking semconv, after which a new minor version will use the new semconv by default and drop support for the old semconv.
+See [the HTTP migration guide](https://opentelemetry.io/docs/specs/semconv/non-normative/http-migration/) and the [database migration guide](https://opentelemetry.io/docs/specs/semconv/non-normative/db-migration/) for details.
 
 Attributes collected:
 
-| Attribute               | Short Description                                                              |
-| ----------------------- | ------------------------------------------------------------------------------ |
-| `db.connection_string`  | The connection string used to connect to the database.                         |
-| `db.name`               | This attribute is used to report the name of the database being accessed.      |
-| `db.operation`          | The name of the operation being executed.                                      |
-| `db.statement`          | The database statement being executed.                                         |
-| `db.system`             | An identifier for the database management system (DBMS) product being used.    |
-| `db.user`               | Username for accessing the database.                                           |
-| `net.peer.name`         | Remote hostname or similar.                                                    |
-| `net.peer.port`         | Remote port number.                                                            |
+| Old semconv            | Stable semconv   | Description |
+| ---------------------- | ---------------- | ----------- |
+| `db.system`            | `db.system.name` | 'mssql' (old), 'microsoft.sql_server' (stable) |
+| `db.connection_string` | Removed          | The connection string used to connect to the database. |
+| `db.statement`         | `db.query.text`  | The database query being executed. |
+| `db.user`              | Removed          | Username for accessing the database. |
+| `db.name`              | Removed          | Integrated into new `db.namespace`. |
+| (not included)         | `db.namespace`   | The database associated with the connection, as provided at connection time. (This does not track changes made via `SELECT DATABASE()`.) |
+| `net.peer.name`        | `server.address` | Remote hostname or similar. |
+| `net.peer.port`        | `server.port`    | Remote port number. |
+
+Metrics collected:
+
+| Old semconv                   | Stable semconv | Description |
+| ----------------------------- | -------------- | ----------- |
+| `db.client.connections.usage` | (removed)      | The number of connections currently in a given state. See note below. |
+
+Note: While `db.client.connections.usage` was replaced with `db.client.connection.count` in the [semconv database migration](https://opentelemetry.io/docs/specs/semconv/non-normative/db-migration/#database-client-connection-count), the replacement metric is still unstable, so cannot be enabled via `OTEL_SEMCONV_STABILITY_OPT_IN=database`.
 
 ## Useful links
 

packages/instrumentation-mysql/README.md

@@ -56,6 +56,7 @@
   },
   "dependencies": {
     "@opentelemetry/instrumentation": "^0.208.0",
+    "@opentelemetry/semantic-conventions": "^1.33.0",
     "@types/mysql": "2.15.27"
   },
   "homepage": "https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/packages/instrumentation-mysql#readme"

packages/instrumentation-mysql/package.json

@@ -21,27 +21,44 @@ import {
   Span,
   SpanKind,
   SpanStatusCode,
+  type Attributes,
 } from '@opentelemetry/api';
 import {
   InstrumentationBase,
   InstrumentationNodeModuleDefinition,
   isWrapped,
+  SemconvStability,
+  semconvStabilityFromStr,
 } from '@opentelemetry/instrumentation';
 import {
-  DB_SYSTEM_VALUE_MYSQL,
+  ATTR_DB_NAMESPACE,
+  ATTR_DB_QUERY_TEXT,
+  ATTR_DB_SYSTEM_NAME,
+  ATTR_SERVER_ADDRESS,
+  ATTR_SERVER_PORT,
+  DB_SYSTEM_NAME_VALUE_MYSQL,
+} from '@opentelemetry/semantic-conventions';
+import {
+  ATTR_DB_CONNECTION_STRING,
+  ATTR_DB_NAME,
   ATTR_DB_STATEMENT,
   ATTR_DB_SYSTEM,
+  ATTR_DB_USER,
+  ATTR_NET_PEER_NAME,
+  ATTR_NET_PEER_PORT,
+  DB_SYSTEM_VALUE_MYSQL,
   METRIC_DB_CLIENT_CONNECTIONS_USAGE,
 } from './semconv';
 import type * as mysqlTypes from 'mysql';
 import { AttributeNames } from './AttributeNames';
 import { MySQLInstrumentationConfig } from './types';
 import {
-  getConnectionAttributes,
-  getDbStatement,
+  getConfig,
+  getDbQueryText,
   getDbValues,
+  getJDBCString,
   getSpanName,
-  getPoolName,
+  getPoolNameOld,
 } from './utils';
 /** @knipignore */
 import { PACKAGE_NAME, PACKAGE_VERSION } from './version';
@@ -53,24 +70,48 @@ type getConnectionCallbackType = (
 ) => void;
 
 export class MySQLInstrumentation extends InstrumentationBase<MySQLInstrumentationConfig> {
-  static readonly COMMON_ATTRIBUTES = {
-    [ATTR_DB_SYSTEM]: DB_SYSTEM_VALUE_MYSQL,
-  };
-  declare private _connectionsUsage: UpDownCounter;
+  private _netSemconvStability!: SemconvStability;
+  private _dbSemconvStability!: SemconvStability;
+  declare private _connectionsUsageOld: UpDownCounter;
 
   constructor(config: MySQLInstrumentationConfig = {}) {
     super(PACKAGE_NAME, PACKAGE_VERSION, config);
+    this._setSemconvStabilityFromEnv();
   }
 
-  protected override _updateMetricInstruments() {
-    this._connectionsUsage = this.meter.createUpDownCounter(
-      METRIC_DB_CLIENT_CONNECTIONS_USAGE,
-      {
-        description:
-          'The number of connections that are currently in state described by the state attribute.',
-        unit: '{connection}',
-      }
+  // Used for testing.
+  private _setSemconvStabilityFromEnv() {
+    this._netSemconvStability = semconvStabilityFromStr(
+      'http',
+      process.env.OTEL_SEMCONV_STABILITY_OPT_IN
+    );
+    this._dbSemconvStability = semconvStabilityFromStr(
+      'database',
+      process.env.OTEL_SEMCONV_STABILITY_OPT_IN
     );
+    this._updateMetricInstruments();
+  }
+
+  protected override _updateMetricInstruments() {
+    if (this._dbSemconvStability & SemconvStability.OLD) {
+      this._connectionsUsageOld = this.meter.createUpDownCounter(
+        METRIC_DB_CLIENT_CONNECTIONS_USAGE,
+        {
+          description:
+            'The number of connections that are currently in state described by the state attribute.',
+          unit: '{connection}',
+        }
+      );
+    }
+  }
+
+  /**
+   * Convenience function for updating the `db.client.connections.usage` metric.
+   * The name "count" comes from the eventually replacement for this metric per
+   * https://opentelemetry.io/docs/specs/semconv/non-normative/db-migration/#database-client-connection-count
+   */
+  private _connCountAdd(n: number, poolNameOld: string, state: string) {
+    this._connectionsUsageOld?.add(n, { state, name: poolNameOld });
   }
 
   protected init() {
@@ -154,28 +195,23 @@ export class MySQLInstrumentation extends InstrumentationBase<MySQLInstrumentati
           thisPlugin._patchGetConnection(pool)
         );
         thisPlugin._wrap(pool, 'end', thisPlugin._patchPoolEnd(pool));
-        thisPlugin._setPoolcallbacks(pool, thisPlugin, '');
+        thisPlugin._setPoolCallbacks(pool, '');
 
         return pool;
       };
     };
   }
+
   private _patchPoolEnd(pool: any) {
     return (originalPoolEnd: Function) => {
       const thisPlugin = this;
       return function end(callback?: unknown) {
         const nAll = (pool as any)._allConnections.length;
         const nFree = (pool as any)._freeConnections.length;
         const nUsed = nAll - nFree;
-        const poolName = getPoolName(pool);
-        thisPlugin._connectionsUsage.add(-nUsed, {
-          state: 'used',
-          name: poolName,
-        });
-        thisPlugin._connectionsUsage.add(-nFree, {
-          state: 'idle',
-          name: poolName,
-        });
+        const poolNameOld = getPoolNameOld(pool);
+        thisPlugin._connCountAdd(-nUsed, poolNameOld, 'used');
+        thisPlugin._connCountAdd(-nFree, poolNameOld, 'idle');
         originalPoolEnd.apply(pool, arguments);
       };
     };
@@ -218,7 +254,7 @@ export class MySQLInstrumentation extends InstrumentationBase<MySQLInstrumentati
               : String(id);
 
           const pool = nodes[nodeId].pool;
-          thisPlugin._setPoolcallbacks(pool, thisPlugin, id);
+          thisPlugin._setPoolCallbacks(pool, id);
         }
       };
     };
@@ -303,16 +339,43 @@ export class MySQLInstrumentation extends InstrumentationBase<MySQLInstrumentati
           return originalQuery.apply(connection, arguments);
         }
 
+        const attributes: Attributes = {};
+        const { host, port, database, user } = getConfig(connection.config);
+        const portNumber = parseInt(port, 10);
+        const dbQueryText = getDbQueryText(query);
+        if (thisPlugin._dbSemconvStability & SemconvStability.OLD) {
+          attributes[ATTR_DB_SYSTEM] = DB_SYSTEM_VALUE_MYSQL;
+          attributes[ATTR_DB_CONNECTION_STRING] = getJDBCString(
+            host,
+            port,
+            database
+          );
+          attributes[ATTR_DB_NAME] = database;
+          attributes[ATTR_DB_USER] = user;
+          attributes[ATTR_DB_STATEMENT] = dbQueryText;
+        }
+        if (thisPlugin._dbSemconvStability & SemconvStability.STABLE) {
+          attributes[ATTR_DB_SYSTEM_NAME] = DB_SYSTEM_NAME_VALUE_MYSQL;
+          attributes[ATTR_DB_NAMESPACE] = database;
+          attributes[ATTR_DB_QUERY_TEXT] = dbQueryText;
+        }
+        if (thisPlugin._netSemconvStability & SemconvStability.OLD) {
+          attributes[ATTR_NET_PEER_NAME] = host;
+          if (!isNaN(portNumber)) {
+            attributes[ATTR_NET_PEER_PORT] = portNumber;
+          }
+        }
+        if (thisPlugin._netSemconvStability & SemconvStability.STABLE) {
+          attributes[ATTR_SERVER_ADDRESS] = host;
+          if (!isNaN(portNumber)) {
+            attributes[ATTR_SERVER_PORT] = portNumber;
+          }
+        }
         const span = thisPlugin.tracer.startSpan(getSpanName(query), {
           kind: SpanKind.CLIENT,
-          attributes: {
-            ...MySQLInstrumentation.COMMON_ATTRIBUTES,
-            ...getConnectionAttributes(connection.config),
-          },
+          attributes,
         });
 
-        span.setAttribute(ATTR_DB_STATEMENT, getDbStatement(query));
-
         if (thisPlugin.getConfig().enhancedDatabaseReporting) {
           let values;
 
@@ -388,41 +451,22 @@ export class MySQLInstrumentation extends InstrumentationBase<MySQLInstrumentati
       };
     };
   }
-  private _setPoolcallbacks(
-    pool: mysqlTypes.Pool,
-    thisPlugin: MySQLInstrumentation,
-    id: string
-  ) {
-    //TODO:: use semantic convention
-    const poolName = id || getPoolName(pool);
-
-    pool.on('connection', connection => {
-      thisPlugin._connectionsUsage.add(1, {
-        state: 'idle',
-        name: poolName,
-      });
+
+  private _setPoolCallbacks(pool: mysqlTypes.Pool, id: string) {
+    const poolNameOld = id || getPoolNameOld(pool);
+
+    pool.on('connection', _connection => {
+      this._connCountAdd(1, poolNameOld, 'idle');
     });
 
-    pool.on('acquire', connection => {
-      thisPlugin._connectionsUsage.add(-1, {
-        state: 'idle',
-        name: poolName,
-      });
-      thisPlugin._connectionsUsage.add(1, {
-        state: 'used',
-        name: poolName,
-      });
+    pool.on('acquire', _connection => {
+      this._connCountAdd(-1, poolNameOld, 'idle');
+      this._connCountAdd(1, poolNameOld, 'used');
     });
 
-    pool.on('release', connection => {
-      thisPlugin._connectionsUsage.add(-1, {
-        state: 'used',
-        name: poolName,
-      });
-      thisPlugin._connectionsUsage.add(1, {
-        state: 'idle',
-        name: poolName,
-      });
+    pool.on('release', _connection => {
+      this._connCountAdd(1, poolNameOld, 'idle');
+      this._connCountAdd(-1, poolNameOld, 'used');
     });
   }
 }