Merge branch 'main' into hectorhdzg/renameinmem

Author: hectorhdzg

CHANGELOG.md

@@ -11,17 +11,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
-## Version 1.34.0/0.55b0 (2025-06-04)
-
 ### Fixed
 
+- `opentelemetry-instrumentation-asgi`: fix excluded_urls in instrumentation-asgi
+  ([#3567](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3567))
 - `opentelemetry-resource-detector-containerid`: make it more quiet on platforms without cgroups
   ([#3579](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3579))
 
 ### Added
 
-- `opentelemetry-instrumentation-aiokafka` Add instrumentation of `consumer.getmany` (batch)
-  ([#3257](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3257))
+- `opentelemetry-util-http` Added support for redacting specific url query string values and url credentials in instrumentations
+  ([#3508](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3508))
+  
+## Version 1.34.0/0.55b0 (2025-06-04)
 
 ### Fixed
 
@@ -78,7 +80,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ([#3513](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3513))
 - `opentelemetry-instrumentation` Allow re-raising exception when instrumentation fails
   ([#3545](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3545))
-
+- `opentelemetry-instrumentation-aiokafka` Add instrumentation of `consumer.getmany` (batch)
+  ([#3257](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3257))
+  
 ### Deprecated
 
 - Drop support for Python 3.8, bump baseline to Python 3.9.

README.md

@@ -111,7 +111,16 @@ We meet weekly on Thursday at 9AM PT. The meeting is subject to change depending
 
 Meeting notes are available as a public [Google doc](https://docs.google.com/document/d/1CIMGoIOZ-c3-igzbd6_Pnxx1SjAkjwqoYSUWxPY8XIs/edit). For edit access, get in touch on [GitHub Discussions](https://github.com/open-telemetry/opentelemetry-python/discussions).
 
-Approvers ([@open-telemetry/python-approvers](https://github.com/orgs/open-telemetry/teams/python-approvers)):
+### Maintainers
+
+- [Aaron Abbott](https://github.com/aabmass), Google
+- [Leighton Chen](https://github.com/lzchen), Microsoft
+- [Riccardo Magliocchetti](https://github.com/xrmx), Elastic
+- [Shalev Roda](https://github.com/shalevr), Cisco
+
+For more information about the maintainer role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#maintainer).
+
+### Approvers
 
 - [Emídio Neto](https://github.com/emdneto), PicPay
 - [Jeremy Voss](https://github.com/jeremydvoss), Microsoft
@@ -121,34 +130,29 @@ Approvers ([@open-telemetry/python-approvers](https://github.com/orgs/open-telem
 - [Srikanth Chekuri](https://github.com/srikanthccv), signoz.io
 - [Tammy Baylis](https://github.com/tammy-baylis-swi), SolarWinds
 
-Emeritus Approvers:
-
-- [Ashutosh Goel](https://github.com/ashu658), Cisco
-- [Héctor Hernández](https://github.com/hectorhdzg), Microsoft
-- [Nikolay Sokolik](https://github.com/oxeye-nikolay), Oxeye
-- [Nikolay Sokolik](https://github.com/nikosokolik), Oxeye
-- [Nathaniel Ruiz Nowell](https://github.com/NathanielRN), AWS
-
-*Find more about the approver role in [community repository](https://github.com/open-telemetry/community/blob/main/community-membership.md#approver).*
+For more information about the approver role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#approver).
 
-Maintainers ([@open-telemetry/python-maintainers](https://github.com/orgs/open-telemetry/teams/python-maintainers)):
-
-- [Aaron Abbott](https://github.com/aabmass), Google
-- [Leighton Chen](https://github.com/lzchen), Microsoft
-- [Riccardo Magliocchetti](https://github.com/xrmx), Elastic
-- [Shalev Roda](https://github.com/shalevr), Cisco
-
-Emeritus Maintainers:
+### Emeritus Maintainers
 
 - [Alex Boten](https://github.com/codeboten), Lightstep
 - [Diego Hurtado](https://github.com/ocelotl), Lightstep
 - [Owais Lone](https://github.com/owais), Splunk
 - [Yusuke Tsutsumi](https://github.com/toumorokoshi), Google
 
-*Find more about the maintainer role in [community repository](https://github.com/open-telemetry/community/blob/main/community-membership.md#maintainer).*
+For more information about the emeritus role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#emeritus-maintainerapprovertriager).
+
+### Emeritus Approvers
+
+- [Ashutosh Goel](https://github.com/ashu658), Cisco
+- [Héctor Hernández](https://github.com/hectorhdzg), Microsoft
+- [Nikolay Sokolik](https://github.com/oxeye-nikolay), Oxeye
+- [Nikolay Sokolik](https://github.com/nikosokolik), Oxeye
+- [Nathaniel Ruiz Nowell](https://github.com/NathanielRN), AWS
+
+For more information about the emeritus role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#emeritus-maintainerapprovertriager).
 
-### Thanks to all the people who already contributed
+### Thanks to all of our contributors!
 
 <a href="https://github.com/open-telemetry/opentelemetry-python-contrib/graphs/contributors">
-  <img src="https://contributors-img.web.app/image?repo=open-telemetry/opentelemetry-python-contrib" />
+  <img alt="Repo contributors" src="https://contrib.rocks/image?repo=open-telemetry/opentelemetry-python-contrib" />
 </a>

instrumentation/opentelemetry-instrumentation-aiohttp-client/src/opentelemetry/instrumentation/aiohttp_client/__init__.py

@@ -135,7 +135,7 @@ def response_hook(span: Span, params: typing.Union[
 )
 from opentelemetry.trace import Span, SpanKind, TracerProvider, get_tracer
 from opentelemetry.trace.status import Status, StatusCode
-from opentelemetry.util.http import remove_url_credentials, sanitize_method
+from opentelemetry.util.http import redact_url, sanitize_method
 
 _UrlFilterT = typing.Optional[typing.Callable[[yarl.URL], str]]
 _RequestHookT = typing.Optional[
@@ -311,9 +311,9 @@ async def on_request_start(
         method = params.method
         request_span_name = _get_span_name(method)
         request_url = (
-            remove_url_credentials(trace_config_ctx.url_filter(params.url))
+            redact_url(trace_config_ctx.url_filter(params.url))
             if callable(trace_config_ctx.url_filter)
-            else remove_url_credentials(str(params.url))
+            else redact_url(str(params.url))
         )
 
         span_attributes = {}

instrumentation/opentelemetry-instrumentation-aiohttp-client/tests/test_aiohttp_client_integration.py

@@ -762,16 +762,16 @@ async def do_request(url):
         )
         self.memory_exporter.clear()
 
-    def test_credential_removal(self):
+    def test_remove_sensitive_params(self):
         trace_configs = [aiohttp_client.create_trace_config()]
 
-        app = HttpServerMock("test_credential_removal")
+        app = HttpServerMock("test_remove_sensitive_params")
 
         @app.route("/status/200")
         def index():
             return "hello"
 
-        url = "http://username:password@localhost:5000/status/200"
+        url = "http://username:password@localhost:5000/status/200?Signature=secret"
 
         with app.run("localhost", 5000):
             with self.subTest(url=url):
@@ -793,7 +793,9 @@ async def do_request(url):
                     (StatusCode.UNSET, None),
                     {
                         HTTP_METHOD: "GET",
-                        HTTP_URL: ("http://localhost:5000/status/200"),
+                        HTTP_URL: (
+                            "http://REDACTED:REDACTED@localhost:5000/status/200?Signature=REDACTED"
+                        ),
                         HTTP_STATUS_CODE: int(HTTPStatus.OK),
                     },
                 )

instrumentation/opentelemetry-instrumentation-aiohttp-server/src/opentelemetry/instrumentation/aiohttp_server/__init__.py

@@ -72,7 +72,7 @@ async def hello(request):
 )
 from opentelemetry.semconv.metrics import MetricInstruments
 from opentelemetry.trace.status import Status, StatusCode
-from opentelemetry.util.http import get_excluded_urls, remove_url_credentials
+from opentelemetry.util.http import get_excluded_urls, redact_url
 
 _duration_attrs = [
     HTTP_METHOD,
@@ -148,6 +148,7 @@ def collect_request_attributes(request: web.Request) -> Dict:
         request.url.port,
         str(request.url),
     )
+
     query_string = request.query_string
     if query_string and http_url:
         if isinstance(query_string, bytes):
@@ -161,7 +162,7 @@ def collect_request_attributes(request: web.Request) -> Dict:
         HTTP_ROUTE: _get_view_func(request),
         HTTP_FLAVOR: f"{request.version.major}.{request.version.minor}",
         HTTP_TARGET: request.path,
-        HTTP_URL: remove_url_credentials(http_url),
+        HTTP_URL: redact_url(http_url),
     }
 
     http_method = request.method

instrumentation/opentelemetry-instrumentation-aiohttp-server/tests/test_aiohttp_server_integration.py

@@ -152,3 +152,46 @@ async def test_suppress_instrumentation(
     await client.get("/test-path")
 
     assert len(memory_exporter.get_finished_spans()) == 0
+
+
+@pytest.mark.asyncio
+async def test_remove_sensitive_params(tracer, aiohttp_server):
+    """Test that sensitive information in URLs is properly redacted."""
+    _, memory_exporter = tracer
+
+    # Set up instrumentation
+    AioHttpServerInstrumentor().instrument()
+
+    # Create app with test route
+    app = aiohttp.web.Application()
+
+    async def handler(request):
+        return aiohttp.web.Response(text="hello")
+
+    app.router.add_get("/status/200", handler)
+
+    # Start the server
+    server = await aiohttp_server(app)
+
+    # Make request with sensitive data in URL
+    url = f"http://username:password@{server.host}:{server.port}/status/200?Signature=secret"
+    async with aiohttp.ClientSession() as session:
+        async with session.get(url) as response:
+            assert response.status == 200
+            assert await response.text() == "hello"
+
+    # Verify redaction in span attributes
+    spans = memory_exporter.get_finished_spans()
+    assert len(spans) == 1
+
+    span = spans[0]
+    assert span.attributes[HTTP_METHOD] == "GET"
+    assert span.attributes[HTTP_STATUS_CODE] == 200
+    assert (
+        span.attributes[HTTP_URL]
+        == f"http://{server.host}:{server.port}/status/200?Signature=REDACTED"
+    )
+
+    # Clean up
+    AioHttpServerInstrumentor().uninstrument()
+    memory_exporter.clear()

instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/__init__.py

@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-# pylint: disable=too-many-locals
+# pylint: disable=too-many-locals,too-many-lines
 
 """
 The opentelemetry-instrumentation-asgi package provides an ASGI middleware that can be used
@@ -81,19 +81,38 @@ async def hello():
 
 .. code-block:: python
 
-    def server_request_hook(span: Span, scope: dict[str, Any]):
+    from opentelemetry.trace import Span
+    from typing import Any
+    from asgiref.typing import Scope, ASGIReceiveEvent, ASGISendEvent
+    from opentelemetry.instrumentation.asgi import OpenTelemetryMiddleware
+
+    async def application(scope: Scope, receive: ASGIReceiveEvent, send: ASGISendEvent):
+        await send({
+            'type': 'http.response.start',
+            'status': 200,
+            'headers': [
+                [b'content-type', b'text/plain'],
+            ],
+        })
+
+        await send({
+            'type': 'http.response.body',
+            'body': b'Hello, world!',
+        })
+
+    def server_request_hook(span: Span, scope: Scope):
         if span and span.is_recording():
             span.set_attribute("custom_user_attribute_from_request_hook", "some-value")
 
-    def client_request_hook(span: Span, scope: dict[str, Any], message: dict[str, Any]):
+    def client_request_hook(span: Span, scope: Scope, message: dict[str, Any]):
         if span and span.is_recording():
             span.set_attribute("custom_user_attribute_from_client_request_hook", "some-value")
 
-    def client_response_hook(span: Span, scope: dict[str, Any], message: dict[str, Any]):
+    def client_response_hook(span: Span, scope: Scope, message: dict[str, Any]):
         if span and span.is_recording():
             span.set_attribute("custom_user_attribute_from_response_hook", "some-value")
 
-   OpenTelemetryMiddleware().(application, server_request_hook=server_request_hook, client_request_hook=client_request_hook, client_response_hook=client_response_hook)
+    OpenTelemetryMiddleware(application, server_request_hook=server_request_hook, client_request_hook=client_request_hook, client_response_hook=client_response_hook)
 
 Capture HTTP request and response headers
 *****************************************
@@ -254,12 +273,14 @@ def client_response_hook(span: Span, scope: dict[str, Any], message: dict[str, A
     OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS,
     OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST,
     OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE,
+    ExcludeList,
     SanitizeValue,
     _parse_url_query,
     get_custom_headers,
     normalise_request_header_name,
     normalise_response_header_name,
-    remove_url_credentials,
+    parse_excluded_urls,
+    redact_url,
     sanitize_method,
 )
 
@@ -356,7 +377,7 @@ def collect_request_attributes(
         if _report_old(sem_conv_opt_in_mode):
             _set_http_url(
                 result,
-                remove_url_credentials(http_url),
+                redact_url(http_url),
                 _StabilityMode.DEFAULT,
             )
     http_method = scope.get("method", "")
@@ -537,7 +558,7 @@ class OpenTelemetryMiddleware:
     def __init__(
         self,
         app,
-        excluded_urls=None,
+        excluded_urls: ExcludeList | str | None = None,
         default_span_details=None,
         server_request_hook: ServerRequestHook = None,
         client_request_hook: ClientRequestHook = None,
@@ -619,6 +640,8 @@ def __init__(
         self.active_requests_counter = create_http_server_active_requests(
             self.meter
         )
+        if isinstance(excluded_urls, str):
+            excluded_urls = parse_excluded_urls(excluded_urls)
         self.excluded_urls = excluded_urls
         self.default_span_details = (
             default_span_details or get_default_span_details

instrumentation/opentelemetry-instrumentation-asgi/tests/test_asgi_middleware.py

@@ -1633,6 +1633,29 @@ async def test_no_metric_for_websockets(self):
         await self.get_all_output()
         self.assertIsNone(self.memory_metrics_reader.get_metrics_data())
 
+    async def test_excluded_urls(self):
+        self.scope["path"] = "/test_excluded_urls"
+        app = otel_asgi.OpenTelemetryMiddleware(
+            simple_asgi, excluded_urls="test_excluded_urls"
+        )
+        self.seed_app(app)
+        await self.send_default_request()
+        await self.get_all_output()
+        spans = self.memory_exporter.get_finished_spans()
+        self.assertEqual(len(spans), 0)
+
+    async def test_no_excluded_urls(self):
+        self.scope["path"] = "/test_excluded_urls"
+        app = otel_asgi.OpenTelemetryMiddleware(
+            simple_asgi, excluded_urls="test_excluded_urls"
+        )
+        self.seed_app(app)
+        self.scope["path"] = "/test_no_excluded_urls"
+        await self.send_default_request()
+        await self.get_all_output()
+        spans = self.memory_exporter.get_finished_spans()
+        self.assertGreater(len(spans), 0)
+
 
 class TestAsgiAttributes(unittest.TestCase):
     def setUp(self):
@@ -1809,12 +1832,14 @@ def test_response_attributes_invalid_status_code(self):
         otel_asgi.set_status_code(self.span, "Invalid Status Code")
         self.assertEqual(self.span.set_status.call_count, 1)
 
-    def test_credential_removal(self):
+    def test_remove_sensitive_params(self):
         self.scope["server"] = ("username:password@mock", 80)
         self.scope["path"] = "/status/200"
+        self.scope["query_string"] = b"X-Goog-Signature=1234567890"
         attrs = otel_asgi.collect_request_attributes(self.scope)
         self.assertEqual(
-            attrs[SpanAttributes.HTTP_URL], "http://mock/status/200"
+            attrs[SpanAttributes.HTTP_URL],
+            "http://REDACTED:REDACTED@mock/status/200?X-Goog-Signature=REDACTED",
         )
 
     def test_collect_target_attribute_missing(self):

instrumentation/opentelemetry-instrumentation-httpx/src/opentelemetry/instrumentation/httpx/__init__.py

@@ -259,7 +259,7 @@ async def async_response_hook(span, request, response):
 from opentelemetry.trace import SpanKind, Tracer, TracerProvider, get_tracer
 from opentelemetry.trace.span import Span
 from opentelemetry.trace.status import StatusCode
-from opentelemetry.util.http import remove_url_credentials, sanitize_method
+from opentelemetry.util.http import redact_url, sanitize_method
 
 _logger = logging.getLogger(__name__)
 
@@ -313,7 +313,7 @@ def _extract_parameters(
         # In httpx >= 0.20.0, handle_request receives a Request object
         request: httpx.Request = args[0]
         method = request.method.encode()
-        url = httpx.URL(remove_url_credentials(str(request.url)))
+        url = httpx.URL(str(request.url))
         headers = request.headers
         stream = request.stream
         extensions = request.extensions
@@ -382,7 +382,7 @@ def _apply_request_client_attributes_to_span(
     )
 
     # http semconv transition: http.url -> url.full
-    _set_http_url(span_attributes, str(url), semconv)
+    _set_http_url(span_attributes, redact_url(str(url)), semconv)
 
     # Set HTTP method in metric labels
     _set_http_method(