feat: [Many APIs] Add REST Interceptors which support reading metadata (#13500)

BEGIN_COMMIT_OVERRIDE
feat: Add REST Interceptors which support reading metadata
feat: Add support for reading selective GAPIC generation methods from
service YAML
chore: Update gapic-generator-python to v1.22.0
END_COMMIT_OVERRIDE

- [ ] Regenerate this pull request now.

feat: Add support for reading selective GAPIC generation methods from
service YAML
chore: Update gapic-generator-python to v1.22.0

PiperOrigin-RevId: 724026024

Source-Link:
googleapis/googleapis@ad99638

Source-Link:
googleapis/googleapis-gen@e291c4d
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXBhcmFtZXRlcm1hbmFnZXIvLk93bEJvdC55YW1sIiwiaCI6ImUyOTFjNGRkMWQ2NzBlZGExOTk5OGRlNzZmOTY3ZTE2MDNhNDg5OTMifQ==
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXBoaXNoaW5nLXByb3RlY3Rpb24vLk93bEJvdC55YW1sIiwiaCI6ImUyOTFjNGRkMWQ2NzBlZGExOTk5OGRlNzZmOTY3ZTE2MDNhNDg5OTMifQ==
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXBvbGljeS10cm91Ymxlc2hvb3Rlci8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXBvbGljeXNpbXVsYXRvci8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXBvbGljeXRyb3VibGVzaG9vdGVyLWlhbS8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXByaXZhdGUtY2EvLk93bEJvdC55YW1sIiwiaCI6ImUyOTFjNGRkMWQ2NzBlZGExOTk5OGRlNzZmOTY3ZTE2MDNhNDg5OTMifQ==
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXByaXZhdGUtY2F0YWxvZy8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXByaXZpbGVnZWRhY2Nlc3NtYW5hZ2VyLy5Pd2xCb3QueWFtbCIsImgiOiJlMjkxYzRkZDFkNjcwZWRhMTk5OThkZTc2Zjk2N2UxNjAzYTQ4OTkzIn0=
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXB1YmxpYy1jYS8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXF1b3Rhcy8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJhcGlkbWlncmF0aW9uYXNzZXNzbWVudC8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlY2FwdGNoYS1lbnRlcnByaXNlLy5Pd2xCb3QueWFtbCIsImgiOiJlMjkxYzRkZDFkNjcwZWRhMTk5OThkZTc2Zjk2N2UxNjAzYTQ4OTkzIn0=
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlY29tbWVuZGF0aW9ucy1haS8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlY29tbWVuZGVyLy5Pd2xCb3QueWFtbCIsImgiOiJlMjkxYzRkZDFkNjcwZWRhMTk5OThkZTc2Zjk2N2UxNjAzYTQ4OTkzIn0=
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlZGlzLWNsdXN0ZXIvLk93bEJvdC55YW1sIiwiaCI6ImUyOTFjNGRkMWQ2NzBlZGExOTk5OGRlNzZmOTY3ZTE2MDNhNDg5OTMifQ==
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlZGlzLy5Pd2xCb3QueWFtbCIsImgiOiJlMjkxYzRkZDFkNjcwZWRhMTk5OThkZTc2Zjk2N2UxNjAzYTQ4OTkzIn0=
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlc291cmNlLW1hbmFnZXIvLk93bEJvdC55YW1sIiwiaCI6ImUyOTFjNGRkMWQ2NzBlZGExOTk5OGRlNzZmOTY3ZTE2MDNhNDg5OTMifQ==
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJlc291cmNlLXNldHRpbmdzLy5Pd2xCb3QueWFtbCIsImgiOiJlMjkxYzRkZDFkNjcwZWRhMTk5OThkZTc2Zjk2N2UxNjAzYTQ4OTkzIn0=
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXJldGFpbC8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9
Copy-Tag:
eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLXNjaGVkdWxlci8uT3dsQm90LnlhbWwiLCJoIjoiZTI5MWM0ZGQxZDY3MGVkYTE5OTk4ZGU3NmY5NjdlMTYwM2E0ODk5MyJ9

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <[email protected]>

Author: gcf-owl-bot[bot]

packages/google-cloud-phishing-protection/README.rst

@@ -26,12 +26,12 @@ In order to use this library, you first need to go through the following steps:
 1. `Select or create a Cloud Platform project.`_
 2. `Enable billing for your project.`_
 3. `Enable the Phishing Protection.`_
-4. `Setup Authentication.`_
+4. `Set up Authentication.`_
 
 .. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project
 .. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project
 .. _Enable the Phishing Protection.:  https://cloud.google.com/phishing-protection/docs/
-.. _Setup Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html
+.. _Set up Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html
 
 Installation
 ~~~~~~~~~~~~

packages/google-cloud-phishing-protection/google/cloud/phishingprotection/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.13.0"  # {x-release-please-version}
+__version__ = "0.0.0"  # {x-release-please-version}

packages/google-cloud-phishing-protection/google/cloud/phishingprotection_v1beta1/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "1.13.0"  # {x-release-please-version}
+__version__ = "0.0.0"  # {x-release-please-version}

packages/google-cloud-phishing-protection/google/cloud/phishingprotection_v1beta1/services/phishing_protection_service_v1_beta1/client.py

@@ -14,6 +14,8 @@
 # limitations under the License.
 #
 from collections import OrderedDict
+from http import HTTPStatus
+import json
 import logging as std_logging
 import os
 import re
@@ -470,6 +472,33 @@ def _validate_universe_domain(self):
         # NOTE (b/349488459): universe validation is disabled until further notice.
         return True
 
+    def _add_cred_info_for_auth_errors(
+        self, error: core_exceptions.GoogleAPICallError
+    ) -> None:
+        """Adds credential info string to error details for 401/403/404 errors.
+
+        Args:
+            error (google.api_core.exceptions.GoogleAPICallError): The error to add the cred info.
+        """
+        if error.code not in [
+            HTTPStatus.UNAUTHORIZED,
+            HTTPStatus.FORBIDDEN,
+            HTTPStatus.NOT_FOUND,
+        ]:
+            return
+
+        cred = self._transport._credentials
+
+        # get_cred_info is only available in google-auth>=2.35.0
+        if not hasattr(cred, "get_cred_info"):
+            return
+
+        # ignore the type check since pypy test fails when get_cred_info
+        # is not available
+        cred_info = cred.get_cred_info()  # type: ignore
+        if cred_info and hasattr(error._details, "append"):
+            error._details.append(json.dumps(cred_info))
+
     @property
     def api_endpoint(self):
         """Return the API endpoint used by the client instance.

packages/google-cloud-phishing-protection/google/cloud/phishingprotection_v1beta1/services/phishing_protection_service_v1_beta1/transports/rest.py

@@ -102,12 +102,38 @@ def post_report_phishing(
     ) -> phishingprotection.ReportPhishingResponse:
         """Post-rpc interceptor for report_phishing
 
-        Override in a subclass to manipulate the response
+        DEPRECATED. Please use the `post_report_phishing_with_metadata`
+        interceptor instead.
+
+        Override in a subclass to read or manipulate the response
         after it is returned by the PhishingProtectionServiceV1Beta1 server but before
-        it is returned to user code.
+        it is returned to user code. This `post_report_phishing` interceptor runs
+        before the `post_report_phishing_with_metadata` interceptor.
         """
         return response
 
+    def post_report_phishing_with_metadata(
+        self,
+        response: phishingprotection.ReportPhishingResponse,
+        metadata: Sequence[Tuple[str, Union[str, bytes]]],
+    ) -> Tuple[
+        phishingprotection.ReportPhishingResponse,
+        Sequence[Tuple[str, Union[str, bytes]]],
+    ]:
+        """Post-rpc interceptor for report_phishing
+
+        Override in a subclass to read or manipulate the response or metadata after it
+        is returned by the PhishingProtectionServiceV1Beta1 server but before it is returned to user code.
+
+        We recommend only using this `post_report_phishing_with_metadata`
+        interceptor in new development instead of the `post_report_phishing` interceptor.
+        When both interceptors are used, this `post_report_phishing_with_metadata` interceptor runs after the
+        `post_report_phishing` interceptor. The (possibly modified) response returned by
+        `post_report_phishing` will be passed to
+        `post_report_phishing_with_metadata`.
+        """
+        return response, metadata
+
 
 @dataclasses.dataclass
 class PhishingProtectionServiceV1Beta1RestStub:
@@ -325,6 +351,10 @@ def __call__(
             json_format.Parse(response.content, pb_resp, ignore_unknown_fields=True)
 
             resp = self._interceptor.post_report_phishing(resp)
+            response_metadata = [(k, str(v)) for k, v in response.headers.items()]
+            resp, _ = self._interceptor.post_report_phishing_with_metadata(
+                resp, response_metadata
+            )
             if CLIENT_LOGGING_SUPPORTED and _LOGGER.isEnabledFor(
                 logging.DEBUG
             ):  # pragma: NO COVER

packages/google-cloud-phishing-protection/noxfile.py

@@ -382,20 +382,29 @@ def docfx(session):
     ["python", "upb", "cpp"],
 )
 def prerelease_deps(session, protobuf_implementation):
-    """Run all tests with prerelease versions of dependencies installed."""
+    """
+    Run all tests with pre-release versions of dependencies installed
+    rather than the standard non pre-release versions.
+    Pre-releases versions can be installed using
+    `pip install --pre <package>`.
+    """
 
     if protobuf_implementation == "cpp" and session.python in ("3.11", "3.12", "3.13"):
         session.skip("cpp implementation is not supported in python 3.11+")
 
     # Install all dependencies
-    session.install("-e", ".[all, tests, tracing]")
+    session.install("-e", ".")
+
     unit_deps_all = UNIT_TEST_STANDARD_DEPENDENCIES + UNIT_TEST_EXTERNAL_DEPENDENCIES
+    # Install dependencies for the unit test environment
     session.install(*unit_deps_all)
+
     system_deps_all = (
         SYSTEM_TEST_STANDARD_DEPENDENCIES
         + SYSTEM_TEST_EXTERNAL_DEPENDENCIES
         + SYSTEM_TEST_EXTRAS
     )
+    # Install dependencies for the system test environment
     session.install(*system_deps_all)
 
     # Because we test minimum dependency versions on the minimum Python
@@ -417,6 +426,7 @@ def prerelease_deps(session, protobuf_implementation):
         )
     ]
 
+    # Install dependencies specified in `testing/constraints-X.txt`.
     session.install(*constraints_deps)
 
     prerel_deps = [
@@ -458,3 +468,70 @@ def prerelease_deps(session, protobuf_implementation):
             "PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION": protobuf_implementation,
         },
     )
+
+
+@nox.session(python="3.13")
+@nox.parametrize(
+    "protobuf_implementation",
+    ["python", "upb"],
+)
+def core_deps_from_source(session, protobuf_implementation):
+    """Run all tests with local versions of core dependencies installed,
+    rather than pulling core dependencies from PyPI.
+    """
+
+    # Install all dependencies
+    session.install(".")
+
+    # Install dependencies for the unit test environment
+    unit_deps_all = UNIT_TEST_STANDARD_DEPENDENCIES + UNIT_TEST_EXTERNAL_DEPENDENCIES
+    session.install(*unit_deps_all)
+
+    # Install dependencies for the system test environment
+    system_deps_all = (
+        SYSTEM_TEST_STANDARD_DEPENDENCIES
+        + SYSTEM_TEST_EXTERNAL_DEPENDENCIES
+        + SYSTEM_TEST_EXTRAS
+    )
+    session.install(*system_deps_all)
+
+    # Because we test minimum dependency versions on the minimum Python
+    # version, the first version we test with in the unit tests sessions has a
+    # constraints file containing all dependencies and extras that should be installed.
+    with open(
+        CURRENT_DIRECTORY
+        / "testing"
+        / f"constraints-{UNIT_TEST_PYTHON_VERSIONS[0]}.txt",
+        encoding="utf-8",
+    ) as constraints_file:
+        constraints_text = constraints_file.read()
+
+    # Ignore leading whitespace and comment lines.
+    constraints_deps = [
+        match.group(1)
+        for match in re.finditer(
+            r"^\s*(\S+)(?===\S+)", constraints_text, flags=re.MULTILINE
+        )
+    ]
+
+    # Install dependencies specified in `testing/constraints-X.txt`.
+    session.install(*constraints_deps)
+
+    core_dependencies_from_source = [
+        "google-api-core @ git+https://github.com/googleapis/python-api-core.git",
+        "google-auth @ git+https://github.com/googleapis/google-auth-library-python.git",
+        f"{CURRENT_DIRECTORY}/../googleapis-common-protos",
+        f"{CURRENT_DIRECTORY}/../grpc-google-iam-v1",
+        "proto-plus @ git+https://github.com/googleapis/proto-plus-python.git",
+    ]
+
+    for dep in core_dependencies_from_source:
+        session.install(dep, "--ignore-installed", "--no-deps")
+
+    session.run(
+        "py.test",
+        "tests/unit",
+        env={
+            "PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION": protobuf_implementation,
+        },
+    )

packages/google-cloud-phishing-protection/samples/generated_samples/snippet_metadata_google.cloud.phishingprotection.v1beta1.json

@@ -8,7 +8,7 @@
     ],
     "language": "PYTHON",
     "name": "google-cloud-phishing-protection",
-    "version": "1.13.0"
+    "version": "0.1.0"
   },
   "snippets": [
     {

packages/google-cloud-phishing-protection/tests/unit/gapic/phishingprotection_v1beta1/test_phishing_protection_service_v1_beta1.py

@@ -59,6 +59,13 @@
 )
 from google.cloud.phishingprotection_v1beta1.types import phishingprotection
 
+CRED_INFO_JSON = {
+    "credential_source": "/path/to/file",
+    "credential_type": "service account credentials",
+    "principal": "[email protected]",
+}
+CRED_INFO_STRING = json.dumps(CRED_INFO_JSON)
+
 
 async def mock_async_gen(data, chunk_size=1):
     for i in range(0, len(data)):  # pragma: NO COVER
@@ -353,6 +360,49 @@ def test__get_universe_domain():
     assert str(excinfo.value) == "Universe Domain cannot be an empty string."
 
 
+@pytest.mark.parametrize(
+    "error_code,cred_info_json,show_cred_info",
+    [
+        (401, CRED_INFO_JSON, True),
+        (403, CRED_INFO_JSON, True),
+        (404, CRED_INFO_JSON, True),
+        (500, CRED_INFO_JSON, False),
+        (401, None, False),
+        (403, None, False),
+        (404, None, False),
+        (500, None, False),
+    ],
+)
+def test__add_cred_info_for_auth_errors(error_code, cred_info_json, show_cred_info):
+    cred = mock.Mock(["get_cred_info"])
+    cred.get_cred_info = mock.Mock(return_value=cred_info_json)
+    client = PhishingProtectionServiceV1Beta1Client(credentials=cred)
+    client._transport._credentials = cred
+
+    error = core_exceptions.GoogleAPICallError("message", details=["foo"])
+    error.code = error_code
+
+    client._add_cred_info_for_auth_errors(error)
+    if show_cred_info:
+        assert error.details == ["foo", CRED_INFO_STRING]
+    else:
+        assert error.details == ["foo"]
+
+
+@pytest.mark.parametrize("error_code", [401, 403, 404, 500])
+def test__add_cred_info_for_auth_errors_no_get_cred_info(error_code):
+    cred = mock.Mock([])
+    assert not hasattr(cred, "get_cred_info")
+    client = PhishingProtectionServiceV1Beta1Client(credentials=cred)
+    client._transport._credentials = cred
+
+    error = core_exceptions.GoogleAPICallError("message", details=[])
+    error.code = error_code
+
+    client._add_cred_info_for_auth_errors(error)
+    assert error.details == []
+
+
 @pytest.mark.parametrize(
     "client_class,transport_name",
     [
@@ -1936,11 +1986,15 @@ def test_report_phishing_rest_interceptors(null_interceptor):
         transports.PhishingProtectionServiceV1Beta1RestInterceptor,
         "post_report_phishing",
     ) as post, mock.patch.object(
+        transports.PhishingProtectionServiceV1Beta1RestInterceptor,
+        "post_report_phishing_with_metadata",
+    ) as post_with_metadata, mock.patch.object(
         transports.PhishingProtectionServiceV1Beta1RestInterceptor,
         "pre_report_phishing",
     ) as pre:
         pre.assert_not_called()
         post.assert_not_called()
+        post_with_metadata.assert_not_called()
         pb_message = phishingprotection.ReportPhishingRequest.pb(
             phishingprotection.ReportPhishingRequest()
         )
@@ -1966,6 +2020,10 @@ def test_report_phishing_rest_interceptors(null_interceptor):
         ]
         pre.return_value = request, metadata
         post.return_value = phishingprotection.ReportPhishingResponse()
+        post_with_metadata.return_value = (
+            phishingprotection.ReportPhishingResponse(),
+            metadata,
+        )
 
         client.report_phishing(
             request,
@@ -1977,6 +2035,7 @@ def test_report_phishing_rest_interceptors(null_interceptor):
 
         pre.assert_called_once()
         post.assert_called_once()
+        post_with_metadata.assert_called_once()
 
 
 def test_initialize_client_w_rest():

packages/google-cloud-private-catalog/README.rst

@@ -26,12 +26,12 @@ In order to use this library, you first need to go through the following steps:
 1. `Select or create a Cloud Platform project.`_
 2. `Enable billing for your project.`_
 3. `Enable the Private Catalog.`_
-4. `Setup Authentication.`_
+4. `Set up Authentication.`_
 
 .. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project
 .. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project
 .. _Enable the Private Catalog.:  https://cloud.google.com/private-catalog/
-.. _Setup Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html
+.. _Set up Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html
 
 Installation
 ~~~~~~~~~~~~

packages/google-cloud-private-catalog/google/cloud/privatecatalog/gapic_version.py

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-__version__ = "0.9.15"  # {x-release-please-version}
+__version__ = "0.0.0"  # {x-release-please-version}