Merge pull request rails#55353 from drcapulet/alexc-gcp-metadata-server

Remove unnecessary calls to the GCP metadata server

Author: byroot

activestorage/CHANGELOG.md

@@ -1,3 +1,22 @@
+*   Remove unnecessary calls to the GCP metadata server.
+
+    Calling Google::Auth.get_application_default triggers an explicit call to
+    the metadata server - given it was being called for significant number of
+    file operations, it can lead to considerable tail latencies and even metadata
+    server overloads. Instead, it's preferable (and significantly more efficient)
+    that applications use:
+
+    ```ruby
+    Google::Apis::RequestOptions.default.authorization = Google::Auth.get_application_default(...)
+    ```
+
+    In the cases applications do not set that, the GCP libraries automatically determine credentials.
+
+    This also enables using credentials other than those of the associated GCP
+    service account like when using impersonation.
+
+    *Alex Coomans*
+
 *   Direct upload progress accounts for server processing time.
 
     *Jeremy Daer*

activestorage/lib/active_storage/service/gcs_service.rb

@@ -213,8 +213,16 @@ def signer
         lambda do |string_to_sign|
           iam_client = Google::Apis::IamcredentialsV1::IAMCredentialsService.new
 
-          scopes = ["https://www.googleapis.com/auth/iam"]
-          iam_client.authorization = Google::Auth.get_application_default(scopes)
+          # We explicitly do not set iam_client.authorization so that it uses the
+          # credentials set by the application at Google::Apis::RequestOptions.default.authorization.
+          # If the application does not set it, the GCP libraries will automatically
+          # determine it on each call. This code previously explicitly set the
+          # authorization to Google::Auth.get_application_default which triggers
+          # an explicit call to the metadata server - given this lambda is called
+          # for a significant number of file operations, it can lead to considerable
+          # tail latencies and even metadata server overloads. Additionally, that
+          # prevented applications from being able to configure the credentials
+          # used to perform the signature operation.
 
           request = Google::Apis::IamcredentialsV1::SignBlobRequest.new(
             payload: string_to_sign