Google Cloud clients for ZIO

Collection of Google Cloud clients generated by Google REST API code generator including authentication. based on ZIO, Sttp (v4) and Jsoniter.

Released for Scala 3 with cross-platform support.
Supported platforms:

• ✅ JVM
  • tested java versions: 21
• ✅ Native with LLVM (via scala-native)
  • backed by libcurl
  • http backend uses sync curl implementation, async backend might be added at some point...
• ❌ JavaScript (via scala-js, could be potentially added)

Included modules / clients

• zio-gcp-auth Module providing authentication methods and http backends.
  More details about authentication under Authentication section.
• zio-gcp-storage Google Cloud Storage package based on zio-gcp-storage-v1 and zio-gcp-iamcredentials-v1 client code with support for creating Signed URLs.
• zio-gcp-sheets Google Spreadsheets package based on zio-gcp-sheets-v4 with extra methods / json codecs for convenience (see usage example).

generated clients

• zio-gcp-aiplatform-v1 Client code for Google Cloud Vertex AI API.
• zio-gcp-pubsub-v1 Client code for Google Cloud Pub/Sub API.
• zio-gcp-storage-v1Client code for Google Cloud Storage API.
• zio-gcp-iamcredentials-v1Client code for Google Cloud IAM Credentials API.
• zio-gcp-sheets-v4 Client code for Google Sheets API.

On how to add new API clients see section Adding new clients.

Getting started

To get started with sbt, add the dependency to your project in build.sbt

libraryDependencies ++= Seq(
  "com.anymindgroup" %% "zio-gcp-auth" % "0.2.7",
  // add clients based on needs
  "com.anymindgroup" %% "zio-gcp-storage" % "0.2.7", // includes zio-gcp-storage-v1 and zio-gcp-iamcredentials-v1
  "com.anymindgroup" %% "zio-gcp-sheets" % "0.2.7", // includes zio-gcp-sheets-v4
  // generated clients
  "com.anymindgroup" %% "zio-gcp-aiplatform-v1" % "0.2.7",
  "com.anymindgroup" %% "zio-gcp-pubsub-v1" % "0.2.7",
  "com.anymindgroup" %% "zio-gcp-storage-v1" % "0.2.7",
  "com.anymindgroup" %% "zio-gcp-iamcredentials-v1" % "0.2.7",
  "com.anymindgroup" %% "zio-gcp-sheets-v4" % "0.2.7",
)

In a cross-platform project via sbt-crossproject use %%% operator:

libraryDependencies += "com.anymindgroup" %%% "zio-gcp-auth" % "0.2.7"
// etc.

Client usage examples

Generate content via Vertex AI API:

//> using scala 3.7.4
//> using dep com.anymindgroup::zio-gcp-aiplatform-v1::0.2.7

import zio.*, com.anymindgroup.gcp.*, auth.defaultAccessTokenBackend
import aiplatform.v1.*, aiplatform.v1.resources.*, aiplatform.v1.schemas.*

object vertex_ai_generate_content extends ZIOAppDefault:
  def run = for
    authedBackend <- defaultAccessTokenBackend()
    endpoint       = Endpoint.`asia-northeast1`
    request        = projects.locations.publishers.Models.generateContent(
                projectsId = "my-gcp-project",
                locationsId = endpoint.location,
                publishersId = "google",
                modelsId = "gemini-2.5-flash",
                request = GoogleCloudAiplatformV1GenerateContentRequest(
                  contents = Chunk(
                    GoogleCloudAiplatformV1Content(
                      parts = Chunk(
                        GoogleCloudAiplatformV1Part(
                          text = Some("hello how are you doing?")
                        )
                      ),
                      role = Some("user"),
                    )
                  ),
                  generationConfig = Some(
                    GoogleCloudAiplatformV1GenerationConfig(
                      thinkingConfig =
                        Some(GoogleCloudAiplatformV1GenerationConfigThinkingConfig(includeThoughts = Some(true)))
                    )
                  ),
                ),
                endpointUrl = endpoint.url,
              )
    _ <- authedBackend
           .send(request)
           .flatMap:
             _.body match
               case Right(body) => ZIO.logInfo(s"Response ok: $body")
               case Left(err)   => ZIO.logError(s"Failure: $err")
  yield ()

Upload file to storage bucket, create signed url, delete file

//> using scala 3.7.4
//> using dep com.anymindgroup::zio-gcp-storage::0.2.7

import zio.*, com.anymindgroup.gcp.*, storage.*, auth.defaultAccessTokenBackend
import v1.resources.Objects, sttp.model.{Header, MediaType, Method}

object storage_example extends ZIOAppDefault:
  def run =
    for
      backend   <- defaultAccessTokenBackend()
      bucket     = "my-bucket"
      objPath    = List("folder", "my_file.txt")
      objContent = "my file content".getBytes("UTF-8")
      // insert file
      _ <- backend
             .send(
               Objects
                 .insert(bucket = bucket, name = Some(objPath.mkString("/")))
                 .headers(
                   Header.contentType(MediaType.TextPlain),
                   Header.contentLength(objContent.length),
                 )
                 .body(objContent)
             )
             .map(_.body)
             .flatMap:
               case Right(body) => ZIO.logInfo(s"Upload ok: $body")
               case Left(err)   => ZIO.dieMessage(s"Failure on upload: $err")

      // create signed url
      signedUrl <- V4SignUrlRequestBuilder
                     .create()
                     .signUrlRequest(
                       bucket = bucket,
                       resourcePath = objPath,
                       contentType = None,
                       method = Method.GET,
                       serviceAccountEmail = "example@example-project.iam.gserviceaccount.com",
                       signAlgorithm = V4SignAlgorithm.`GOOG4-RSA-SHA256`,
                       expiresInSeconds = V4SignatureExpiration.inSeconds(300),
                     )
                     .flatMap(_.send(backend).flatMap(r => ZIO.fromEither(r.body)))
      _ <- ZIO.logInfo(s"✅ Created signed url: $signedUrl")

      // delete file
      _ <- backend
             .send(Objects.delete(`object` = objPath.mkString("/"), bucket = bucket))
             .flatMap:
               _.body match
                 case Right(body) => ZIO.logInfo(s"Object deleted.")
                 case Left(err)   => ZIO.logError(s"Failure on deleting: $err")
    yield ()

Adding new clients

Look up and place the discovery document specs into the codegen/src/main/resources folder.
E.g. like:

curl 'https://redis.googleapis.com/$discovery/rest?version=v1' > codegen/src/main/resources/redis_v1.json

In build.sbt find and extend the config for clients code to generate:

lazy val gcpClientsCrossProjects: Seq[CrossProject] = for {
  (apiName, apiVersion) <- Seq(
                             "aiplatform"     -> "v1",
                             "iamcredentials" -> "v1",
                             "pubsub"         -> "v1",
                             "storage"        -> "v1",
                             // new clients can be added here
                             // 1. Place the specs into codegen/src/main/resources folder e.g.:
                             // curl 'https://redis.googleapis.com/$discovery/rest?version=v1' > codegen/src/main/resources/redis_v1.json
                             // 2. add to configuration here according to the json file name "redis_v1.json" like:
                             // "redis"          -> "v1",
                           )
  // ...
}                         

This step could be automated in the future.

Done. The package will be available as "com.anymindgroup::zio-gcp-redis-v1" on publishing.

Authentication

The module zio-gcp-auth provides methods for authentication.
It's primarily meant to run on a VM in Google Cloud and make use of compute metadata.

Currently supported credentials and tokens:

Credentials	Access token	ID token
Service account (via compute metadata)	✅	✅
User credentials	✅	❌
Service account (via private key)	❌	❌

Authentication / token provider usage examples

import zio.*, zio.Console.*, com.anymindgroup.gcp.auth.*, com.anymindgroup.http.*

object AccessTokenByUser extends ZIOAppDefault:
  def run =
    for
      // choose the required token provider
      //
      // use TokenProvider[AccessToken] if the application doesn't require identity information
      // see https://cloud.google.com/docs/authentication/token-types#access for more information
      //
      // use TokenProvider[IdToken] if the token needs to be inspected by the application
      // see https://cloud.google.com/docs/authentication/token-types#id for more information
      //
      // use TokenProvider[Token] if it doesn't matter whether the provided token is an Access or ID token
      tokenProvider: TokenProvider[Token] <-
        httpBackendScoped().flatMap: backend =>
          // Default token provider looks up credentials in the following order
          // 1. Credentials key file under the location set via GOOGLE_APPLICATION_CREDENTIALS environment variable
          // 2. Default applications credentials
          //    Linux, macOS: $HOME/.config/gcloud/application_default_credentials.json
          //    Windows: %APPDATA%\gcloud\application_default_credentials.json
          // 3. Attached service account via compute metadata service https://cloud.google.com/compute/docs/metadata/overview
          TokenProvider.defaultAccessTokenProvider(
            backend = backend,
            // Optional parameter: whether to lookup credentials from the compute metadata service before applications credentials
            // Default: false
            lookupComputeMetadataFirst = false,
            // Optional parameter: retry Schedule on token retrieval failures.
            // Dafault: Schedule.recurs(5)
            refreshRetrySchedule = Schedule.recurs(5),
            // Optional parameter: at what stage of expiration in percent to request a new token.
            // Default: 0.9 (90%)
            // e.g. a token that expires in 3600 seconds, will be refreshed after 3240 seconds (6 mins before expiry)
            refreshAtExpirationPercent = 0.9,
          )
      tokenReceipt <- tokenProvider.token
      token         = tokenReceipt.token
      _            <- printLine(s"Pass as bearer token to a Google Cloud API: ${token.token}")
      _            <- printLine(s"Received token at ${tokenReceipt.receivedAt}")
      _            <- printLine(s"Token expires in ${token.expiresIn.getSeconds()}s")
    yield ()

// access token retrieval without caching and auto refreshing
object SimpleTokenRetrieval extends ZIOAppDefault:
  def run = httpBackendScoped()
    .flatMap(TokenProvider.defaultAccessTokenProvider(_).flatMap(_.token))
    .flatMap(r => printLine(s"got access token: ${r.token.token} at ${r.receivedAt}"))

object PassSpecificUserAccount extends ZIOAppDefault:
  def run =
    httpBackendScoped().flatMap: backend =>
      TokenProvider
        .accessTokenProvider(
          Credentials.UserAccount(
            refreshToken = "refresh_token",
            clientId = "123.apps.googleusercontent.com",
            clientSecret = Config.Secret("user_secret"),
          ),
          backend,
        )

object SetLogLevelToDebug extends ZIOAppDefault:
  def run = ZIO.logLevel(LogLevel.Debug)(httpBackendScoped().flatMap(TokenProvider.defaultAccessTokenProvider(_)))

Change log level

All logging is using ZIO.log. This allows you to override the log level as e.g. described in this zio logging tutorial guide.
Example of setting the token provider log level to debug:

import zio.*, com.anymindgroup.gcp.auth.*, com.anymindgroup.http.*

object SetLogLevelToDebug extends ZIOAppDefault:
  def run = ZIO.logLevel(LogLevel.Debug)(httpBackendScoped().flatMap(TokenProvider.defaultAccessTokenProvider(_)))

Run examples with sbt:

sbt examples/run

Documentation

Contributing

Code of Conduct

Support

License