google_pubsub.md

title	Google Pub/Sub Source
disable_toc	false
products	name icon Logs logs

{{< product-availability >}}

Use Observability Pipelines' Google Pub/Sub source to pull logs from the Google Cloud Pub/Sub messaging system. Select and set up this source when you set up a pipeline.

Prerequisites

{{% observability_pipelines/prerequisites/google_pubsub %}}

Set up the source in the pipeline UI

Select and set up this source when you set up a pipeline. The information below is for the source settings in the pipeline UI.

1. Enter the name of the source project.
2. If you have a credentials JSON file, enter the path to your credentials JSON file. The credentials file must be placed under DD_OP_DATA_DIR/config. Alternatively, you can use the GOOGLE_APPLICATION_CREDENTIALS environment variable to provide the credential path.
   • If you're using workload identity on Google Kubernetes Engine (GKE), the GOOGLE_APPLICATION_CREDENTIALS is provided for you.
   • The Worker uses standard Google authentication methods.
3. Enter the subscription name.
4. Select the decoder you want to use (Bytes, GELF, JSON, syslog).
5. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.
   Note: All file paths are made relative to the configuration data directory, which is /var/lib/observability-pipelines-worker/config/ by default. See Advanced Worker Configurations for more information. The file must be owned by the observability-pipelines-worker group and observability-pipelines-worker user, or at least readable by the group or user.
   • Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) root file in DER or PEM (X.509).
   • CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) root file in DER or PEM (X.509).
   • Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.

Set secrets

{{% observability_pipelines/set_secrets_intro %}}

{{< tabs >}} {{% tab "Secrets Management" %}}

There are no default secret identifiers for this source.

{{% /tab %}}

{{% tab "Environment Variables" %}}

{{% observability_pipelines/configure_existing_pipelines/source_env_vars/google_pubsub %}}

{{% /tab %}} {{< /tabs >}}