|
| 1 | +--- |
| 2 | +title: Stream Google Cloud Platform into Microsoft Sentinel |
| 3 | +description: This article describes how to stream audit log data from the Google Cloud Platform (GCP) into Microsoft Sentinel. |
| 4 | +author: limwainstein |
| 5 | +ms.topic: how-to |
| 6 | +ms.date: 03/23/2023 |
| 7 | +ms.author: lwainstein |
| 8 | +#Customer intent: As a security operator, I want to ingest GCP audit log data into Microsoft Sentinel to get full security coverage and analyze and detect attacks in my multicloud environment. |
| 9 | +--- |
| 10 | + |
| 11 | +# Stream Google Cloud Platform logs into Microsoft Sentinel |
| 12 | + |
| 13 | +Organizations are increasingly moving to multicloud architectures, whether by design or due to ongoing requirements. A growing number of these organizations use applications and store data on multiple public clouds, including the Google Cloud Platform (GCP). |
| 14 | + |
| 15 | +This article describes how to ingest GCP data into Microsoft Sentinel to get full security coverage and analyze and detect attacks in your multicloud environment. |
| 16 | + |
| 17 | +With the **GCP Pub/Sub Audit Logs** connector, based on our [Codeless Connector Platform](create-codeless-connector.md?tabs=deploy-via-arm-template%2Cconnect-via-the-azure-portal) (CCP), you can ingest logs from your GCP environment using the GCP [Pub/Sub capability](https://cloud.google.com/pubsub/docs/overview). |
| 18 | + |
| 19 | +> [!IMPORTANT] |
| 20 | +> The GCP Pub/Sub Audit Logs connector is currently in PREVIEW. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability. |
| 21 | +
|
| 22 | +Once you ingest the GCP data, you can view the details of three types of audit logs: |
| 23 | +- Admin activity logs |
| 24 | +- Data access logs |
| 25 | +- Access transparency logs |
| 26 | + |
| 27 | +With these, Google's Cloud Audit Logs records a trail that practitioners can use to monitor access and detect potential threats across GCP resources. |
| 28 | + |
| 29 | +## Prerequisites |
| 30 | + |
| 31 | +Before you begin, verify that you have: |
| 32 | + |
| 33 | +- The Microsoft Sentinel solution enabled. |
| 34 | +- A defined Microsoft Sentinel workspace. |
| 35 | +- A GCP environment collecting GCP audit logs. |
| 36 | +- The Microsoft Sentinel Contributor role. |
| 37 | +- Access to edit and create resources in the GCP project. |
| 38 | + |
| 39 | +## Set up GCP environment |
| 40 | + |
| 41 | +You can set up the GCP environment in one of two ways: |
| 42 | + |
| 43 | +- [Create GCP resources via the Terraform API](#create-gcp-resources-via-the-terraform-api): Terraform provides an API for the Identity and Access Management (IAM) that creates the resources: The topic, a subscription for the topic, a workload identity pool, a workload identity provider, a service account, and a role. |
| 44 | +- [Set up GCP environment manually](#) via the GCP console. |
| 45 | + |
| 46 | +### Create GCP resources via the Terraform API |
| 47 | + |
| 48 | +1. Open [GCP Cloud Shell](https://cloud.google.com/shell/). |
| 49 | +1. Open the editor and type: |
| 50 | + |
| 51 | + ``` |
| 52 | + gcloud config set project {projectId} |
| 53 | + ``` |
| 54 | +1. In the next window, select **Authorize**. |
| 55 | +1. Copy the Terraform [GCPInitialAuthenticationSetup script](https://github.com/danielohfeld/Azure-Sentinel/tree/feature/danielohfeld/add_gcp_terraform/DataConnectors/GCP/Terraform/sentinel_resources_creation/GCPInitialAuthenticationSetup), paste the script to a new file, and save it as a .tf file. |
| 56 | +1. In the editor, type: |
| 57 | +
|
| 58 | + ``` |
| 59 | + terraform init |
| 60 | + ``` |
| 61 | +1. Type: |
| 62 | + |
| 63 | + ``` |
| 64 | + terraform apply |
| 65 | + ``` |
| 66 | +
|
| 67 | +1. Type your Microsoft tenant ID. Learn how to [find your tenant ID](../active-directory/fundamentals/active-directory-how-to-find-tenant.md). |
| 68 | +1. When asked if a workload Identity Pool has already been created for Azure, type *yes* or *no*. |
| 69 | +1. When asked if you want to create the resources listed, type *yes*. |
| 70 | +1. Save the resources parameters for later use. |
| 71 | +1. In a new folder, copy the Terraform `GCPAuditLogsSetup` script into a new file, and save it as a .tf file: |
| 72 | +
|
| 73 | + ``` |
| 74 | + cd {foldername} |
| 75 | + ``` |
| 76 | +1. In the editor, type: |
| 77 | +
|
| 78 | + ``` |
| 79 | + terraform init |
| 80 | + ``` |
| 81 | +
|
| 82 | +1. Type: |
| 83 | +
|
| 84 | + ``` |
| 85 | + terraform apply |
| 86 | + ``` |
| 87 | +
|
| 88 | + To ingest logs from an entire organization using a single Pub/Sub, type: |
| 89 | +
|
| 90 | + ``` |
| 91 | + terraform apply -var="organization-id= {organizationId} " |
| 92 | + ``` |
| 93 | +
|
| 94 | +1. Type *yes*. |
| 95 | +
|
| 96 | +1. Save the resource parameters for later use. |
| 97 | +
|
| 98 | +1. Wait five minutes before moving to the next step. |
| 99 | +
|
| 100 | +## Set up the GCP Pub/Sub connector in Microsoft Sentinel |
| 101 | +
|
| 102 | +1. Open the [Azure portal](https://portal.azure.com/) and navigate to the **Microsoft Sentinel** service. |
| 103 | +1. In the **Content hub**, in the search bar, type *Google Cloud Platform Audit Logs*. |
| 104 | +1. Install the **Google Cloud Platform Audit Logs** solution. |
| 105 | +1. Select **Data connectors**, and in the search bar, type *GCP Pub/Sub Audit Logs*. |
| 106 | +1. Select the **GCP Pub/Sub Audit Logs (Preview)** connector. |
| 107 | +1. Below the connector description, select **Open connector page**. |
| 108 | +1. In the **Configuration** area, select **Add new**. |
| 109 | +1. Type the resource parameters you created when you [created the GCP resources](#create-gcp-resources-via-the-terraform-api). Make sure that the Data Collection Endpoint Name and the Data Collection Rule Name begin with **Microsoft-Sentinel-** and select **Connect**. |
| 110 | +
|
| 111 | +## Verify that the GCP data is in the Microsoft Sentinel environment |
| 112 | +
|
| 113 | +1. To ensure that the GCP logs were successfully ingested into Microsoft Sentinel, run the following query 30 minutes after you finish to [set up the connector](#set-up-the-gcp-pubsub-connector-in-microsoft-sentinel). |
| 114 | +
|
| 115 | + ``` |
| 116 | + GCPAuditLogs |
| 117 | + | take 10 |
| 118 | + ``` |
| 119 | +
|
| 120 | +1. Enable the [health feature](enable-monitoring.md) for data connectors. |
| 121 | +
|
| 122 | +### Set up the GCP environment manually via the GCP portal |
| 123 | +
|
| 124 | +This section shows you how to set up the GCP environment manually. Alternatively, you can set up the environment [via the Terraform API](#create-gcp-resources-via-the-terraform-api). If you already set up the environment via the API, skip this section. |
| 125 | +
|
| 126 | +#### Create the role |
| 127 | +
|
| 128 | +1. In the GCP console, navigate to **IAM & Admin**. |
| 129 | +1. Select **Roles** and select **Create role**. |
| 130 | +1. Fill in the relevant details and add permissions as needed. |
| 131 | +1. Filter the permissions by the **Pub/Sub Subscriber** and **Pub/Sub Viewer** roles, and select **pubsub.subscriptions.consume** and **pubsub.subscriptions.get** permissions. |
| 132 | +1. To confirm, select **ADD**. |
| 133 | +
|
| 134 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-create-role.png" alt-text="Screenshot of adding permissions when adding a GCP role."::: |
| 135 | +
|
| 136 | +1. To create the role, select **Create**. |
| 137 | +
|
| 138 | +#### Create the service account |
| 139 | +
|
| 140 | +1. In the GCP Console, navigate to **Service Accounts**, and select **Create Service Account**. |
| 141 | +1. Fill in the relevant details and select **Create and continue**. |
| 142 | +1. Select [the role you created previously](#create-the-role), and select **Done** to create the service account. |
| 143 | +
|
| 144 | +#### Create the workload identity federation |
| 145 | +
|
| 146 | +1. In the GCP Console, navigate to **Workload Identity Federation**. |
| 147 | +1. If it's your first time using this feature, select **Get started**. Otherwise, select **Create pool**. |
| 148 | +1. Fill in the required details, and make sure that the **Tenant ID** and **Tenant name** is the TenantID **without dashes**. |
| 149 | + |
| 150 | + > [!NOTE] |
| 151 | + > To find the tenant ID, in the Azure portal, navigate to **All Services > Azure Active Directory > Overview** and copy the **TenantID**. |
| 152 | +
|
| 153 | +1. Make sure that **Enable pool** is selected. |
| 154 | +
|
| 155 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-create-identity-pool.png" alt-text="Screenshot of creating the identity pool as part of creating the GCP workload identity federation."::: |
| 156 | +
|
| 157 | +1. To add a provider to the pool: |
| 158 | + - Select **OIDC** |
| 159 | + - Type the **Issuer (URL)**: \https://sts.windows.net/33e01921-4d64-4f8c-a055-5bdaffd5e33d |
| 160 | + - Next to **Audiences**, select **Allowed audiences**, and next to **Audience 1**, type: *api://2041288c-b303-4ca0-9076-9612db3beeb2*. |
| 161 | +
|
| 162 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-add-provider-pool.png" alt-text="Screenshot of adding the provider to the pool when creating the GCP workload identity federation."::: |
| 163 | +
|
| 164 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-add-provider-pool-audiences.png" alt-text="Screenshot of adding the provider pool audiences when creating the GCP workload identity federation."::: |
| 165 | +
|
| 166 | +#### Configure the provider attributes |
| 167 | + |
| 168 | +1. Under **OIDC 1**, select **assertion.sub**. |
| 169 | +
|
| 170 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-configure-provider-attributes.png" alt-text="Screenshot of configuring the GCP provider attributes."::: |
| 171 | + |
| 172 | +1. Select **Continue** and **Save**. |
| 173 | +1. In the **Workload Identity Pools** main page, select the created pool. |
| 174 | +1. Select **Grant access**, select the [service account you created previously](#create-the-service-account), and select **All identities in the pool** as the principals. |
| 175 | +
|
| 176 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-grant-access.png" alt-text="Screenshot of granting access to the GCP service account."::: |
| 177 | +
|
| 178 | +1. Confirm that the connected service account is displayed. |
| 179 | +
|
| 180 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-connected-service-account.png" alt-text="Screenshot of viewing the connected GCP service accounts."::: |
| 181 | +
|
| 182 | +#### Create a topic |
| 183 | +
|
| 184 | +1. In the GCP console, navigate to **Topics**. |
| 185 | +1. Create a new topic and select a **Topic ID**. |
| 186 | +1. Select **Add default subscription** and under **Encryption**, select **Google-managed encryption key**. |
| 187 | +
|
| 188 | +#### Create a sink |
| 189 | +
|
| 190 | +1. In the GCP console, navigate to **Log Router**. |
| 191 | +1. Select **Create sink** and fill in the relevant details. |
| 192 | +1. Under **Sink destination**, select **Cloud Pub/Sub topic** and select [the topic you created previously](#create-a-topic). |
| 193 | +
|
| 194 | + :::image type="content" source="media/connect-google-cloud-platform/gcp-sink-destination.png" alt-text="Screenshot of defining the GCP sink destination."::: |
| 195 | +
|
| 196 | +1. If needed, filter the logs by selecting specific logs to include. Otherwise, all logs are sent. |
| 197 | +1. Select **Create sink**. |
| 198 | +
|
| 199 | +> [!NOTE] |
| 200 | +> To ingest logs for the entire organization: |
| 201 | +> 1. Select the organization under **Project**. |
| 202 | +> 1. Repeat steps 2-4, and under **Choose logs to include in the sink** in the **Log Router** section, select **Include logs ingested by this organization and all child resources**. |
| 203 | +
|
| 204 | +:::image type="content" source="media/connect-google-cloud-platform/gcp-choose-logs.png" alt-text="Screenshot of choosing which GCP logs to include in the sink."::: |
| 205 | + |
| 206 | +#### Verify that GCP can receive incoming messages |
| 207 | +
|
| 208 | +1. In the GCP console, navigate to **Subscriptions**. |
| 209 | +1. Select **Messages**, and select **PULL** to initiate a manual pull. |
| 210 | +1. Check the incoming messages. |
| 211 | +
|
| 212 | +## Next steps |
| 213 | +In this article, you learned how to ingest GCP data into Microsoft Sentinel using the GCP Pub/Sub Audit Logs connector. To learn more about Microsoft Sentinel, see the following articles: |
| 214 | +- Learn how to [get visibility into your data, and potential threats](get-visibility.md). |
| 215 | +- Get started [detecting threats with Microsoft Sentinel](detect-threats-built-in.md). |
| 216 | +- [Use workbooks](monitor-your-data.md) to monitor your data. |
0 commit comments