|
| 1 | +--- |
| 2 | +id: databricks-audit-source |
| 3 | +title: Databricks Audit Source |
| 4 | +sidebar_label: Databricks Audit |
| 5 | +tags: |
| 6 | + - cloud-to-cloud |
| 7 | + - databricks-audit-logs |
| 8 | +description: Learn how to collect Databricks Audit logs and send them to Sumo Logic for monitoring, auditing, and compliance. |
| 9 | +--- |
| 10 | + |
| 11 | +import ForwardToSiem from '/docs/reuse/forward-to-siem.md'; |
| 12 | +import useBaseUrl from '@docusaurus/useBaseUrl'; |
| 13 | + |
| 14 | +<img src={useBaseUrl('img/collector/databricks-icon.png')} alt="icon" width="80"/> |
| 15 | + |
| 16 | +The Sumo Logic source for Databricks enables you to collect audit logs from Databricks into Sumo Logic. This integration helps you to capture structured records of user and system activities within the Databricks workspace, including SQL queries, job executions, cluster events, and workspace changes. These logs facilitate auditing, security monitoring, and regulatory compliance. |
| 17 | + |
| 18 | +## Data collected |
| 19 | + |
| 20 | +| Polling Interval | Data | |
| 21 | +| :--- | :--- | |
| 22 | +| 5 minutes | [Audit Logs](https://docs.databricks.com/api/workspace/statementexecution/executestatement) | |
| 23 | + |
| 24 | +## Setup |
| 25 | + |
| 26 | +### Vendor configuration |
| 27 | + |
| 28 | +The Databricks Audit source requires you to provide the Databricks Audit Base URL (API Gateway URL), Warehouse ID, Client ID, and Client Secret to configure the source. Follow the below steps to generate the required values: |
| 29 | + |
| 30 | +#### Base URL |
| 31 | + |
| 32 | +Follow the below steps to get the Base URL for user configuration: |
| 33 | +1. [Sign in](https://accounts.cloud.databricks.com/login?tuuid=ee23ab7a-a2b9-4512-8973-fc787eb3c01e) to your **Databricks** workspace. |
| 34 | +2. The Base URL is visible in your browser’s address bar after login. For example, `https://YOUR-INSTANCE.databricks.com`. |
| 35 | + |
| 36 | +#### Warehouse ID |
| 37 | + |
| 38 | +The Warehouse ID is required to query Databricks SQL and fetch audit logs. Follow the below steps to get the Warehouse ID for user configuration: |
| 39 | +1. [Sign in](https://accounts.cloud.databricks.com/login?tuuid=ee23ab7a-a2b9-4512-8973-fc787eb3c01e) to your **Databricks** workspace. |
| 40 | +2. Navigate to **SQL Warehouses** in the sidebar. |
| 41 | +3. Click the warehouse name you want to use. |
| 42 | +4. On the warehouse **details page**, select the **Properties** tab. |
| 43 | +5. Locate and copy the **Warehouse ID**. For example, `bd4dc8ef7e54782c`. |
| 44 | + |
| 45 | +#### Client ID and Client Secret |
| 46 | + |
| 47 | +To generate the Client ID and Client Secret, refer to the [Create an OAuth Secret](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m?language=Environment#-step-1-create-an-oauth-secret) section. |
| 48 | + |
| 49 | +### Source configuration |
| 50 | + |
| 51 | +When you create a Databricks Audit Source, you add it to a Hosted Collector. Before creating the Source, identify the Hosted Collector you want to use or create a new Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector). |
| 52 | + |
| 53 | +To configure Databricks Audit Source: |
| 54 | +1. [**New UI**](/docs/get-started/sumo-logic-ui). In the Sumo Logic main menu select **Data Management**, and then under **Data Collection** select **Collection**. You can also click the **Go To...** menu at the top of the screen and select **Collection**.<br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Collection > Collection**. |
| 55 | +1. On the Collectors page, click **Add Source** next to a Hosted Collector. |
| 56 | +1. Search for and select the **Databricks Audit** icon. |
| 57 | +1. Enter a **Name** to display for the Source in Sumo Logic. The description is optional. |
| 58 | +1. (Optional) For **Source Category**, enter any string to tag the output collected from the Source. Category metadata is stored in a searchable field called `_sourceCategory`. |
| 59 | +1. (Optional) **Fields**. Click the **+Add Field** link to define the fields you want to associate. Each field needs a name (key) and value. |
| 60 | + * <img src={useBaseUrl('img/reuse/green-check-circle.png')} alt="green check circle.png" width="20"/> A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema. |
| 61 | + * <img src={useBaseUrl('img/reuse/orange-exclamation-point.png')} alt="orange exclamation point.png" width="20"/> An orange triangle with an exclamation point is shown when the field doesn't exist in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic that does not exist in the Fields schema it is ignored, known as dropped. |
| 62 | +1. Enter the [Base URL](#base-url) of your account. |
| 63 | +1. Enter the [Warehouse ID](#warehouse-id) collected from the [vendor configuration](#vendor-configuration] to fetch audit logs. |
| 64 | +1. Enter the [Client ID and Client Secret](#client-id-and-client-secret) collected from the [vendor configuration](#vendor-configuration] to authorize access to your Databricks resources. |
| 65 | +1. The **Polling Interval** is set for 5 minutes by default. You can adjust it based on your needs. |
| 66 | +1. When you are finished configuring the Source, click **Save**. |
| 67 | + |
| 68 | +## JSON schema |
| 69 | + |
| 70 | +Sources can be configured using UTF-8 encoded JSON files with the Collector Management API. See [Use JSON to Configure Sources](/docs/send-data/use-json-configure-sources) for details. |
| 71 | + |
| 72 | +| Parameter | Type | Value | Required | Description | |
| 73 | +|:--|:--|:--|:--|:--| |
| 74 | +| schemaRef | JSON Object | `{"type":"Databricks Audit Logs"}` | Yes | Define the specific schema type. | |
| 75 | +| sourceType | String | `"Universal"` | Yes | Type of source. | |
| 76 | +| config | JSON Object | [Configuration object](#configuration-object) | Yes | Source type specific values. | |
| 77 | + |
| 78 | +### Configuration Object |
| 79 | + |
| 80 | +| Parameter | Type | Required | Default | Description | Example | |
| 81 | +|:--|:--|:--|:--|:--|:--| |
| 82 | +| name | String | Yes | `null` | Type a desired name of the source. The name must be unique per Collector. This value is assigned to the [metadata](/docs/search/get-started-with-search/search-basics/built-in-metadata) field `_source`. | `"mySource"` | |
| 83 | +| description | String | No | `null` | Type a description of the source. | `"Testing source"` |
| 84 | +| category | String | No | `null` | Type a category of the source. This value is assigned to the [metadata](/docs/search/get-started-with-search/search-basics/built-in-metadata) field `_sourceCategory`. See [best practices](/docs/send-data/best-practices) for details. | `"mySource/test"` |
| 85 | +| fields | JSON Object | No | `null` | JSON map of key-value fields (metadata) to apply to the Collector or Source. Use the boolean field `_siemForward` to enable forwarding to SIEM.|`{"_siemForward": false, "fieldA": "valueA"}` | |
| 86 | +| baseURL | String | Yes | `null` | Base URL of the Databricks workspace. For example, `https://<workspace-name>.databricks.com`. | | |
| 87 | +| warehouseID | String | Yes | `null` | Unique identifier of the SQL Warehouse within the Databricks workspace, used to query and fetch audit logs. For example, `bd4dc8ef7e54782c`. | | |
| 88 | +| clientID | String | Yes | `null` | Client ID of the account. | | |
| 89 | +| clientSecret | String | Yes | `null` | Client Secret of the account | | |
| 90 | +| pollingIntervalMin | Integer | No | 5 minutes | Time interval after which the source will check for new data. <br/>Minimum: 5 minutes <br/>Maximum: 24 hours | | |
| 91 | + |
| 92 | +### JSON example |
| 93 | + |
| 94 | +```json reference |
| 95 | +https://github.com/SumoLogic/sumologic-documentation/blob/main/static/files/c2c/databricks-audit/example.json |
| 96 | +``` |
| 97 | + |
| 98 | +### Terraform example |
| 99 | + |
| 100 | +```sh reference |
| 101 | +https://github.com/SumoLogic/sumologic-documentation/blob/main/static/files/c2c/databricks-audit/example.tf |
| 102 | +``` |
| 103 | + |
| 104 | + |
| 105 | +## FAQ |
| 106 | + |
| 107 | +:::info |
| 108 | +Click [here](/docs/c2c/info) for more information about Cloud-to-Cloud sources. |
| 109 | +::: |
0 commit comments