You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Grafana](https://grafana.com/) is a monitoring dashboard system, which is an open-source monitoring tool developed by Grafana Labs. It can greatly simplify the complexity of monitoring by allowing us to provide the data to be monitored, and it generates various visualizations. Additionally, it has an alarm function that sends notifications when there is an issue with the system. Databend and Databend Cloud can integrate with Grafana through the [Grafana Databend Data Source Plugin](https://github.com/databendlabs/grafana-databend-datasource).
6
+
[Grafana](https://grafana.com/) is a monitoring dashboard system, which is an open-source monitoring tool developed by Grafana Labs. It can greatly simplify the complexity of monitoring by allowing us to provide the data to be monitored, and it generates various visualizations. Additionally, it has an alarm function that sends notifications when there is an issue with the system.
7
7
8
-
## Tutorial: Integrating with Grafana
8
+
Databend Cloud and Databend can integrate with Grafana in two ways:
9
9
10
-
This tutorial guides you through the process of integrating Databend / Databend Cloud with Grafana using the Grafana Databend Data Source Plugin.
10
+
-**Loki Protocol (Recommended for Databend Cloud)**: Use Grafana's built-in Loki data source to connect to Databend Cloud via Loki-compatible API endpoints.
11
+
-**Custom Plugin**: Use the [Grafana Databend Data Source Plugin](https://github.com/databendlabs/grafana-databend-datasource) for direct SQL access.
11
12
12
-
### Step 1. Set up Environment
13
+
## Using Loki Protocol (Recommended)
14
+
15
+
Databend Cloud provides a Loki-compatible API that allows you to use Grafana's native Loki data source without installing additional plugins. This is the recommended approach for most use cases.
16
+
17
+
:::note
18
+
The Loki protocol feature requires activation. Please contact support to enable this feature for your account.
19
+
:::
20
+
21
+
### Step 1. Configure Table
22
+
23
+
Before connecting to Grafana, configure your Databend Cloud table for log data visualization. Below are two recommended schema types:
24
+
25
+
#### Loki Schema
26
+
27
+
This schema stores labels as a VARIANT/MAP alongside the log body:
28
+
29
+
```sql
30
+
CREATETABLElogs (
31
+
`timestamp`TIMESTAMPNOT NULL,
32
+
`labels` VARIANT NOT NULL,
33
+
`line` STRING NOT NULL,
34
+
`stream_hash` UInt64 NOT NULLAS (city64withseed(labels, 0)) STORED
35
+
) CLUSTER BY (to_start_of_hour(timestamp), stream_hash);
36
+
37
+
CREATE INVERTED INDEX logs_line_idx ON logs(line);
38
+
REFRESH INVERTED INDEX logs_line_idx ON logs;
39
+
```
40
+
41
+
-`timestamp`: log event timestamp
42
+
-`labels`: VARIANT storing serialized Loki labels
43
+
-`line`: raw log line
44
+
-`stream_hash`: computed hash for clustering
45
+
46
+
#### Flat Schema
47
+
48
+
This schema uses a wide table where each attribute is a separate column:
49
+
50
+
```sql
51
+
CREATETABLEnginx_logs (
52
+
`agent` STRING,
53
+
`client` STRING,
54
+
`host` STRING,
55
+
`path` STRING,
56
+
`request` STRING,
57
+
`status`INT,
58
+
`timestamp`TIMESTAMPNOT NULL
59
+
) CLUSTER BY (to_start_of_hour(timestamp), host, status);
60
+
61
+
CREATE INVERTED INDEX nginx_request_idx ON nginx_logs(request);
62
+
REFRESH INVERTED INDEX nginx_request_idx ON nginx_logs;
63
+
```
64
+
65
+
Every column except the timestamp and line column becomes a LogQL label.
For detailed information on obtaining connection details, see [Connecting to a Warehouse](/guides/cloud/resources/warehouses#connecting).
83
+
84
+
### Step 3. Configure Grafana Data Source
13
85
14
-
Before you start, please refer to the official installation guide to install Grafana: [https://grafana.com/docs/grafana/latest/setup-grafana/installation](https://grafana.com/docs/grafana/latest/setup-grafana/installation).
86
+
1. In Grafana, navigate to **Connections** > **Data sources** > **Add data source**.
15
87
16
-
For this tutorial, you can integrate either with Databend or Databend Cloud:
88
+
2. Search for and select **Loki**.
17
89
18
-
- If you choose to integrate with a local Databend instance, follow the [Deployment Guide](/guides/self-hosted) to deploy it if you don't have one already.
19
-
- If you prefer to integrate with Databend Cloud, make sure you can log in to your account and obtain the connection information for a warehouse. For more details, see [Connecting to a Warehouse](/guides/cloud/resources/warehouses#connecting).
90
+
3. Configure the basic settings:
91
+
-**Name**: Give your data source a descriptive name (e.g., "Databend Cloud Logs")
92
+
-**URL**: Enter `https://<host>` using the host from Step 2
93
+
94
+
95
+
96
+
4. Configure authentication:
97
+
- Enable **Basic auth** under the Authentication section
98
+
-**User**: Enter your username (typically `cloudapp`)
99
+
-**Password**: Enter your password or API key
100
+
101
+
5. Add custom HTTP headers. Under **Custom HTTP Headers**, add the following:
102
+
-**Header**: `X-Databend-Warehouse`, **Value**: Your warehouse name
103
+
-**Header**: `X-Databend-Database`, **Value**: Your database name
104
+
-**Header**: `X-Databend-Table`, **Value**: Your table name
105
+
106
+
107
+
108
+
6. Click **Save & test** to verify the connection.
109
+
110
+
111
+
112
+
### Step 4. Test Queries
113
+
114
+
1. Navigate to **Explore** in Grafana.
115
+
116
+
2. Select your Databend Cloud Loki data source.
117
+
118
+
3. Use LogQL queries to visualize your data. For example:
119
+
-`{service="api"}` - Filter logs by service label
120
+
-`{level="error"}` - Show only error-level logs
121
+
-`{service="api"} |= "timeout"` - Search for specific text in logs
122
+
-`count_over_time({status="500"}[5m])` - Count errors over time
123
+
124
+
4. Customize the visualization as needed using Grafana's panel options.
125
+
126
+
127
+
128
+
## Using Custom Plugin (Alternative)
129
+
130
+
For advanced use cases requiring direct SQL access or when working with self-hosted Databend, you can use the Grafana Databend Data Source Plugin.
131
+
132
+
### Step 1. Set up Environment
133
+
134
+
Before you start, ensure you have:
135
+
136
+
- Grafana installed. Refer to the official installation guide: [https://grafana.com/docs/grafana/latest/setup-grafana/installation](https://grafana.com/docs/grafana/latest/setup-grafana/installation)
137
+
- Either:
138
+
- A local Databend instance (follow the [Deployment Guide](/guides/self-hosted) to deploy)
139
+
- Or Databend Cloud access with connection information for a warehouse (see [Connecting to a Warehouse](/guides/cloud/resources/warehouses#connecting))
2. Get the download URL for the plugin zip package, for example, `https://github.com/databendlabs/grafana-databend-datasource/releases/download/v1.0.2/databend-datasource-1.0.2.zip`.
35
155
36
-
3. Get the Grafana plugins folder and unzip the downloaded zip package into it.
156
+
3. Get the Grafana plugins folder and unzip the downloaded zip package into it:
1.Goto the `Add new connection` page, for example, `http://localhost:3000/connections/add-new-connection?search=databend`, search for `databend`, and select it.
173
+
1.Go to the `Add new connection` page, for example, `http://localhost:3000/connections/add-new-connection?search=databend`, search for `databend`, and select it.
54
174
55
175
2. Click **Add new data source** on the top right corner of the page.
56
176
57
-
3. Input the `DSN` field for your Databend instance. For example, `databend://root:@localhost:8000?sslmode=disable`, or `databend://cloudapp:******@tnxxxxxxx.gw.aws-us-east-2.default.databend.com:443/default?warehouse=xsmall-fsta`.
177
+
3. Input the `DSN` field for your Databend instance. For example:
0 commit comments