Merge remote-tracking branch 'upstream/main' into pivot

Author: SkyFan2002

Cargo.lock

@@ -0,0 +1,113 @@
+---
+title: Prometheus & Grafana
+---
+
+Databend captures real-time metrics of the meta and query services, which you can access through a web browser using the following URLs:
+
+- Meta Metrics: `http://<admin_api_address>/v1/metrics`
+- Query Metrics: `http://<metric_api_address>/metrics`
+
+Alternatively, you can use Prometheus to capture and store the metrics data from Databend. Then, you can visualize the captured time series data on a dashboard with Grafana.
+
+[Prometheus](https://prometheus.io/) is an open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach. [Grafana](https://grafana.com/grafana) is an open-source tool used for analyzing and visualizing metrics.
+
+This following tutorial guides you through deploying and integrating Databend, Prometheus, and Grafana. In this tutorial, you'll deploy a local Databend and install Prometheus and Grafana with Docker. Before you start, ensure that you have Docker installed.
+
+## Tutorial: Monitor Databend with Prometheus & Grafana
+
+### Step 1. Deploy Databend
+
+Follow the [Deployment Guide](https://databend.rs/doc/deploy) to deploy a standalone Databend.
+
+:::tip
+This tutorial uses the [default configuration files](https://github.com/datafuselabs/databend/tree/main/scripts/distribution/configs) in the `configs` folder of the install package. The metrics API for databend-meta is `0.0.0.0:28101/v1/metrics`, and the metrics API for databend-query is `0.0.0.0:7070/metrics`.
+:::
+
+### Step 2. Deploy Prometheus
+
+The steps below describe how to install and deploy Prometheus using Docker.
+
+1. Pull the latest Docker image of Prometheus from the Docker Hub registry.
+
+   ```bash
+   docker pull prom/prometheus
+   ```
+
+2. Edit the configuration file **prometheus.yml**.
+
+   Add the following script to the end of the file prometheus.yml that can be found in the `/etc/prometheus/prometheus.yml` directory. Please note that, with Docker, there are multiple ways to modify a file for a container. In this tutorial, we demonstrate how to achieve this by saving the file to a local folder and mapping it when running the Prometheus image.
+
+   :::tip
+   Docker containers can connect to local services running on the host by using `host.docker.internal`. This feature is available by default only on Docker for Windows/Mac. However, it is also available on Linux starting from version **20.03**.
+   :::
+
+   ```yaml
+   - job_name: "databend-query"
+
+     # metrics_path defaults to '/metrics'
+     # scheme defaults to 'http'.
+
+     static_configs:
+       - targets: ["host.docker.internal:7070"]
+
+   - job_name: "databend-meta"
+
+     metrics_path: "/v1/metrics"
+     # scheme defaults to 'http'.
+
+     static_configs:
+       - targets: ["host.docker.internal:28101"]
+   ```
+
+3. Deploy Prometheus.
+
+   If you saved and edited the file `prometheus.yml` in a local folder, you need to create a mapping using the `-v` option in the command. To do so, replace `/path/to/prometheus.yml` in the command below with the path to your local `prometheus.yml`.
+
+   ```bash
+   docker run \
+   -p 9090:9090 \
+   --add-host=host.docker.internal:host-gateway \
+   -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
+   prom/prometheus
+   ```
+
+4. Check Metrics Status
+
+   Check the value on the right of each instance. `1` means the instance is healthy, and `0` means that the scrape failed.
+
+   ![Prometheus up](../../public/img/tracing/prometheus-up.png)
+
+### Step 3. Deploy Grafana
+
+The steps below describe how to install and deploy Grafana using Docker.
+
+1. Pull the latest Docker image of Grafana from the Docker Hub registry.
+
+   ```bash
+   docker pull grafana/grafana
+   ```
+
+2. Deploy Grafana.
+
+   ```bash
+   docker run \
+   -p 3000:3000 \
+   --add-host=host.docker.internal:host-gateway \
+   grafana/grafana
+   ```
+
+3. Add a data source of Prometheus type.
+
+   Open your web browser and go to `http://0.0.0.0:3000`. Log in with the user name `admin` and password `admin` first, and then add a data source of Prometheus type on **Configuration** > **Data Sources** > **Add data source**.
+
+   Please note that set the URL to `http://host.docker.internal:9090` for the data source.
+
+   ![Grafana data source](../../public/img/tracing/grafana-datasource.png)
+
+4. Create dashboards.
+
+   Databend recommend import the files in [datafuselabs/helm-charts - dashboards](https://github.com/datafuselabs/helm-charts/tree/main/dashboards) to create your dashboards. To do so, download the files first, then go to `http://0.0.0.0:3000/dashboard/import` to import the downloaded files one by one and select the `Prometheus` data source for each dashboard.
+
+   ![Grafana import query json](../../public/img/tracing/grafana-query-json.png)
+
+   ![Grafana query dashboard](../../public/img/tracing/grafana-query-dashboard.png)

docs/doc/13-monitor/prometheus-and-granafa.md

@@ -0,0 +1,34 @@
+---
+title: LIST
+---
+
+Aggregate function.
+
+The LIST() function converts all the values of a column to an Array.
+
+## Syntax
+
+```sql
+LIST(expression)
+```
+
+## Arguments
+
+| Arguments   | Description    |
+| ----------- | -------------- |
+| expression  | Any expression |
+
+## Return Type
+
+the Array type that use the type of the value as inner type.
+
+## Examples
+
+```sql
+SELECT LIST(number) FROM numbers(10);
++-----------------------+
+| list(number)          |
++-----------------------+
+| [0,1,2,3,4,5,6,7,8,9] |
++-----------------------+
+```

docs/doc/15-sql-functions/10-aggregate-functions/aggregate-list.md

@@ -30,4 +30,5 @@ These functions help you extract and summarize data from databases to gain valua
 | [QUANTILE](aggregate-quantile.md)                           | Calculates the quantile for a specific column               | 
 | [RETENTION](aggregate-retention.md)                         | Calculates retention for a set of events                    | 
 | [WINDOW_FUNNEL](aggregate-windowfunnel.md)                  | Analyzes user behavior in a time-ordered sequence of events | 
+| [LIST](aggregate-list.md)                                   | Converts all the values of a column to an Array             |
 

docs/doc/15-sql-functions/10-aggregate-functions/index.md

@@ -14,9 +14,17 @@ The SQL query statements generated adhere to the PostgreSQL standards, so they m
 
 ```sql
 USE <your-database>;
-SELECT * FROM ai_to_sql('<natural-language-instruction>', '<openai-api-key>');
+SELECT * FROM ai_to_sql('<natural-language-instruction>'[, '<openai-api-key>']);
 ```
-To obtain your openAI API key, please visit https://platform.openai.com/account/api-keys and generate a new key.
+
+:::tip Obtain and Save openAI API Key
+- To obtain your openAI API key, please visit https://platform.openai.com/account/api-keys and generate a new key.
+- Instead of manually entering your OpenAI API secret key every time you run the function, you can save time by adding it to the configuration file **databend-query.toml** through the `openai_api_key` setting. Databend will automatically use that secret key whenever you run the function without explicitly specifying a key as a parameter.
+
+```toml
+openai_api_key = "<your-key>"
+```
+:::
 
 ## Examples
 

docs/doc/15-sql-functions/61-ai-functions/01-ai-to-sql.md

@@ -692,6 +692,7 @@ impl<'ast> Visitor<'ast> for AstFormatVisitor {
             ExplainKind::Raw => "Raw",
             ExplainKind::Plan => "Plan",
             ExplainKind::Memo(_) => "Memo",
+            ExplainKind::JOIN => "JOIN",
             ExplainKind::AnalyzePlan => "Analyze",
         });
         let format_ctx = AstFormatContext::with_children(name, 1);