docs: add more caution for the OpenAI api (#10901)

* docs: add more caution for the OpenAI apii

* fix link

* fix link

Author: BohuTANG

docs/doc/01-guides/index.md

@@ -86,8 +86,8 @@ These tutorials are intended to help you get started with Databend:
 
 * [Generating SQL with AI](../15-sql-functions/61-ai-functions/01-ai-to-sql.md)
 * [Creating Embedding Vectors](../15-sql-functions/61-ai-functions/02-ai-embedding-vector.md)
-* [Computing Text Similarities](../15-sql-functions/61-ai-functions/03-ai-cosine-distance.md)
-* [Text Completion with AI](../15-sql-functions/61-ai-functions/04-ai-text-completion.md)
+* [Text Completion with AI](../15-sql-functions/61-ai-functions/03-ai-text-completion.md)
+* [Computing Text Similarities](../15-sql-functions/61-ai-functions/04-ai-cosine-distance.md)
 
 ## Backup & Restore
 

docs/doc/15-sql-functions/61-ai-functions/01-ai-to-sql.md

@@ -2,14 +2,24 @@
 title: 'AI_TO_SQL'
 ---
 
-Converts natural language instructions into SQL queries with the latest [Codex](https://platform.openai.com/docs/models/codex) model `code-davinci-002`.
+Converts natural language instructions into SQL queries with the latest model `text-davinci-003`.
 
 Databend offers an efficient solution for constructing SQL queries by incorporating OLAP and AI. Through this function, instructions written in a natural language can be converted into SQL query statements that align with the table schema. For example, the function can be provided with a sentence like "Get all items that cost 10 dollars or less" as an input and generate the corresponding SQL query "SELECT * FROM items WHERE price <= 10" as output.
 
+The main code implementation can be found [here](https://github.com/datafuselabs/databend/blob/1e93c5b562bd159ecb0f336bb88fd1b7f9dc4a62/src/query/service/src/table_functions/openai/ai_to_sql.rs).
+
 :::note
 The SQL query statements generated adhere to the PostgreSQL standards, so they might require manual revisions to align with the syntax of Databend.
 :::
 
+:::caution
+Databend relies on OpenAI for `AI_TO_SQL` but only sends the table schema to OpenAI, not the data.
+
+They will only work when the Databend configuration includes the `openai_api_key`, otherwise they will be inactive.
+
+This function is available by default on [Databend Cloud](https://databend.com) using our self OpenAI key. If you use them, you acknowledge that your table schema will be sent to OpenAI by us.
+:::
+
 ## Syntax
 
 ```sql

docs/doc/15-sql-functions/61-ai-functions/02-ai-embedding-vector.md

@@ -5,6 +5,16 @@ description: 'Creating embeddings using the ai_embedding_vector function in Data
 
 This document provides an overview of the ai_embedding_vector function in Databend and demonstrates how to create document embeddings using this function.
 
+The main code implementation can be found [here](https://github.com/datafuselabs/databend/blob/1e93c5b562bd159ecb0f336bb88fd1b7f9dc4a62/src/common/openai/src/embedding.rs).
+
+:::caution
+Databend relies on OpenAI for `AI_EMBEDDING_VECTOR` and sends the embedding column data to OpenAI.
+
+They will only work when the Databend configuration includes the `openai_api_key`, otherwise they will be inactive.
+
+This function is available by default on [Databend Cloud](https://databend.com) using our self OpenAI key. If you use them, you acknowledge that your table schema will be sent to OpenAI by us.
+:::
+
 ## Overview of ai_embedding_vector
 
 

…61-ai-functions/04-ai-text-completion.md …61-ai-functions/03-ai-text-completion.mddocs/doc/15-sql-functions/61-ai-functions/04-ai-text-completion.md renamed to docs/doc/15-sql-functions/61-ai-functions/03-ai-text-completion.md docs/doc/15-sql-functions/61-ai-functions/04-ai-text-completion.md renamed to docs/doc/15-sql-functions/61-ai-functions/03-ai-text-completion.md

@@ -3,7 +3,18 @@ title: 'AI_TEXT_COMPLETION'
 description: 'Generating text completions using the ai_text_completion function in Databend'
 ---
 
-This document provides an overview of the ai_text_completion function in Databend and demonstrates how to generate text completions using this function.
+This document provides an overview of the `ai_text_completion` function in Databend and demonstrates how to generate text completions using this function.
+
+The main code implementation can be found [here](https://github.com/datafuselabs/databend/blob/1e93c5b562bd159ecb0f336bb88fd1b7f9dc4a62/src/common/openai/src/completion.rs).
+
+:::caution
+Databend relies on OpenAI for `AI_TEXT_COMPLETION` and sends the completion prompt data to OpenAI.
+
+They will only work when the Databend configuration includes the `openai_api_key`, otherwise they will be inactive.
+
+This function is available by default on [Databend Cloud](https://databend.com) using our self OpenAI key. If you use them, you acknowledge that your table schema will be sent to OpenAI by us.
+:::
+
 
 ## Overview of ai_text_completion
 
@@ -27,4 +38,4 @@ Result:
 +--------------------------------------------------------------------------------------------------------------------+
 ```
 
-In this example, we provide the prompt "What is artificial intelligence?" to the ai_text_completion function, and it returns a generated completion that briefly describes artificial intelligence.
+In this example, we provide the prompt "What is artificial intelligence?" to the `ai_text_completion` function, and it returns a generated completion that briefly describes artificial intelligence.

…61-ai-functions/03-ai-cosine-distance.md …61-ai-functions/04-ai-cosine-distance.mddocs/doc/15-sql-functions/61-ai-functions/03-ai-cosine-distance.md renamed to docs/doc/15-sql-functions/61-ai-functions/04-ai-cosine-distance.md docs/doc/15-sql-functions/61-ai-functions/03-ai-cosine-distance.md renamed to docs/doc/15-sql-functions/61-ai-functions/04-ai-cosine-distance.md

@@ -5,6 +5,10 @@ description: 'Measuring document similarity using the cosine_distance function i
 
 This document provides an overview of the `cosine_distance` function in Databend and demonstrates how to measure document similarity using this function.
 
+:::info
+The `cosine_distance` function performs vector computations within Databend and does not rely on the OpenAI API.
+:::
+
 ## Overview of cosine_distance
 
 The `cosine_distance` function in Databend is a built-in function that calculates the cosine distance between two vectors. It is commonly used in natural language processing tasks, such as document similarity and recommendation systems.

docs/doc/15-sql-functions/61-ai-functions/index.md

@@ -1,34 +1,66 @@
 ---
 title: 'AI Functions'
-description: 'SQL-based Knowledge Base Search and Completion using Databend'
+description: 'Using SQL-based AI Functions for Knowledge Base Search and Text Completion'
 ---
 
-This document demonstrates how to leverage Databend's built-in AI functions for creating document embeddings, searching for similar documents, and generating text completions based on context. 
+This document demonstrates how to leverage Databend's built-in AI functions for creating document embeddings, searching for similar documents, and generating text completions based on context. We will guide you through a simple example that shows how to create and store embeddings, find related documents, and generate completions using various AI functions.
 
-We will guide you through a simple example that shows how to create and store embeddings using the `ai_embedding_vector` function, find related documents with the `cosine_distance` function, and generate completions using the `ai_text_completion` function.
+:::caution
+
+Databend relies on OpenAI for embeddings and text completions, which means your data will be sent to OpenAI. Exercise caution when using these functions.
+
+They will only work when the Databend configuration includes the `openai_api_key`, otherwise they will be inactive.
+
+These functions are available by default on [Databend Cloud](https://databend.com) using our self OpenAI key. If you use them, you acknowledge that your data will be sent to OpenAI by us.
+
+:::
 
 ## Introduction to embeddings
 
-Embeddings are vector representations of text data that capture the semantic meaning and context of the original text. They can be used to compare and analyze text in various natural language processing tasks, such as document similarity, clustering, and recommendation.
+Embeddings are vector representations of text data that capture the semantic meaning and context of the original text. They can be used to compare and analyze text in various natural language processing tasks, such as document similarity, clustering, and recommendation systems.
 
 ## How do embeddings work?
 
-Embeddings work by converting text into high-dimensional vectors in such a way that similar texts are closer together in the vector space. This is achieved by training a model on a large corpus of text, which learns to represent the words and phrases in a continuous space that captures their semantic relationships.
-Embeddings are vector representations of text data that capture the semantic meaning and context of the original text. They are widely used in various natural language processing tasks, such as document similarity, clustering, and recommendation systems.
+Embeddings work by converting text into high-dimensional vectors in such a way that similar texts are closer together in the vector space.
+
+This is achieved by training a model on a large corpus of text, which learns to represent the words and phrases in a continuous space that captures their semantic relationships.
+
+To illustrate how embeddings work, let's consider a simple example. Suppose we have the following sentences:
+1. `"The cat sat on the mat."`
+2. `"The dog sat on the rug."`
+3. `"The quick brown fox jumped over the lazy dog."`
+
+When creating embeddings for these sentences, the model will convert the text into high-dimensional vectors in such a way that similar sentences are closer together in the vector space.
+
+For instance, the embeddings of sentences 1 and 2 will be closer to each other because they share a similar structure and meaning (both involve an animal sitting on something). On the other hand, the embedding of sentence 3 will be farther from the embeddings of sentences 1 and 2 because it has a different structure and meaning.
+
+The embeddings could look like this (simplified for illustration purposes):
+
+1. `[0.2, 0.3, 0.1, 0.7, 0.4]`
+2. `[0.25, 0.29, 0.11, 0.71, 0.38]`
+3. `[-0.1, 0.5, 0.6, -0.3, 0.8]`
+
+In this simplified example, you can see that the embeddings of sentences 1 and 2 are closer to each other in the vector space, while the embedding of sentence 3 is farther away. This illustrates how embeddings can capture semantic relationships and be used to compare and analyze text data.
+
+
+## What is a Vector Database?
+
+A vector database is a specialized database designed to store, manage, and search high-dimensional vector data efficiently. These databases are optimized for similarity search operations, such as finding the nearest neighbors of a given vector. They are particularly useful in scenarios where the data has high dimensionality, like embeddings in natural language processing tasks, image feature vectors, and more.
+
+Typically, embedding vectors are stored in specialized vector databases like milvus, pinecone, qdrant, or weaviate. Databend can also store embedding vectors using the ARRAY(FLOAT32) data type and perform similarity computations with the cosine_distance function in SQL. To create embeddings for a text document using Databend, you can use the built-in ai_embedding_vector function directly in your SQL query.
 
 ## Databend AI Functions
 
 Databend provides built-in AI functions for various natural language processing tasks. The main functions covered in this document are:
 
 - [ai_embedding_vector](./02-ai-embedding-vector.md): Generates embeddings for text documents.
-- [cosine_distance](./03-ai-cosine-distance.md): Calculates the cosine distance between two embeddings.
-- [ai_text_completion](./04-ai-text-completion.md): Generates text completions based on a given prompt.
-These functions are powered by open-source natural language processing models and can be used directly within SQL queries.
+- [ai_text_completion](./03-ai-text-completion.md): Generates text completions based on a given prompt.
+- [cosine_distance](./04-ai-cosine-distance.md): Calculates the cosine distance between two embeddings.
 
 ## Creating and storing embeddings using Databend
 
-To create embeddings for a text document using Databend, you can use the built-in ai_embedding_vector function directly in your SQL query. Here's an example:
 
+Here's an example:
 ```sql
 CREATE TABLE documents (
     doc_id INT,
@@ -52,11 +84,11 @@ SELECT doc_id, text_content, ai_embedding_vector(text_content)
 FROM documents;
 ```
 
-This SQL script creates a documents table, inserts the example documents, and then generates embeddings using the ai_embedding_vector function. The embeddings are stored in the embeddings table with the ARRAY(FLOAT32) column type.
+This SQL script creates a `documents` table, inserts the example documents, and then generates embeddings using the `ai_embedding_vector` function. The embeddings are stored in the embeddings table with the `ARRAY(FLOAT32)` column type.
 
-## Searching for related documents using cosine distance
+## Searching for similarity documents using cosine distance
 
-Suppose you have a question, "What is a subfield of artificial intelligence?", and you want to find the most related document from the stored embeddings. First, generate an embedding for the question using the ai_embedding_vector function:
+Suppose you have a question, "What is a subfield of artificial intelligence?", and you want to find the most related document from the stored embeddings. First, generate an embedding for the question using the `ai_embedding_vector` function:
 ```sql
 SELECT doc_id, text_content, cosine_distance(embedding, ai_embedding_vector('What is a subfield of artificial intelligence?')) AS distance
 FROM embeddings