diff --git a/docs/pages/product/apis-integrations/ai-api.mdx b/docs/pages/product/apis-integrations/ai-api.mdx index 8af4124616c4d..91783b86ad7cc 100644 --- a/docs/pages/product/apis-integrations/ai-api.mdx +++ b/docs/pages/product/apis-integrations/ai-api.mdx @@ -138,3 +138,83 @@ One way of handling this is to pass the error message back into the AI API; it m #### 3. Continue wait When using `"runQuery": true`, you might sometimes receive a query result containing `{ "error": "Continue wait" }`. If this happens, you should use `/load` ([described above](#2-load)) instead of `runQuery` to run the query, and handle retries as described in the [REST API documentation](/product/apis-integrations/rest-api#continue-wait). + +## Advanced Usage + + + The advanced features discussed here are available on Cube version 1.1.7 and above. + + +### Custom prompts + +You can prompt the AI API with custom instructions. For example, you may want it to always +respond in a particular language, or to refer to itself by a name matching your brand. +Custom prompts also allow you to give the model more context on your company and data model, +for example if it should usually prefer a particular view. + +To use a custom prompt, set the `CUBE_CLOUD_AI_API_PROMPT` environment variable in your deployment. + + + Custom prompts add to, rather than overwrite, the AI API's existing prompting, so you + do not need to re-write instructions around how to generate the query itself. + + +### Meta tags + +The AI API can read [meta tags](/reference/data-model/view#meta) on your dimensions, measures, +segments, and views. + +Use the `ai` meta tag to give context that is specific to AI and goes beyond what is +included in the description. This can have any keys that you want. For example, you can use it +to give the AI context on possible values in a categorical dimension: +```yaml + - name: status + sql: status + type: string + meta: + ai: + values: + - shipped + - processing + - completed +``` + +### Other LLM providers + + + These environment variables also apply to the [AI Assistant](/product/workspace/ai-assistant), + if it is enabled on your deployment. + + +If desired, you may "bring your own" LLM model by providing a model and API credentials +for a supported model provider. Do this by setting environment variables in your Cube +deployment. See below for required variables by provider (required unless noted): + +#### AWS Bedrock + + + The AI API currently supports only Anthropic Claude models on AWS Bedrock. Other + models may work but are not fully supported. + + +- `CUBE_BEDROCK_MODEL_ID` - A supported [AWS Bedrock chat model](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), for example `anthropic.claude-3-5-sonnet-20241022-v2:0` +- `CUBE_BEDROCK_ACCESS_KEY` - An access key for an IAM user with `InvokeModelWithResponseStream` permissions on the desired region/model. +- `CUBE_BEDROCK_ACCESS_SECRET` - The corresponding access secret +- `CUBE_BEDROCK_REGION_ID` - A supported AWS Bedrock region, for example `us-west-2` + +#### GCP Vertex + + + The AI API currently supports only Anthropic Claude models on GCP Vertex. Other + models may work but are not fully supported. + + +- `CUBE_VERTEX_MODEL_ID` - A supported GCP Vertex chat model, for example `claude-3-5-sonnet@20240620` +- `CUBE_VERTEX_PROJECT_ID` - The GCP project the model is deployed in +- `CUBE_VERTEX_REGION` - The GCP region the model is deployed in, for example `us-east5` +- `CUBE_VERTEX_CREDENTIALS` - The private key for a service account with permissions to run the chosen model + +#### OpenAI + +- `OPENAI_MODEL` - An OpenAI chat model ID, for example `gpt-4o` +- `OPENAI_API_KEY` - An OpenAI API key (we recommend creating a service account for the AI API) \ No newline at end of file diff --git a/docs/pages/product/workspace/ai-assistant.mdx b/docs/pages/product/workspace/ai-assistant.mdx index 17979cb677ba8..c673364952c2a 100644 --- a/docs/pages/product/workspace/ai-assistant.mdx +++ b/docs/pages/product/workspace/ai-assistant.mdx @@ -58,6 +58,49 @@ The query will automatically run in the sidebar and can be opened in the [Playgr +## Advanced Usage + + + The advanced features discussed here are available on Cube version 1.1.7 and above. + + +### Custom prompts + +You can prompt the AI Assistant with custom instructions. For example, you may want it to always +respond in a particular language, or to refer to itself by a name matching your brand. +Custom prompts also allow you to give the model more context on your company and data model, +for example if it should usually prefer a particular view. + +To use a custom prompt, set the `CUBE_CLOUD_AI_ASSISTANT_PROMPT` environment variable in your deployment. + + + Custom prompts add to, rather than overwrite, the AI Assistant's existing prompting. + + +### Meta tags + +The AI Assistant can read [meta tags](/reference/data-model/view#meta) on your dimensions, measures, +segments, and views. + +Use the `ai` meta tag to give context that is specific to AI and goes beyond what is +included in the description. This can have any keys that you want. For example, you can use it +to give the AI context on possible values in a categorical dimension: +```yaml + - name: status + sql: status + type: string + meta: + ai: + values: + - shipped + - processing + - completed +``` + +### Other LLM providers + +See the [AI API's documentation][ref-ai-api-providers] for information on how to "bring your own" LLM. + ## FAQ and limitations ### 1. What language model(s) does the AI Assistant use? @@ -83,3 +126,4 @@ The query will automatically run in the sidebar and can be opened in the [Playgr [ref-catalog]: /product/workspace/semantic-catalog [ref-playground]: /product/workspace/playground [ref-catalog-downstream]: /product/workspace/semantic-catalog#connecting-downstream-tools +[ref-ai-api-providers]: /product/apis-integrations/ai-api#other-llm-providers diff --git a/docs/pages/reference/ai-api.mdx b/docs/pages/reference/ai-api.mdx index 251d5bb7b9596..776092566e00f 100644 --- a/docs/pages/reference/ai-api.mdx +++ b/docs/pages/reference/ai-api.mdx @@ -11,11 +11,21 @@ Generate a Cube query that can be used to answer a user's question, and (optiona | `messages` | ✅ Yes | An array of messages in the format: `{ "role": "user" \| "assistant", "content": "string" }` | | `views` | | An array of view names (used to limit the views that the AI API can use to generate its answer) | | `runQuery` | | Boolean (true or false) whether to run the query and return its results | +| `options` | | An object in the format `{ "chart": true \| false }` Response - `message` - A message from the AI assistant describing the query, how it was chosen, why it could not generate the requested query, etc. - `cube_query` - A Cube [Query](/product/apis-integrations/rest-api/query-format) that could be used to answer the given question +- `chart` - If the `chart` option is set to `true`, an object containing a chart spec for the generated query in the following format: + ```json + { + "type": "bar" | "line" | "pie" | "table" | "area" | "scatter", + "x": string, + "y": string[], + "pivot": string // optional; the field to pivot by, if any + } + ``` ### Examples @@ -28,7 +38,7 @@ curl \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: EXAMPLE-API-TOKEN" \ - --data '{ "messages": [{ "role": "user", "content": "What cities have the highest aov this year?" }]}' \ + --data '{ "messages": [{ "role": "user", "content": "What cities have the highest aov this year?", "views": ["orders_view"] }]}' \ https://YOUR_CUBE_API/cubejs-api/v1/ai/query/completions ```