Add "Switch to Agent Builder" doc for AI assistants (#4438)

Addresses #4204, and partly addresses
[#522](/elastic/docs-content-internal/issues/522) by creating a new page
in the explore-analyze AI section that describes the new AI Agent chat
experience, and informs users about how to opt in to it in 9.3, and the
feature differences.

I put the new page in a new subsection in the AI section we've been
working on, in a new folder called ai-chat-experiences.

## Generative AI disclosure
1. Did you use a generative AI (GenAI) tool to assist in creating this
contribution?
- [ ] Yes  
- [ x] No 

## Live preview:
https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/4438/explore-analyze/ai-features/ai-chat-experiences/ai-agent-or-ai-assistant

---------

Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
Co-authored-by: Liam Thompson <leemthompo@gmail.com>

Author: 4 people

deploy-manage/reference-architectures/hotfrozen-high-availability.md

@@ -27,7 +27,7 @@ This Hot/Frozen – High Availability architecture is intended for organizations
 * Have a requirement for cost effective long term data storage (many months or years).
 * Provide insights and alerts using logs, metrics, traces, or various event types to ensure optimal performance and quick issue resolution for applications.
 * Apply [machine learning anomaly detection](/explore-analyze/machine-learning/anomaly-detection.md) to help detect patterns in time series data to find root cause and resolve problems faster.
-* Use an AI assistant ([Observability](/explore-analyze/ai-features/ai-assistant.md), [Security](/solutions/security/ai/ai-assistant.md), or [Playground](/solutions/search/rag/playground.md)) for investigation, incident response, reporting, query generation, or query conversion from other languages using natural language.
+* Use an AI assistant ([Observability](/explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md), [Security](/solutions/security/ai/ai-assistant.md), or [Playground](/solutions/search/rag/playground.md)) for investigation, incident response, reporting, query generation, or query conversion from other languages using natural language.
 * Deploy an architecture model that allows for maximum flexibility between storage cost and performance.
 
 ::::{important}

explore-analyze/ai-features/ai-agent-or-ai-assistant.md

@@ -0,0 +1,21 @@
+---
+applies_to:
+  stack: preview 9.3+
+  serverless: preview
+products:
+  - id: kibana
+  - id: observability
+  - id: security
+  - id: cloud-serverless
+---
+
+# AI chat experiences
+
+AI chat can help with a wide range of tasks throughout {{kib}}, such as helping to answer questions about your data, suggesting next steps, and explaining how to use Elastic's software.
+
+There are two AI chat experiences which you can choose from:
+
+- {applies_to}`stack: preview 9.3+` {applies_to}`serverless: preview` **AI Agent**: Powered by [{{agent-builder}}](/explore-analyze/ai-features/elastic-agent-builder.md), this chat experience lets you create custom agents for specific tasks, with specialized tools, and provides a unified chat experience across all of {{kib}}.
+- [AI Assistant](/explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md): Elastic's original AI chat, AI Assistant offers features like chat sharing, data anonymization, and knowledge base that are not yet present in Agent Builder. The AI Assistant for {{observability}} and {{es}} operates independently from the AI Assistant for Security.
+
+To compare them and learn how to switch between them, refer to [](/explore-analyze/ai-features/ai-chat-experiences/ai-agent-or-ai-assistant.md). 

explore-analyze/ai-features/ai-chat-experiences.md

@@ -0,0 +1,101 @@
+---
+description: Learn about the differences between the AI Assistant and Agent Builder chat experiences and how to switch between the two.
+applies_to:
+  stack: preview 9.3+
+  serverless: preview
+products:
+  - id: kibana
+  - id: observability
+  - id: security
+  - id: cloud-serverless
+---
+
+# Compare Agent Builder and AI Assistant
+
+::::{admonition} Requirements
+- {{stack}} users: an **Enterprise [subscription](/deploy-manage/license.md)**.
+- {{sec-serverless}} users: the **Security Analytics Complete** or **Elastic AI Soc Engine (EASE)** feature tier.
+- {{obs-serverless}} and {{es-serverless}} users: the **Complete** feature tier. 
+::::
+
+[{{agent-builder}}](/explore-analyze/ai-features/elastic-agent-builder.md) is a powerful and flexible platform for building AI agents, tools, and workflows. Agent Builder comes with built-in agents and [tools](/explore-analyze/ai-features/agent-builder/tools.md) for common use cases, and lets you create custom agents and tools for your specific needs. Eventually, it will power the default chat experience for all solutions and replace AI Assistant.
+
+Agent Builder powers the AI Agent chat experience. Currently, AI Agent is available by default for {{es}} solution users, and as an opt-in feature for Security solution and {{observability}} solution users. When you opt in, it replaces the AI Assistant chat experience. You can opt in to Agent Builder and switch back to AI Assistant at any time.
+
+While Agent Builder offers expanded functionality compared to AI Assistant, it does not yet support a number of AI Assistant features. If you rely on AI Assistant capabilities today, consider the differences between each experience before opting in.
+
+Use this page to learn about:
+
+- [How to switch between the AI Assistant and Agent Builder chat experiences](#switch-between-chat-experiences)
+- [Feature differences between AI Assistant and Agent Builder](#feature-differences-between-agent-builder-and-ai-assistant)
+
+## Switch between chat experiences
+
+:::{important}
+Agent Builder cannot access your chats, prompts, or knowledge base entries from AI Assistant. However, this data remains accessible if you switch back to the AI Assistant chat experience.
+:::
+
+{{kib}} will prompt you to switch to the Agent Builder chat experience if your deployment supports it and you have sufficient privileges. You can opt-in from this prompt immediately.
+
+You can also switch chat experiences at any time:
+
+1. Use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md) to find [**GenAI Settings**](/explore-analyze/ai-features/manage-access-to-ai-assistant.md). 
+2. Toggle between the two experiences under **Chat Experience**.
+
+% TODO: Link to /solutions/search/agent-builder/standalone-and-flyout-modes.md once that page exists
+
+## Feature differences between Agent Builder and AI Assistant
+
+Agent Builder doesn't yet support all AI Assistant features. The specific differences vary by solution:
+
+::::{tab-set}
+:group: example-group
+
+:::{tab-item} {{elastic-sec}}
+:sync: tab1
+
+| Feature | Agent Builder | AI Assistant |
+| :--- | :---: | :---: |
+| **Knowledge Base** | ❌ | ✅ |
+| **Data anonymization** | ❌ | ✅ |
+| **Time awareness** | ❌ | ✅ |
+| **Chat sharing** | ❌ | ✅ |
+| **Citations** | ❌ | ✅ |
+| **Audit logging** | ❌ | ✅ |
+| **Quick prompts** | ❌ | ✅ |
+| **In-chat previews of attached data** | ❌ | ✅ |
+| **AI insights** | ✅ | ✅ |
+| **Use-case specific agents** | ✅ | ❌ |
+| **Custom agent creation** | ✅ | ❌ |
+| **Custom tool selection** | ✅ | ❌ |
+| **Integration with Elastic workflows** | ✅ | ❌ |
+
+:::
+
+:::{tab-item} Elastic {{observability}} and Search
+:sync: tab2
+
+| Feature | Agent Builder | AI Assistant |
+| :--- | :---: | :---: |
+| **Knowledge Base** | ❌ | ✅ |
+| **Data anonymization** | ❌ | ✅ |
+| **Chat sharing** | ❌ | ✅ |
+| **Chat duplication** | ❌ | ✅ |
+| **Chat archiving** | ❌ | ✅ |
+| **Alerting rule connector action** | ❌ | ✅ |
+| **AI insights** | ✅ | ✅ |
+| **Use-case specific agents** | ✅ | ❌ |
+| **Custom agent creation** | ✅ | ❌ |
+| **Custom tool selection** | ✅ | ❌ |
+
+:::
+
+::::
+
+## API differences between Agent Builder and AI Assistant
+
+Each AI chat experience has different APIs. To learn which actions each option supports, refer to:
+
+- [Agent Builder](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-agent-builder)
+- [AI Assistant for Security](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-security-ai-assistant-api)
+- [AI Assistant for Observability](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-observability_ai_assistant)

explore-analyze/ai-features/ai-chat-experiences/ai-agent-or-ai-assistant.md

@@ -33,8 +33,8 @@ $$$token-limits$$$
 
 AI Assistant requires specific privileges and a large language model (LLM) connector. The capabilities and ways to interact with AI Assistant can differ for each solution. To learn more about how it works in each solution, refer to:
 
-- [{{obs-ai-assistant}}](../../solutions/observability/observability-ai-assistant.md)
-- [AI Assistant for Security](../../solutions/security/ai/ai-assistant.md)
+- [{{obs-ai-assistant}}](/solutions/observability/observability-ai-assistant.md)
+- [AI Assistant for Security](/solutions/security/ai/ai-assistant.md)
 
 To learn more about configuring LLM connectors, refer to:
 
@@ -43,7 +43,7 @@ To learn more about configuring LLM connectors, refer to:
 ## Prompt best practices [rag-for-esql]
 Elastic AI Assistant allows you to take full advantage of the Elastic platform to improve your operations. It can help you write an ES|QL query for a particular use case, or answer general questions about how to use the platform. Its ability to assist you depends on the specificity and detail of your questions. The more context and detail you provide, the more tailored and useful its responses will be.
 
-To maximize its usefulness, consider using more detailed prompts or asking for additional information. For instance, after asking for an ES|QL query example, you could ask a follow-up question like, “Could you give me some other examples?” You can also ask for clarification or further exposition, for example "Provide comments explaining the query you just gave."
+To maximize its usefulness, consider using more detailed prompts or asking for additional information. For instance, after asking for an ES|QL query example, you could ask a follow-up question like, “Could you give me some other examples?” You can also ask for clarification or further exposition, for example "Provide comments explaining the query you just gave".
 
 In addition to practical advice, AI Assistant can offer conceptual advice, tips, and best practices for enhancing your security measures. You can ask it, for example:
 
@@ -57,5 +57,5 @@ Elastic does not control third-party tools, and assumes no responsibility or lia
 
 ## Elastic Managed LLM [elastic-managed-llm-ai-assistant]
 
-:::{include} ../../solutions/_snippets/elastic-managed-llm.md
+:::{include} /solutions/_snippets/elastic-managed-llm.md
 :::

…lore-analyze/ai-features/ai-assistant.md …ures/ai-chat-experiences/ai-assistant.mdexplore-analyze/ai-features/ai-assistant.md renamed to explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md explore-analyze/ai-features/ai-assistant.md renamed to explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md

@@ -18,7 +18,7 @@ Elastic's [AI features](/explore-analyze/ai-features.md) work with the out-of-th
 
 ## Elastic Managed LLMs
 
-:::{include} ../../../solutions/_snippets/elastic-managed-llm.md
+:::{include} /solutions/_snippets/elastic-managed-llm.md
 :::
 
 ## Connect to a third-party or self-managed LLM

explore-analyze/ai-features/llm-guides/llm-connectors.md

@@ -1,6 +1,6 @@
 ---
 applies_to:
-  stack: ga 9.2
+  stack: ga 9.2+
   serverless: ga
 products:
   - id: kibana
@@ -15,7 +15,7 @@ The GenAI Settings page lets you control access to AI-powered features in the fo
 
 - Manage which AI connectors are available in your environment. 
 - Enable or disable AI Assistant and other AI-powered features in your environment.
-- {applies_to}`stack: ga 9.2` {applies_to}`serverless: unavailable` Specify in which Elastic solutions the `AI Assistant for Observability and Search` and the `AI Assistant for Security` appear.
+- {applies_to}`stack: ga 9.2+` {applies_to}`serverless: unavailable` Specify in which Elastic solutions the `AI Assistant for Observability and Search` and the `AI Assistant for Security` appear.
 
 ## Requirements
 
@@ -39,7 +39,7 @@ The **GenAI Settings** page has the following settings:
 
 :::
 
-:::{applies-item} stack: ga 9.2
+:::{applies-item} stack: ga 9.2+
 
 ![GenAI Settings page for Stack](/explore-analyze/images/ai-assistant-settings-page.png "")
 
@@ -51,11 +51,12 @@ The **GenAI Settings** page has the following settings:
   - If **Elastic Managed LLM** or a custom LLM connector is selected, AI-powered features will default to that connector regardless of which connector was used most recently in your environment. 
 - **Disallow all other connectors**: When this setting is disabled, whenever you use an AI-powered feature you can select which connector should power it. Enable it to prevent connectors other than the default connector from being used in your space. Default setting: disabled.
 - **AI feature visibility**: This button opens the current Space's settings page, where you can specify which features are enabled in your environment, including AI-powered features. 
+- **Chat experience**: {applies_to}`stack: preview 9.3+` Select whether to use AI Assistant or AI Agent. For information about choosing one, refer to [Compare AI Agent and AI Assistant](/explore-analyze/ai-features/ai-chat-experiences/ai-agent-or-ai-assistant.md).
 - **AI Assistant visibility**: This setting allows you to choose which AI Assistants are available to use and where. There are several options:
   - **Only in their solutions** (default): The Security AI Assistant appears in {{elastic-sec}}, and the {{obs-ai-assistant}} appears in {{es}} and {{observability}}.
   - **{{obs-ai-assistant}} in other apps**: The {{obs-ai-assistant}} appears throughout {{kib}} regardless of solution. The Security AI Assistant does not appear anywhere.
   - **Security AI Assistant in other apps**: The Security AI Assistant appears throughout {{kib}} regardless of solution. The {{obs-ai-assistant}} does not appear anywhere.
-  - **Hide all assistants**: Disables AI Assistant throughout {{kib}}.
+  - **Hide all assistants**: Turns off AI Assistant throughout {{kib}}.
 
 :::
 

explore-analyze/ai-features/manage-access-to-ai-assistant.md

@@ -188,7 +188,10 @@ toc:
           - file: ai-features/agent-builder/monitor-usage.md
           - file: ai-features/agent-builder/permissions.md
           - file: ai-features/agent-builder/limitations-known-issues.md
-      - file: ai-features/ai-assistant.md
+      - file: ai-features/ai-chat-experiences.md
+        children:
+          - hidden: ai-features/ai-chat-experiences/ai-agent-or-ai-assistant.md
+          - file: ai-features/ai-chat-experiences/ai-assistant.md
       - file: ai-features/manage-access-to-ai-assistant.md
       - file: ai-features/llm-guides/llm-connectors.md
         children:
@@ -201,7 +204,6 @@ toc:
               - file: ai-features/llm-guides/connect-to-lmstudio-observability.md
               - file: ai-features/llm-guides/connect-to-lmstudio-security.md
               - file: ai-features/llm-guides/connect-to-vLLM.md
-      - hidden: ai-features/ai-agent-or-ai-assistant.md
   - file: discover.md
     children:
       - file: discover/discover-get-started.md

explore-analyze/toc.yml

@@ -84,7 +84,7 @@ With {{kib}}, you can:
 - Configure notifications for significant data events and track incidents with [alerts and cases](/explore-analyze/alerts-cases.md).
 - Manage resources such as processors, pipelines, data streams, trained models, and more.
 
-Each solution or project type provides access to customized features in {{kib}} such as built-in dashboards and [AI assistants](/explore-analyze/ai-features/ai-assistant.md).
+Each solution or project type provides access to customized features in {{kib}} such as built-in dashboards and [AI assistants](/explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md).
 
 {{kib}} also has [query tools](/explore-analyze/query-filter/tools.md) such as **Console**, which provides an interactive way to send requests directly to the {{es}} API and view the responses.
 For secure, automated access, you can create and manage API keys to authenticate your scripts and applications.

get-started/the-stack.md

@@ -585,7 +585,7 @@ redirects:
 
 # Moved explore-analyze AI assistant content into a subfolder (docs-content-internal/issues/455)
   'explore-analyze/manage-access-to-ai-assistant.md': 'explore-analyze/ai-features/manage-access-to-ai-assistant.md'
-  'explore-analyze/ai-assistant.md': 'explore-analyze/ai-features/ai-assistant.md'
+  'explore-analyze/ai-assistant.md': 'explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md'
 
 # Related to https://github.com/elastic/docs-content/pull/3685
   'deploy-manage/monitor/autoops/cc-cloud-connect-autoops-faq.md': 'deploy-manage/monitor/autoops/ec-autoops-faq.md'
@@ -632,6 +632,9 @@ redirects:
   'solutions/search/cross-cluster-search.md': 'explore-analyze/cross-cluster-search.md'
   'solutions/search/using-resolve-cluster-endpoint-before-cross-cluster-search.md': 'explore-analyze/cross-cluster-search/using-resolve-cluster-endpoint-before-cross-cluster-search.md'
 
+  # Related to https://github.com/elastic/docs-content/pull/4438
+  'explore-analyze/ai-features/ai-agent-or-ai-assistant.md': 'explore-analyze/ai-features/ai-chat-experiences/ai-agent-or-ai-assistant.md'
+  
   # Related to agent builder restructure - moved agent builder docs to explore-analyze/
   'solutions/search/elastic-agent-builder.md': 'explore-analyze/ai-features/elastic-agent-builder.md'
   'solutions/search/agent-builder/tools/esql-tools.md': 'explore-analyze/ai-features/agent-builder/tools/esql-tools.md'
@@ -647,4 +650,7 @@ redirects:
   'solutions/search/agent-builder/monitor-usage.md': 'explore-analyze/ai-features/agent-builder/monitor-usage.md'
   'solutions/search/agent-builder/permissions.md': 'explore-analyze/ai-features/agent-builder/permissions.md'
   'solutions/search/agent-builder/programmatic-access.md': 'explore-analyze/ai-features/agent-builder/programmatic-access.md'
-  'solutions/search/agent-builder/tools.md': 'explore-analyze/ai-features/agent-builder/tools.md'
+  'solutions/search/agent-builder/tools.md': 'explore-analyze/ai-features/agent-builder/tools.md'
+
+# Related to https://github.com/elastic/docs-content/pull/4438
+  'explore-analyze/ai-features/ai-assistant.md': 'explore-analyze/ai-features/ai-chat-experiences/ai-assistant.md'