Skip to content

Commit 056c4e1

Browse files
committed
Many updates. Almost ready for review
1 parent 348fdb8 commit 056c4e1

File tree

3 files changed

+54
-40
lines changed

3 files changed

+54
-40
lines changed
129 KB
Loading

solutions/security/ai/ai-assistant-knowledge-base.md

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,9 @@ Refer to the following video for an example of adding a document to Knowledge Ba
119119

120120
[![Add knowledge document video](https://play.vidyard.com/rQsTujEfikpx3vv1vrbfde.jpg)](https://videos.elastic.co/watch/rQsTujEfikpx3vv1vrbfde?)
121121

122-
### Add a text, PDF, ODF, Word, Excel, PowerPoint, NDJSON, CSV, TSV, or log file [add-specific-file]
122+
### Add an individual file [add-specific-file]
123123

124-
To add an individual file to knowledge base, you first need to ingest it into an index and ensure that it includes a semantic text field.
124+
To add an individual file to knowledge base, you first need to ingest it into an index and ensure that it includes a semantic text or text field. Supported file types include text, PDF, ODF, Word, Excel, PowerPoint, NDJSON, CSV, and TSV.
125125

126126
1. Access the **Upload file** interface by using the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md) to find "File upload".
127127
2. Review the list of currently supported file formats and sizes, then select the file you want to upload. Click **Import**.
@@ -132,9 +132,6 @@ To add an individual file to knowledge base, you first need to ingest it into an
132132
- For **Copy to field**, enter a name for your new semantic text field.
133133
- For **Inference service**, use the default or select another model that's enabled in your environment.
134134
- Click **Add**. The new field appears in the **Mappings** section.
135-
:::{note}
136-
To learn more about semantic search and inference models, refer to [Elasticsearch semantic_text mapping](https://www.elastic.co/search-labs/blog/semantic-search-simplified-semantic-text).
137-
:::
138135
6. Click **Import**. File ingest begins and should complete within a few seconds.
139136
7. Once your file has been ingested to an index, add it to Knowledge Base by following the steps to [add an index](#knowledge-base-add-knowledge-index).
140137

@@ -158,6 +155,11 @@ Indices added to Knowledge Base must have at least one field mapped as [semantic
158155
4. Under **Sharing**, select whether this knowledge should be **Global** or **Private**.
159156
5. Under **Index**, enter the name of the index you want to use as a knowledge source.
160157
6. Under **Field**, enter the names of one or more semantic text fields within the index.
158+
159+
:::{note}
160+
{applies_to}`stack: ga 9.1` {applies_to}`serverless: security` You can use a text field instead of a semantic text field, though semantic text fields still offer better performance.
161+
:::
162+
161163
7. Under **Data Description**, describe when this information should be used by AI Assistant.
162164
8. Under **Query Instruction**, describe how AI Assistant should query this index to retrieve relevant documents.
163165
9. Under **Output Fields**, list the fields which AI Assistant should look at when reviewing documents in this index. If none are listed, all fields are sent.
@@ -167,16 +169,20 @@ Indices added to Knowledge Base must have at least one field mapped as [semantic
167169
:::
168170

169171

170-
### Add knowledge to an index using a connector or web crawler [knowledge-base-crawler-or-connector]
172+
### Add knowledge to an index using a content connector or web crawler [knowledge-base-crawler-or-connector]
171173

172174
You can use an {{es}} connector or web crawler to create an index that contains data you want to add to Knowledge Base.
173175

174-
This section provides an example of adding a threat intelligence feed to Knowledge Base using a web crawler. For more information on adding data to {{es}} using a connector, refer to [Ingest data with Elastic connectors](elasticsearch://reference/search-connectors/index.md). For more information on web crawlers, refer to [Elastic web crawler](https://www.elastic.co/guide/en/enterprise-search/current/crawler.html).
176+
#### Use a content connector to ingest data from third-party applications to Knowledge Base
177+
178+
You can ingest data from third-party platforms such as Github, Jira, Teams, Google Drive, Slack, email, and [more](elasticsearch://reference/search-connectors/index.md) using [content connectors](/solutions/security/get-started/content-connectors.md).
179+
180+
Once you've set up a content connector, data from the selected source is ingested to an {{es}} index. To add it knowledge base, follow the steps to [add an index](#knowledge-base-add-knowledge-index).
175181

176182

177183
#### Use a web crawler to add threat intelligence to Knowledge Base [_use_a_web_crawler_to_add_threat_intelligence_to_knowledge_base]
178184

179-
First, you’ll need to set up a web crawler to add the desired data to an index, then you’ll need to add that index to Knowledge Base.
185+
First, you’ll need to set up a web crawler to add the desired data to an index, then you’ll need to add that index to Knowledge Base. For more information on web crawlers, refer to [Elastic web crawler](https://www.elastic.co/guide/en/enterprise-search/current/crawler.html).
180186

181187
1. From the **Search** section of {{kib}}, find **Web crawlers** in the navigation menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md).
182188
2. Click **New web crawler**.
@@ -205,4 +211,10 @@ Your new threat intelligence data is now included in Knowledge Base and can info
205211

206212
Refer to the following video for an example of creating a web crawler to ingest threat intelligence data and adding it to Knowledge Base.
207213

208-
[![Add knowledge via web crawler video](https://play.vidyard.com/eYo1e1ZRwT2mjfM7Yr9MuZ.jpg)](https://videos.elastic.co/watch/eYo1e1ZRwT2mjfM7Yr9MuZ?)
214+
[![Add knowledge via web crawler video](https://play.vidyard.com/eYo1e1ZRwT2mjfM7Yr9MuZ.jpg)](https://videos.elastic.co/watch/eYo1e1ZRwT2mjfM7Yr9MuZ?)
215+
216+
217+
## Additional resources
218+
219+
- To learn more about semantic search and inference models, refer to [Elasticsearch semantic_text mapping](https://www.elastic.co/search-labs/blog/semantic-search-simplified-semantic-text).
220+
- For a walkthrough of how Knowledge Base can improve the quality of AI Assistant's responses, refer to [Use AI Assistant's Knowledge Base to improve response quality](/solutions/security/ai/usecase-knowledge-base-walkthrough.md).

solutions/security/ai/usecase-knowledge-base-walkthrough.md

Lines changed: 33 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -8,38 +8,53 @@ products:
88
---
99

1010

11-
# Use AI Assistant's Knowledge Base to Supercharge Security Operations
11+
# Use AI Assistant's Knowledge Base to improve response quality
1212

13-
This guide walks you through an example of how you can give custom information to the AI Assistant to customize it for your needs and improve the quality of its responses. It can remember everything from threat hunting playbooks, to on-call rotations, security research, infrastructure information, your team's internal communications from platforms like Slack or Teams, and more — constrained only by your creativity.
13+
You can use AI Assistant's Knowledge Base to give it information on anything from threat hunting playbooks, to on-call rotations, security research, infrastructure information, your team's internal communications from platforms like Slack or Teams, and more — constrained only by your creativity. This page guides you through an example of how to ingest data from various sources into AI Assistant's Knowledge Base, and shows how this can improve the quality of its responses in a threat response scenario.
1414

1515
## Prerequisites
1616

17-
Before following this guide, review the [Knowlege Base](/solutions/security/ai/ai-assistant-knowledge-base.md) topic for general information and prerequisites, and [enable knowledge base](/solutions/security/ai/ai-assistant-knowledge-base.md#enable-knowledge-base).
17+
Before attempting to follow this guide, review the [Knowlege Base](/solutions/security/ai/ai-assistant-knowledge-base.md) topic for general information and prerequisites, and [enable Knowledge Base](/solutions/security/ai/ai-assistant-knowledge-base.md#enable-knowledge-base).
1818

19-
## Step 3: Add Knowledge Sources
19+
## Add relevant data from various sources to Knowledge Base
2020

21-
### Add Individual Documents
21+
AI Assistant is more useful for incident response when it can access information about your organization's specific infrastructure, threat hunting playbooks, personnel, and processes. How you can add this data to Knowledge Base depends on its format and structure. This section provides several examples of useful data and how to add it.
2222

23-
- Click **New → Document** in the Knowledge Base tab.
24-
- Name the document, choose sharing (Global/Private), and enter content in Markdown.
25-
- Optionally mark as "Required knowledge" to always include as context.
23+
### Add your Slack messages to Knowledge Base
2624

27-
### Add Indices
25+
You can add messages from Slack channels to Knowledge Base using the Slack content connector. For instance, if you have a Slack channel that contains information about ongoing incidents, you could include that information in Knowledge Base to give AI Assistant more context about what your security team is dealing with.
2826

29-
- Click **New → Index**.
30-
- Specify index name, sharing, semantic text field(s), data description, query instructions, and output fields.
31-
- Indices must have at least one [semantic text](https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-text.html) field.
27+
1. Use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md) to find "Content connectors". Click **+ New Connector** to open the **Create a connector** interface.
28+
2. Follow the steps to [create a content connector](/solutions/security/get-started/content-connectors.md). This ingests your selected data into {{es}}. During setup, select `Slack`, and configure the connector to ingest your desired data.
29+
3. Follow the instructions to [add an index to Knowledge Base](/solutions/security/ai/ai-assistant-knowledge-base.md#). Select the index you created while setting up your new connector.
3230

33-
### Add Data via Connectors or Web Crawlers
31+
### Add your on-call rotation to Knowledge Base
3432

35-
- Use Elastic connectors (GitHub, Jira, Google Drive, S3, etc.) or web crawlers to ingest external data into indices.
36-
- Add those indices to the Knowledge Base as above.
33+
If you add information about who is responsible for security incidents at different dates and times to Knowledge Base, AI Assistant can help you quickly follow the correct escalation protocol for potential threats.
3734

38-
> _Comment: Confirm if there are any limitations on connector types or index sizes for Knowledge Base ingestion._
35+
If information about your on-call rotation is contained in a file, you can follow the steps to [add an individual file](/solutions/security/ai/ai-assistant-knowledge-base.md#add-specific-file) to Knowledge Base.
3936

40-
## Step 4: Use Knowledge Base in Conversations
37+
However, you can also copy and paste the information to directly [add it as a markdown document](/solutions/security/ai/ai-assistant-knowledge-base.md#knowledge-base-add-knowledge-document). Adding it as a markdown document is fast, and easy to update when the on-call rotation changes.
4138

42-
- When enabled, the AI Assistant automatically leverages Knowledge Base entries to inform its responses.
39+
### Add your threat hunting playbooks to Knowledge Base
40+
41+
If you have threat hunting playbooks stored in a GitHub repository, you can add them to Knowledge Base using the GitHub content connector. This enables AI Assistant to tell your team about your organization's standard practices for responding to a wide range of potential threats.
42+
43+
1. Use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md) to find "Content connectors". Click **+ New Connector** to open the **Create a connector** interface.
44+
2. Follow the steps to [create a content connector](/solutions/security/get-started/content-connectors.md). This ingests your selected data into {{es}}. During setup, select `GitHub`, and configure the connector to ingest your desired data.
45+
3. Follow the instructions to [add an index to Knowledge Base](/solutions/security/ai/ai-assistant-knowledge-base.md#). Select the index you created while setting up your new connector.
46+
47+
48+
:::{image} /solutions/images/security-knowledge-base-add-on-call-rotation.png
49+
:alt: Knowledge base's Edit document entry menu showing a snippet of an on call rotation document
50+
:::
51+
52+
Whichever method you use to add the information to Knowledge Base, consier making it **Required knowledge**. This will ensure that all of AI Assistant's responses are informed by the on-call rotation, even if your prompt doesn't specify that the information is relevant. This makes it more likely that AI Assistant will suggest appropriate escalation steps when you ask it about a threat.
53+
54+
55+
## Use Knowledge Base in conversations
56+
57+
AI Assistant will automatically use information you've added to Knowledge Base to inform its responses to your questions. With the information we've added in this example
4358
- You can instruct the assistant to "remember" information during chat (creates a private document).
4459
- Required knowledge entries are always included as context.
4560

@@ -49,19 +64,6 @@ Before following this guide, review the [Knowlege Base](/solutions/security/ai/a
4964
- Global entries affect all users in the space; private entries are user-specific.
5065
- Elastic Security Labs research is pre-populated as global knowledge.
5166

52-
## Best Practices
53-
54-
- Include operational details (on-call rotations, escalation contacts, infrastructure maps).
55-
- Add threat intelligence feeds and SOC playbooks.
56-
- Use connectors to keep knowledge sources up-to-date automatically.
57-
- Monitor token limits—too much context may exceed LLM limits.
58-
59-
## Troubleshooting & Known Limitations
60-
61-
- Token/context window limits depend on the selected LLM model.
62-
- Large indices or too many alerts may cause errors—reduce context size if needed.
63-
- ML node sizing and autoscaling are critical for performance.
64-
6567
## Additional Resources
6668

6769
- [Knowledge Base](https://www.elastic.co/guide/en/security/current/ai-assistant-knowledge-base.html)

0 commit comments

Comments
 (0)