add anonymization

solutions/observability/observability-ai-assistant.md

@@ -61,7 +61,7 @@ It's important to understand how your data is handled when using the AI Assistan
 :   Elastic does not use customer data for model training, but all data is processed by third-party AI providers.
 
 **Anonymization**
-:   Data sent to the AI Assistant is *not* anonymized, including alert data, configurations, queries, logs, and chat interactions.
+:   Data sent to the AI Assistant is *not* anonymized, including alert data, configurations, queries, logs, and chat interactions. If you need to anonymize data, use the [anonymization pipeline](#obs-ai-anonymization).
 
 **Permission context**
 :   When the AI Assistant performs searches, it uses the same permissions as the current user.
@@ -418,6 +418,68 @@ Enable this feature from the **Settings** tab in AI Assistant Settings by using
 For air-gapped environments, installing product documentation requires special configuration. See the [{{kib}} AI Assistants settings documentation](kibana://reference/configuration-reference/ai-assistant-settings.md) for detailed instructions.
 ::::
 
+## Anonymization (technical preview) [obs-ai-anonymization]
+
+Anonymization masks personally identifiable or otherwise sensitive information before chat messages leave Kibana for a third-party LLM.
+Enabled rules substitute deterministic tokens (for example EMAIL_ee4587…) so the model can keep context without ever seeing the real value.
+When all rules are disabled (the default), data is forwarded unchanged.
+
+### How it works [obs-ai-anonymization-how]
+
+When anonymization is enabled, every message in the request (system prompt, message content, tool call arguments/response, and tool-call response fields) is run through an *anonymization pipeline* before it leaves Kibana:
+
+1. Each enabled **rule** scans its target text and replaces any match with a deterministic token such as  
+   `EMAIL_ee4587b4ba681e38996a1b716facbf375786bff7`.  
+   The prefix (`EMAIL`, `PER`, `LOC`, …) is the *entity class*; the suffix is a SHA-1 hash of the original value.
+2. The fully masked conversation is sent to the LLM.
+3. After the LLM responds, the original values are restored so the user sees de-anonymised text and any persisted conversation history stores the original content. 
+
+Because the masking is deterministic, the model can still maintain logical consistency (“`EMAIL_x`” always refers to the same address) without ever seeing the real value.
+
+### Rule types [obs-ai-anonymization-rules]
+
+
+**RegExp** — Runs a JavaScript‑style regular expression. Use for fixed patterns such as e‑mail addresses, host names, etc.
+
+```jsonc
+{
+  "type": "RegExp",
+  "pattern": "([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,})",
+  "entityClass": "EMAIL",
+  "enabled": true
+}
+```
+
+**NER** — Runs a named‑entity‑recognition model on free text.
+
+```jsonc
+{
+  "type": "NER",
+  "modelId": "elastic__distilbert-base-uncased-finetuned-conll03-english",
+  "allowedEntityClasses": ["PER", "ORG", "LOC"],
+  "enabled": true
+}
+```
+
+Rules are evaluated **top-to-bottom**; the first rule that captures a given entity wins. Rules can be configured in the [AI Assistant Settings](#obs-ai-settings) page.
+
+### Requirements [obs-ai-anonymization-requirements]
+
+* **Advanced Settings privilege** to edit the configuration and enable rules.  
+  Once saved, *all* users in the same **Space** benefit from the anonymization (the setting is [space-aware](../../deploy-manage/manage-spaces.md)).
+* **ML privilege and resources** if you enable a rule of type NER, you must first [deploy and start a named-entity-recognition model](/explore-analyze/machine-learning/nlp/ml-nlp-ner-example.md#ex-ner-deploy) and have sufficient ML capacity.
+
+::::{important}
+The anonymization pipeline has only been validated with Elastic’s English model  
+[elastic/distilbert-base-uncased-finetuned-conll03-english](https://huggingface.co/elastic/distilbert-base-uncased-finetuned-conll03-english).  
+Results for other languages or models may vary.
+::::
+
+### Limitations [obs-ai-anonymization-limitations]
+* **Performance (NER)** – Running a named entity recognition model can add latency depending on the request.  
+* **Structured JSON** – The NER model we validated (`elastic/distilbert-base-uncased-finetuned-conll03-english`) is trained on natural English text and often misses entities inside JSON or other structured data. If thorough masking is required, prefer regex rules and craft them to account for JSON syntax. 
+* **False negatives / positives** – No model or pattern is perfect. Model accuracy may vary depending on model and input.
+
 ## Known issues [obs-ai-known-issues]
 
 ### Token limits [obs-ai-token-limits]