From fb6484e8560727a3f64189f0ad58ef775d4ffa96 Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein Date: Mon, 7 Apr 2025 09:44:56 -0700 Subject: [PATCH 1/2] updates LLM matrix --- .../large-language-model-performance-matrix.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/solutions/security/ai/large-language-model-performance-matrix.md b/solutions/security/ai/large-language-model-performance-matrix.md index eb491d22ee..45facf7532 100644 --- a/solutions/security/ai/large-language-model-performance-matrix.md +++ b/solutions/security/ai/large-language-model-performance-matrix.md @@ -22,7 +22,7 @@ This page describes the performance of various large language models (LLMs) for Models from third-party LLM providers. -| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **AI-powered SIEM migration** | +| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration** | | --- | --- | --- | --- | --- | --- | --- | --- | | **Model** | **Claude 3: Opus** | Excellent | Excellent | Excellent | Good | Great | Good | | **Claude 3.7: Sonnet** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent @@ -41,9 +41,15 @@ Models from third-party LLM providers. Models you can [deploy yourself](/solutions/security/ai/connect-to-own-local-llm.md). -| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | +| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration** | | --- | --- | --- | --- | --- | --- | --- | -| **Model** | **Mistral Nemo** | Good | Good | Great | Good | Poor | -| | **LLama 3.2** | Good | Poor | Good | Poor | Poor | -| | **LLama 3.1 405b** | Good | Great | Good | Good | Poor | -| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | +| **Model** | **Mistral Nemo** | Good | Good | Great | Good | Poor | Poor | +| | **LLama 3.2** | Good | Poor | Good | Poor | Poor | Good | +| | **LLama 3.1 405b** | Good | Great | Good | Good | Poor | Poor | +| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | Good | + +| *Automatic Migration* +| Poor +| Good +| Poor +| Good \ No newline at end of file From 1302a04bfb4e03d7c731d65fd74c7cd0abca5b55 Mon Sep 17 00:00:00 2001 From: Benjamin Ironside Goldstein Date: Mon, 7 Apr 2025 12:51:49 -0700 Subject: [PATCH 2/2] fixes matrix --- .../ai/large-language-model-performance-matrix.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/solutions/security/ai/large-language-model-performance-matrix.md b/solutions/security/ai/large-language-model-performance-matrix.md index 45facf7532..c8f65804e8 100644 --- a/solutions/security/ai/large-language-model-performance-matrix.md +++ b/solutions/security/ai/large-language-model-performance-matrix.md @@ -41,15 +41,9 @@ Models from third-party LLM providers. Models you can [deploy yourself](/solutions/security/ai/connect-to-own-local-llm.md). -| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration** | +| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration** | --- | --- | --- | --- | --- | --- | --- | | **Model** | **Mistral Nemo** | Good | Good | Great | Good | Poor | Poor | | | **LLama 3.2** | Good | Poor | Good | Poor | Poor | Good | | | **LLama 3.1 405b** | Good | Great | Good | Good | Poor | Poor | -| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | Good | - -| *Automatic Migration* -| Poor -| Good -| Poor -| Good \ No newline at end of file +| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | Good | \ No newline at end of file