From 77985ee98095ef2e2033ef202d51a6ae6731b720 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Mon, 3 Feb 2025 17:04:23 +0100 Subject: [PATCH 1/5] Refine ML intro. --- explore-analyze/machine-learning.md | 59 ++++++++++++++++--- .../serverless/machine-learning.md | 36 ----------- .../stack-docs/machine-learning/index.md | 3 - .../machine-learning-intro.md | 24 -------- 4 files changed, 52 insertions(+), 70 deletions(-) delete mode 100644 raw-migrated-files/docs-content/serverless/machine-learning.md delete mode 100644 raw-migrated-files/stack-docs/machine-learning/index.md delete mode 100644 raw-migrated-files/stack-docs/machine-learning/machine-learning-intro.md diff --git a/explore-analyze/machine-learning.md b/explore-analyze/machine-learning.md index 247442af69..8bc277af91 100644 --- a/explore-analyze/machine-learning.md +++ b/explore-analyze/machine-learning.md @@ -5,14 +5,59 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/machine-learning.html --- -# Machine learning +# What is Elastic Machine Learning? [machine-learning-intro] -% What needs to be done: Align serverless/stateful +{{ml-cap}} features analyze your data and generate models for its patterns of behavior. +The type of analysis that you choose depends on the questions or problems you want to address and the type of data you have available. -% Scope notes: include references to trained model autoscaling where appropriate +## Unsupervised {{ml}} [machine-learning-unsupervised] -% Use migrated content from existing pages that map to this page: +There are two types of analysis that can deduce the patterns and relationships within your data without training or intervention: *{{anomaly-detect}}* and *{{oldetection}}*. -% - [ ] ./raw-migrated-files/stack-docs/machine-learning/index.md -% - [ ] ./raw-migrated-files/stack-docs/machine-learning/machine-learning-intro.md -% - [ ] ./raw-migrated-files/docs-content/serverless/machine-learning.md \ No newline at end of file +[{{anomaly-detect-cap}}](machine-learning/anomaly-detection.md) requires time series data. +It constructs a probability model and can run continuously to identify unusual events as they occur. The model evolves over time; you can use its insights to forecast future behavior. + +[{{oldetection-cap}}](machine-learning/data-frame-analytics/ml-dfa-finding-outliers.md) does not require time series data. +It is a type of {{dfanalytics}} that identifies unusual points in a data set by analyzing how close each data point is to others and the density of the cluster of points around it. +It does not run continuously; it generates a copy of your data set where each data point is annotated with an {{olscore}}. +The score indicates the extent to which a data point is an outlier compared to other data points. + +## Supervised {{ml}} [machine-learning-supervised] + +There are two types of {{dfanalytics}} that require training data sets: *{{classification}}* and *{{regression}}*. + +In both cases, the result is a copy of your data set where each data point is annotated with predictions and a trained model, which you can deploy to make predictions for new data. +For more information, refer to [Introduction to supervised learning](machine-learning/data-frame-analytics/ml-dfa-overview.md#ml-supervised-workflow). + +[{{classification-cap}}](machine-learning/data-frame-analytics/ml-dfa-classification.md) learns relationships between your data points in order to predict discrete categorical values, such as whether a DNS request originates from a malicious or benign domain. + +[{{regression-cap}}](machine-learning/data-frame-analytics/ml-dfa-regression.md) learns relationships between your data points in order to predict continuous numerical values, such as the response time for a web request. + +## Feature availability by project type [machine-learning-serverless-availability] + +The {{ml-features}} that are available vary by project type: + +* {{es-serverless}} projects have trained models. +* {{observability}} projects have {{anomaly-jobs}}. +* {{elastic-sec}} projects have {{anomaly-jobs}}, {{dfanalytics-jobs}}, and trained models. + +## Synchronize saved objects [machine-learning-synchronize-saved-objects] + +Before you can view your {{ml}} {dfeeds}, jobs, and trained models in {{kib}}, they must have saved objects. +For example, if you used APIs to create your jobs, wait for automatic synchronization or go to the **{{ml-app}}** page and click **Synchronize saved objects**. + +## Export and import jobs [machine-learning-export-and-import-jobs] + +You can export and import your {{ml}} job and {{dfeed}} configuration details on the **{{ml-app}}** page. +For example, you can export jobs from your test environment and import them in your production environment. + +The exported file contains configuration details; it does not contain the {{ml}} models. +For {{anomaly-detect}}, you must import and run the job to build a model that is accurate for the new environment. +For {{dfanalytics}}, trained models are portable; you can import the job then transfer the model to the new cluster. +Refer to [Exporting and importing {{dfanalytics}} trained models](machine-learning/data-frame-analytics/ml-trained-models.md#export-import). + +There are some additional actions that you must take before you can successfully import and run your jobs: + +* The {{data-sources}} that are used by {{anomaly-detect}} {dfeeds} and {{dfanalytics}} source indices must exist; otherwise, the import fails. +* If your {{anomaly-jobs}} use custom rules with filter lists, the filter lists must exist; otherwise, the import fails. +* If your {{anomaly-jobs}} were associated with calendars, you must create the calendar in the new environment and add your imported jobs to the calendar. \ No newline at end of file diff --git a/raw-migrated-files/docs-content/serverless/machine-learning.md b/raw-migrated-files/docs-content/serverless/machine-learning.md deleted file mode 100644 index 87b9f8d5e8..0000000000 --- a/raw-migrated-files/docs-content/serverless/machine-learning.md +++ /dev/null @@ -1,36 +0,0 @@ -# {{ml-cap}} [machine-learning] - -This content applies to: [![Elasticsearch](../../../images/serverless-es-badge.svg "")](../../../solutions/search.md) [![Observability](../../../images/serverless-obs-badge.svg "")](../../../solutions/observability.md) [![Security](../../../images/serverless-sec-badge.svg "")](../../../solutions/security/elastic-security-serverless.md) - -To view your {{ml}} resources, go to **{{project-settings}} → {{manage-app}} → {{ml-app}}**: - -:::{image} ../../../images/serverless-ml-security-management.png -:alt: Anomaly detection job management -:class: screenshot -::: - -The {{ml-features}} that are available vary by project type: - -* {{es-serverless}} projects have trained models. -* {{observability}} projects have {{anomaly-jobs}}. -* {{elastic-sec}} projects have {{anomaly-jobs}}, {{dfanalytics-jobs}}, and trained models. - -For more information, go to [{{anomaly-detect-cap}}](../../../explore-analyze/machine-learning/anomaly-detection.md), [{{dfanalytics-cap}}](../../../explore-analyze/machine-learning/data-frame-analytics.md) and [Natural language processing](../../../explore-analyze/machine-learning/nlp.md). - - -## Synchronize saved objects [machine-learning-synchronize-saved-objects] - -Before you can view your {{ml}} {dfeeds}, jobs, and trained models in {{kib}}, they must have saved objects. For example, if you used APIs to create your jobs, wait for automatic synchronization or go to the **{{ml-app}}** page and click **Synchronize saved objects**. - - -## Export and import jobs [machine-learning-export-and-import-jobs] - -You can export and import your {{ml}} job and {{dfeed}} configuration details on the **{{ml-app}}** page. For example, you can export jobs from your test environment and import them in your production environment. - -The exported file contains configuration details; it does not contain the {{ml}} models. For {{anomaly-detect}}, you must import and run the job to build a model that is accurate for the new environment. For {{dfanalytics}}, trained models are portable; you can import the job then transfer the model to the new cluster. Refer to [Exporting and importing {{dfanalytics}} trained models](../../../explore-analyze/machine-learning/data-frame-analytics/ml-trained-models.md#export-import). - -There are some additional actions that you must take before you can successfully import and run your jobs: - -* The {{data-sources}} that are used by {{anomaly-detect}} {dfeeds} and {{dfanalytics}} source indices must exist; otherwise, the import fails. -* If your {{anomaly-jobs}} use custom rules with filter lists, the filter lists must exist; otherwise, the import fails. -* If your {{anomaly-jobs}} were associated with calendars, you must create the calendar in the new environment and add your imported jobs to the calendar. diff --git a/raw-migrated-files/stack-docs/machine-learning/index.md b/raw-migrated-files/stack-docs/machine-learning/index.md deleted file mode 100644 index f5a259a9f6..0000000000 --- a/raw-migrated-files/stack-docs/machine-learning/index.md +++ /dev/null @@ -1,3 +0,0 @@ -# Machine learning - -Migrated files from the Machine learning book. diff --git a/raw-migrated-files/stack-docs/machine-learning/machine-learning-intro.md b/raw-migrated-files/stack-docs/machine-learning/machine-learning-intro.md deleted file mode 100644 index d39c48004f..0000000000 --- a/raw-migrated-files/stack-docs/machine-learning/machine-learning-intro.md +++ /dev/null @@ -1,24 +0,0 @@ -# What is Elastic {{ml-app}}? [machine-learning-intro] - -{{ml-cap}} features analyze your data and generate models for its patterns of behavior. The type of analysis that you choose depends on the questions or problems you want to address and the type of data you have available. - - -## Unsupervised {{ml}} [machine-learning-unsupervised] - -There are two types of analysis that can deduce the patterns and relationships within your data without training or intervention: *{{anomaly-detect}}* and *{{oldetection}}*. - -[{{anomaly-detect-cap}}](../../../explore-analyze/machine-learning/anomaly-detection.md) requires time series data. It constructs a probability model and can run continuously to identify unusual events as they occur. The model evolves over time; you can use its insights to forecast future behavior. - -[{{oldetection-cap}}](../../../explore-analyze/machine-learning/data-frame-analytics/ml-dfa-finding-outliers.md) does not require time series data. It is a type of {{dfanalytics}} that identifies unusual points in a data set by analyzing how close each data point is to others and the density of the cluster of points around it. It does not run continuously; it generates a copy of your data set where each data point is annotated with an {{olscore}}. The score indicates the extent to which a data point is an outlier compared to other data points. - - -## Supervised {{ml}} [machine-learning-supervised] - -There are two types of {{dfanalytics}} that require training data sets: *{{classification}}* and *{{regression}}*. - -In both cases, the result is a copy of your data set where each data point is annotated with predictions and a trained model, which you can deploy to make predictions for new data. For more information, refer to [Introduction to supervised learning](../../../explore-analyze/machine-learning/data-frame-analytics/ml-dfa-overview.md#ml-supervised-workflow). - -[{{classification-cap}}](../../../explore-analyze/machine-learning/data-frame-analytics/ml-dfa-classification.md) learns relationships between your data points in order to predict discrete categorical values, such as whether a DNS request originates from a malicious or benign domain. - -[{{regression-cap}}](../../../explore-analyze/machine-learning/data-frame-analytics/ml-dfa-regression.md) learns relationships between your data points in order to predict continuous numerical values, such as the response time for a web request. - From 698d60f2d5a6ff1442f586a0c0b6fbf03b3eb840 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Mon, 3 Feb 2025 17:14:18 +0100 Subject: [PATCH 2/5] Deletes references from raw-migrated-files TOC. --- raw-migrated-files/toc.yml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index 99775ee2e7..ce654f30e9 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -307,7 +307,6 @@ toc: - file: docs-content/serverless/ingest-third-party-cloud-security-data.md - file: docs-content/serverless/ingest-wiz-data.md - file: docs-content/serverless/intro.md - - file: docs-content/serverless/machine-learning.md - file: docs-content/serverless/maintenance-windows.md - file: docs-content/serverless/maps.md - file: docs-content/serverless/monitor-k8s-otel-edot.md @@ -1032,12 +1031,6 @@ toc: - file: stack-docs/elastic-stack/upgrading-elastic-stack.md - file: stack-docs/elastic-stack/upgrading-elasticsearch.md - file: stack-docs/elastic-stack/upgrading-kibana.md - - file: stack-docs/machine-learning/index.md - children: - - file: stack-docs/machine-learning/index.md - - file: stack-docs/machine-learning/machine-learning-intro.md - - file: stack-docs/machine-learning/ml-ad-overview.md - - file: stack-docs/machine-learning/ml-dfanalytics.md - file: tech-content/starting-with-the-elasticsearch-platform-and-its-solutions/index.md children: - file: tech-content/starting-with-the-elasticsearch-platform-and-its-solutions/get-elastic.md From 050cc6b7dc5e2d71ba06a269cfdc735fb6a965d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Mon, 3 Feb 2025 17:18:53 +0100 Subject: [PATCH 3/5] Put index.md back. --- raw-migrated-files/stack-docs/machine-learning/index.md | 3 +++ raw-migrated-files/toc.yml | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 raw-migrated-files/stack-docs/machine-learning/index.md diff --git a/raw-migrated-files/stack-docs/machine-learning/index.md b/raw-migrated-files/stack-docs/machine-learning/index.md new file mode 100644 index 0000000000..f5a259a9f6 --- /dev/null +++ b/raw-migrated-files/stack-docs/machine-learning/index.md @@ -0,0 +1,3 @@ +# Machine learning + +Migrated files from the Machine learning book. diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index ce654f30e9..ee50348eb0 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -1031,6 +1031,10 @@ toc: - file: stack-docs/elastic-stack/upgrading-elastic-stack.md - file: stack-docs/elastic-stack/upgrading-elasticsearch.md - file: stack-docs/elastic-stack/upgrading-kibana.md + - file: stack-docs/machine-learning/index.md + children: + - file: stack-docs/machine-learning/ml-ad-overview.md + - file: stack-docs/machine-learning/ml-dfanalytics.md - file: tech-content/starting-with-the-elasticsearch-platform-and-its-solutions/index.md children: - file: tech-content/starting-with-the-elasticsearch-platform-and-its-solutions/get-elastic.md From 37397a090cd798702cf57741d98d25188cce9877 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Mon, 3 Feb 2025 17:23:30 +0100 Subject: [PATCH 4/5] Refines page section levels. --- explore-analyze/machine-learning.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/explore-analyze/machine-learning.md b/explore-analyze/machine-learning.md index 8bc277af91..9553229512 100644 --- a/explore-analyze/machine-learning.md +++ b/explore-analyze/machine-learning.md @@ -5,12 +5,14 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/machine-learning.html --- -# What is Elastic Machine Learning? [machine-learning-intro] +# Machine learning + +## What is Elastic machine learning? [machine-learning-intro] {{ml-cap}} features analyze your data and generate models for its patterns of behavior. The type of analysis that you choose depends on the questions or problems you want to address and the type of data you have available. -## Unsupervised {{ml}} [machine-learning-unsupervised] +### Unsupervised {{ml}} [machine-learning-unsupervised] There are two types of analysis that can deduce the patterns and relationships within your data without training or intervention: *{{anomaly-detect}}* and *{{oldetection}}*. @@ -22,7 +24,7 @@ It is a type of {{dfanalytics}} that identifies unusual points in a data set by It does not run continuously; it generates a copy of your data set where each data point is annotated with an {{olscore}}. The score indicates the extent to which a data point is an outlier compared to other data points. -## Supervised {{ml}} [machine-learning-supervised] +### Supervised {{ml}} [machine-learning-supervised] There are two types of {{dfanalytics}} that require training data sets: *{{classification}}* and *{{regression}}*. @@ -33,7 +35,7 @@ For more information, refer to [Introduction to supervised learning](machine-lea [{{regression-cap}}](machine-learning/data-frame-analytics/ml-dfa-regression.md) learns relationships between your data points in order to predict continuous numerical values, such as the response time for a web request. -## Feature availability by project type [machine-learning-serverless-availability] +### Feature availability by project type [machine-learning-serverless-availability] The {{ml-features}} that are available vary by project type: @@ -41,12 +43,12 @@ The {{ml-features}} that are available vary by project type: * {{observability}} projects have {{anomaly-jobs}}. * {{elastic-sec}} projects have {{anomaly-jobs}}, {{dfanalytics-jobs}}, and trained models. -## Synchronize saved objects [machine-learning-synchronize-saved-objects] +### Synchronize saved objects [machine-learning-synchronize-saved-objects] Before you can view your {{ml}} {dfeeds}, jobs, and trained models in {{kib}}, they must have saved objects. For example, if you used APIs to create your jobs, wait for automatic synchronization or go to the **{{ml-app}}** page and click **Synchronize saved objects**. -## Export and import jobs [machine-learning-export-and-import-jobs] +### Export and import jobs [machine-learning-export-and-import-jobs] You can export and import your {{ml}} job and {{dfeed}} configuration details on the **{{ml-app}}** page. For example, you can export jobs from your test environment and import them in your production environment. From b5c8d19825fdb11be8eda7105d8ca60302976325 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Istv=C3=A1n=20Zolt=C3=A1n=20Szab=C3=B3?= Date: Mon, 3 Feb 2025 17:27:53 +0100 Subject: [PATCH 5/5] Changes nav title. --- explore-analyze/machine-learning.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/explore-analyze/machine-learning.md b/explore-analyze/machine-learning.md index 9553229512..aae54209f2 100644 --- a/explore-analyze/machine-learning.md +++ b/explore-analyze/machine-learning.md @@ -1,18 +1,17 @@ --- +navigation_title: Machine learning mapped_urls: - https://www.elastic.co/guide/en/machine-learning/current/index.html - https://www.elastic.co/guide/en/machine-learning/current/machine-learning-intro.html - https://www.elastic.co/guide/en/serverless/current/machine-learning.html --- -# Machine learning - -## What is Elastic machine learning? [machine-learning-intro] +# What is Elastic Machine Learning? [machine-learning-intro] {{ml-cap}} features analyze your data and generate models for its patterns of behavior. The type of analysis that you choose depends on the questions or problems you want to address and the type of data you have available. -### Unsupervised {{ml}} [machine-learning-unsupervised] +## Unsupervised {{ml}} [machine-learning-unsupervised] There are two types of analysis that can deduce the patterns and relationships within your data without training or intervention: *{{anomaly-detect}}* and *{{oldetection}}*. @@ -24,7 +23,7 @@ It is a type of {{dfanalytics}} that identifies unusual points in a data set by It does not run continuously; it generates a copy of your data set where each data point is annotated with an {{olscore}}. The score indicates the extent to which a data point is an outlier compared to other data points. -### Supervised {{ml}} [machine-learning-supervised] +## Supervised {{ml}} [machine-learning-supervised] There are two types of {{dfanalytics}} that require training data sets: *{{classification}}* and *{{regression}}*. @@ -35,7 +34,7 @@ For more information, refer to [Introduction to supervised learning](machine-lea [{{regression-cap}}](machine-learning/data-frame-analytics/ml-dfa-regression.md) learns relationships between your data points in order to predict continuous numerical values, such as the response time for a web request. -### Feature availability by project type [machine-learning-serverless-availability] +## Feature availability by project type [machine-learning-serverless-availability] The {{ml-features}} that are available vary by project type: @@ -43,12 +42,12 @@ The {{ml-features}} that are available vary by project type: * {{observability}} projects have {{anomaly-jobs}}. * {{elastic-sec}} projects have {{anomaly-jobs}}, {{dfanalytics-jobs}}, and trained models. -### Synchronize saved objects [machine-learning-synchronize-saved-objects] +## Synchronize saved objects [machine-learning-synchronize-saved-objects] Before you can view your {{ml}} {dfeeds}, jobs, and trained models in {{kib}}, they must have saved objects. For example, if you used APIs to create your jobs, wait for automatic synchronization or go to the **{{ml-app}}** page and click **Synchronize saved objects**. -### Export and import jobs [machine-learning-export-and-import-jobs] +## Export and import jobs [machine-learning-export-and-import-jobs] You can export and import your {{ml}} job and {{dfeed}} configuration details on the **{{ml-app}}** page. For example, you can export jobs from your test environment and import them in your production environment.