From d02e23164156d59c4abf2aee5248f0a1528498f6 Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Wed, 17 Jan 2024 15:20:47 -0600 Subject: [PATCH 1/4] Grok deprecation --- .../ingestion/cse-ingestion-best-practices.md | 29 ++++------- docs/cse/schema/grok-deprecation-notice.md | 52 +++++++++++++++++++ 2 files changed, 62 insertions(+), 19 deletions(-) create mode 100644 docs/cse/schema/grok-deprecation-notice.md diff --git a/docs/cse/ingestion/cse-ingestion-best-practices.md b/docs/cse/ingestion/cse-ingestion-best-practices.md index d0ad182179..d245b5aeee 100644 --- a/docs/cse/ingestion/cse-ingestion-best-practices.md +++ b/docs/cse/ingestion/cse-ingestion-best-practices.md @@ -27,38 +27,29 @@ You can only send log data that resides in the [Continuous data tier](/docs/mana We recommend the following ingestion processes, starting with the most preferred: -1. **Follow an ingestion guide**. The [Ingestion Guides](/docs/cse/ingestion) section of this help site provides specific collection and ingestion recommendations for many common products and services. An ingestion guide describes the easiest way to get data from a particular product into Cloud SIEM. When you’re ready to start using Cloud SIEM to monitor a new product, if there’s a Cloud SIEM ingestion guide for it, we recommend using it.  -   +1. **Follow an ingestion guide**. The [Ingestion Guides](/docs/cse/ingestion) section of this help site provides specific collection and ingestion recommendations for many common products and services. An ingestion guide describes the easiest way to get data from a particular product into Cloud SIEM. When you’re ready to start using Cloud SIEM to monitor a new product, if there’s a Cloud SIEM ingestion guide for it, we recommend using it.    1. **Use a Cloud-to-Cloud (C2C) connector**. If you don’t see an Ingestion Guide for your data source, check to see if there is a C2C connector. It’s an easy method, because if you configure your C2C source to send logs to Cloud SIEM, it automatically tags messages it sends to Cloud SIEM with fields that contain the mapping hints that Cloud SIEM requires.  Most C2C connectors have a **Forward to SIEM** option in the configuration UI. If a C2C connector lacks that option, you can achieve the same effect by assigning a field named `_siemforward`, set to *true*, to the connector. For information about what C2C sources are available, see Cloud-to-Cloud Integration Framework. -   1. **Use a Sumo Logic Source and parser**. If there isn’t a C2C connector for your data source, your next best option is to use a Sumo Logic Source (running on an Installed Collector or a Hosted Collector, depending on the data source)—and a Sumo Logic parser, if we have one for the data source.  - To check if there’s a parser for your data source, go to the **Manage Data > Logs > Parsers** page in the Sumo Logic UI. If there is a parser for your data source, but you find it doesn’t completely meet your needs–for instance if the parser doesn’t support the particular log format you use–consider customizing the parser with a [local configuration](/docs/cse/schema/parser-editor#create-a-local-configuration-for-a-system-parser). If that’s not practical, you can submit a request for a new parser by filing a ticket at [https://support.sumologic.com](https://support.sumologic.com/). - - When you forward logs to Cloud SIEM for parser processing, there are two bits of important configuration: -   + To check if there’s a parser for your data source, go to the **Manage Data > Logs > Parsers** page in the Sumo Logic UI. If there is a parser for your data source, but you find it doesn’t completely meet your needs–for instance if the parser doesn’t support the particular log format you use–consider customizing the parser with a [local configuration](/docs/cse/schema/parser-editor#create-a-local-configuration-for-a-system-parser). If that’s not practical, you can submit a request for a new parser by filing a ticket at [https://support.sumologic.com](https://support.sumologic.com/). + + When you forward logs to Cloud SIEM for parser processing, there are two bits of important configuration: 1. Configure the source to forward logs. To configure an HTTP source to send log messages to Cloud SIEM, click the **SIEM Processing** checkbox. You can configure other source types to send data to Cloud SIEM by assigning a field named `_siemforward`, set to *true*, to the source. For example: - ``` _siemforward=true ``` - - :::note - A field can also be assigned at the collector level, in which case sources on the collector inherit the field setting, unless the same field is defined with a different value at the source level. - ::: - + :::note + A field can also be assigned at the collector level, in which case sources on the collector inherit the field setting, unless the same field is defined with a different value at the source level. + ::: 2. Configure the source with the path to the appropriate parser, by assigning a field named `_parser`, whose value is the path to parser, for example: - ``` _parser=/Parsers/System/AWS/AWS Network Firewall ``` - - :::note  - You can get the path to a parser on the **Manage Data > Logs > Parsers** page in Sumo Logic. Click the three-dot more options menu in the row for a parser, and select **Copy Path**. - ::: - + :::note + You can get the path to a parser on the **Manage Data > Logs > Parsers** page in Sumo Logic. Click the three-dot more options menu in the row for a parser, and select **Copy Path**. + ::: 1. **Use a Sumo Logic Source and Cloud SIEM Ingest mapping**. This is the least recommended method, as you have to manually configure the mapping hints in an ingestion mapping. For more information, see [Configure a Sumo Logic Ingest Mapping](/docs/cse/ingestion/sumo-logic-ingest-mapping/). diff --git a/docs/cse/schema/grok-deprecation-notice.md b/docs/cse/schema/grok-deprecation-notice.md new file mode 100644 index 0000000000..561d45b460 --- /dev/null +++ b/docs/cse/schema/grok-deprecation-notice.md @@ -0,0 +1,52 @@ +--- +id: grok-deprecation-notice +title: Grok Deprecation Notice +sidebar_label: Grok Deprecation +description: Learn about deprecation of Grok and what you need to know about migrating to Sumo Logic data sources. +--- + + + + +## Update your Cloud SIEM data sources + +On January 19, 2024, Sumo Logic deprecates the Grok parsing service. For over two years Sumo Logic has been using Sumo Parsers and Cloud-to-Cloud (C2C) integrations as the approved methods of ingesting data into Cloud SIEM. If your Sumo Logic Instance contains sources that are utilizing Grok for parsing, you must migrate them to use the supported C2Cs or Sumo Logic parsers. + +If you do not configure your data sources to use the supported C2Cs or Sumo Logic parsers, then you will lose the ability for those data sources to create any Records, Signals, or Insights within Cloud SIEM on the Grok deprecation date and going forward. + +## Setting up Cloud-to-Cloud integrations + +The Cloud-to-Cloud Integration Framework is a fully-managed collection system that collects logs and events directly from SaaS and Cloud platforms. This data often includes custom events and user data critical for operations monitoring, security, and compliance use cases. As a fully managed collection system, integrations running within the Cloud-to-Cloud Integration Framework provide a secure endpoint to receive event data in your account. Integration authentication, scheduling, and state tracking are all managed by the framework. + +Documentation on setting up C2C integrations are available publicly at [Cloud-to-Cloud Integration Framework Sources](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/). + +## Setting up Sumo Logic parsers + +To set up Sumo Logic parsers, we recommend following [Cloud SIEM ingestion best practices](/docs/cse/ingestion/cse-ingestion-best-practices/#cloud-siem-ingestion-best-practices) as described below. + +If there isn’t a C2C connector for your data source, your next best option is to use a Sumo Logic Source (running on an Installed Collector or a Hosted Collector, depending on the data source)—and a Sumo Logic parser, if we have one for the data source.  + +To check if there’s a parser for your data source, go to the **Manage Data > Logs > Parsers** page in the Sumo Logic UI. If there is a parser for your data source, but you find it doesn’t completely meet your needs–for instance if the parser doesn’t support the particular log format you use–consider customizing the parser with a [local configuration](/docs/cse/schema/parser-editor#create-a-local-configuration-for-a-system-parser). If that’s not practical, you can submit a request for a new parser by filing a ticket at [https://support.sumologic.com](https://support.sumologic.com/). + + When you forward logs to Cloud SIEM for parser processing, there are two bits of important configuration: + 1. Configure the source to forward logs. To configure an HTTP source to send log messages to Cloud SIEM, click the **SIEM Processing** checkbox. You can configure other source types to send data to Cloud SIEM by assigning a field named `_siemforward`, set to *true*, to the source. For example: + ``` + _siemforward=true + ``` + :::note + A field can also be assigned at the collector level, in which case sources on the collector inherit the field setting, unless the same field is defined with a different value at the source level. + ::: + 2. Configure the source with the path to the appropriate parser, by assigning a field named `_parser`, whose value is the path to parser, for example: + ``` + _parser=/Parsers/System/AWS/AWS Network Firewall + ``` + :::note + You can get the path to a parser on the **Manage Data > Logs > Parsers** page in Sumo Logic. Click the three-dot more options menu in the row for a parser, and select **Copy Path**. + ::: + +## Additional references + +* [Cloud SIEM Ingestion Best Practices](/docs/cse/ingestion/cse-ingestion-best-practices/) +* [Ingestion Sources for Cloud SIEM](/docs/cse/ingestion/ingestion-sources-for-cloud-siem/) +* [Cloud-to-Cloud Integration Framework Sources](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/) +* [Parser Troubleshooting Tips](/docs/cse/schema/parser-troubleshooting-tips/) From 3ee5669bf8a4a659fefc87873c5dbb290e8c0315 Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Mon, 2 Jun 2025 11:18:16 -0500 Subject: [PATCH 2/4] Fix broken link --- docs/cse/schema/grok-deprecation-notice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cse/schema/grok-deprecation-notice.md b/docs/cse/schema/grok-deprecation-notice.md index 561d45b460..4738764802 100644 --- a/docs/cse/schema/grok-deprecation-notice.md +++ b/docs/cse/schema/grok-deprecation-notice.md @@ -49,4 +49,4 @@ To check if there’s a parser for your data source, go to the **Manage Data > L * [Cloud SIEM Ingestion Best Practices](/docs/cse/ingestion/cse-ingestion-best-practices/) * [Ingestion Sources for Cloud SIEM](/docs/cse/ingestion/ingestion-sources-for-cloud-siem/) * [Cloud-to-Cloud Integration Framework Sources](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/) -* [Parser Troubleshooting Tips](/docs/cse/schema/parser-troubleshooting-tips/) +* [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers//) From 9c7b57e5973da47302bce36177ae2e2b41290624 Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Mon, 2 Jun 2025 11:37:01 -0500 Subject: [PATCH 3/4] Fix broken link again --- docs/cse/schema/grok-deprecation-notice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cse/schema/grok-deprecation-notice.md b/docs/cse/schema/grok-deprecation-notice.md index 4738764802..bb38fbdc3d 100644 --- a/docs/cse/schema/grok-deprecation-notice.md +++ b/docs/cse/schema/grok-deprecation-notice.md @@ -49,4 +49,4 @@ To check if there’s a parser for your data source, go to the **Manage Data > L * [Cloud SIEM Ingestion Best Practices](/docs/cse/ingestion/cse-ingestion-best-practices/) * [Ingestion Sources for Cloud SIEM](/docs/cse/ingestion/ingestion-sources-for-cloud-siem/) * [Cloud-to-Cloud Integration Framework Sources](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/) -* [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers//) +* [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers/) From 305972d4cf71a6f87706d3081ab38f2f372341e7 Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Mon, 2 Jun 2025 13:39:26 -0500 Subject: [PATCH 4/4] Fix broken link --- docs/cse/schema/grok-deprecation-notice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cse/schema/grok-deprecation-notice.md b/docs/cse/schema/grok-deprecation-notice.md index bb38fbdc3d..e82db615f0 100644 --- a/docs/cse/schema/grok-deprecation-notice.md +++ b/docs/cse/schema/grok-deprecation-notice.md @@ -22,7 +22,7 @@ Documentation on setting up C2C integrations are available publicly at [Cloud-to ## Setting up Sumo Logic parsers -To set up Sumo Logic parsers, we recommend following [Cloud SIEM ingestion best practices](/docs/cse/ingestion/cse-ingestion-best-practices/#cloud-siem-ingestion-best-practices) as described below. +To set up Sumo Logic parsers, we recommend following [Cloud SIEM ingestion best practices](/docs/cse/ingestion/cse-ingestion-best-practices/) as described below. If there isn’t a C2C connector for your data source, your next best option is to use a Sumo Logic Source (running on an Installed Collector or a Hosted Collector, depending on the data source)—and a Sumo Logic parser, if we have one for the data source.