From 504612e4e09829041c882f7d4fd7a34e07055997 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Wed, 16 Apr 2025 18:39:14 +0530 Subject: [PATCH 1/9] Best practices for enhanced search performance --- docs/manage/ingestion-volume/log-ingestion.md | 2 +- docs/search/optimize-search-performance.md | 188 ++++++++++++++++++ 2 files changed, 189 insertions(+), 1 deletion(-) diff --git a/docs/manage/ingestion-volume/log-ingestion.md b/docs/manage/ingestion-volume/log-ingestion.md index b28c62143a..6e2caaf459 100644 --- a/docs/manage/ingestion-volume/log-ingestion.md +++ b/docs/manage/ingestion-volume/log-ingestion.md @@ -29,7 +29,7 @@ Log data may not be kept when sent via HTTP Sources or Cloud Syslog Sources, as * Sumo Logic accounts can be upgraded at any time to allow for additional quota. Contact [Sumo Logic Sales](mailto:sales@sumologic.com) to customize your account to meet your organization's needs. :::important -Compressed files are decompressed before they are ingested, so they are ingested at the decompressed file size rate. +[Compressed files](/docs/send-data/hosted-collectors/http-source/logs-metrics/#compressed-data) are decompressed before they are ingested, so they are ingested at the decompressed file size rate. ::: ## Log Throttling diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index d0b030d084..f23ed5ae77 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -70,3 +70,191 @@ Here's a quick look at how to choose the right indexed search optimization tool. As data enters Sumo Logic, it is first routed to any Partitions for indexing. It is then checked against Scheduled Views, and any data that matches the Scheduled Views is indexed. Data can be in both a Partition and a Scheduled View because the two tools are used differently (and are indexed separately). Although Partitions are indexed first, the process does not slow the indexing of Scheduled Views. + +## Additional methods to optimize Search performance + +### Use the smallest Time Range + +Always set the search time range to the minimum duration required for your use case. This reduces the data volume and improve the query efficiency. When working with long time ranges, start by building and testing your search on a shorter time range. Once the search is finalized and validated, extend it to cover the entire period needed for your analysis. + +### Use fields extracted by FERs + +Instead of relying on the `where` operator, filter the data using fields that are already extracted through the Field Extraction Rules (FERs) in the source expression. This approach is more efficient and improves query performance. + +**Recommended approach:** + +``` +sourceCategory=foo and field_a=value_a +``` + +**Not recommended approach:** + +``` +_sourceCategory=foo +| where field_a="value_a" +``` + +### Move terms from parse statement to source expression + +Adding the parsing terms in the source expression will help you enhance the search performace. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse “completed * action“ as actionName` will remove logs that do not have **completed** and **action** terms. + +**Recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog completed action +| parse “completed * action“ as actionName +| count by actionName +``` + +**Not recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “completed * action“ as actionName +| count by actionName +``` + +### Filter data before aggregation + +While filtering the date, reduces the result set to the smallest possible size before performing aggregate operations such as sum, min, max, and average. Also, use subquery in source expression instead of using `if` or `where` search operators. + +**Recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog userName +| parse “userName: *, “ as user +| where user="john" +| count by user +``` + +**Not recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “userName: *, “ as user +| count by user +| where user="john" +``` + +### Remove redundant operators + +Remove the search operators in the query that are not reffered or is not really required for the desired results. + +For example, let’s say you have a `sort` operator before an aggregation and this sorting does not make any difference to the aggregated results, resulting in reducing the performance. + +**Recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “userName: *, “ as user +| count by user +``` + +**Not recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “userName: *, “ as user +| parse “evenName: *, “ as event +| count by user +``` + +### Merge operators + +If the same operators are used multiple times in different levels of query, if possible, try to merge these similar operators. Also, do not use the same operator multiple times to get the same value. This helps in reducing the number of passes performed on the data thereby improving the search performance. + +**Example 1:** + + **Recommended approach:** + + ``` + _sourceCategory=Prod/User/Eventlog + | parse “completed * action in * ms“ as actionName, duration + | pct(duration, 95) by actionName + ``` + + **Not recommended approach:** + + ``` + _sourceCategory=Prod/User/Eventlog + | parse “completed * action“ as actionName + | parse “action in * ms“ as duration + | pct(duration, 95) by actionName + ``` + +**Example 2:** + + **Recommended approach:** + + ``` + _sourceCategory=Prod/User/Eventlog + | parse “completed * action“ as actionName + | toLowerCase(actionName) as actionNameLowered + | where actionNameLowered = “logIn” or actionNameLowered matches “abc*” or actionNameLowered contains “xyz” + ``` + + **Not recommended approach:** + + ``` + _sourceCategory=Prod/User/Eventlog + | parse “completed * action“ as actionName + | where toLowerCase(actionName) = “logIn” or toLowerCase(actionName) matches “abc*” or toLowerCase(actionName) contains “xyz" + ``` + +### Use lookup on the lowest possible dataset + +Minimize the data processed by the `lookup` operator in the query, as lookup is an expensive operation. It can be done in two ways: + +- Use the lookup as late as possible in the query assuming that clauses before lookup are doing additional data filtering. +- Move the lookup after an aggregation to drastically reduce the data processed by lookup, as aggregated data is generally far less than non-aggregated data. + +**Not recommended approach:** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “completed * action in * ms“ as actionName, duration +| lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName +| where actionName in (“login”, “logout”) +| count by actionName, actionType +``` + +**Recommended approach (Option 1):** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “completed * action in * ms“ as actionName, duration +| where actionName in (“login”, “logout”) +| count by actionName +| lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName +``` + +**Recommended approach (Option 2):** + +``` +_sourceCategory=Prod/User/Eventlog +| parse “completed * action in * ms“ as actionName, duration +| where actionName in (“login”, “logout”) +| lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName +| count by actionName, actionType +``` + +### Avoid multiple parse multi statements + +A parse multi statement causes a single log to produce multiple logs in the results. But if a parse multi statement is followed by more parse multi statements, it can lead to data explosion and the query may never finish. Even if the query works the results may not be as expected. + +For example, consider the below query where the assumption is that a single log line contains multiple users and multiple event names. + +``` +_sourceCategory=Prod/User/Eventlog +| parse regex “userName: (?[a-z-A-Z]+), “ multi +| parse regex “eventName: (?[a-z-A-Z]+), “ multi +``` + +But if you write the query like that, it will generate a result for every combination of `userName` and `eventName` values. Now suppose you want to count by `eventName`, it will not give you the desired result, since a single `eventName` has been duplicated for every `userName` in the same log. So, the better query would be: + +``` +_sourceCategory=Prod/User/Eventlog +| parse regex “userName: (?[a-z-A-Z]+), eventName: (?[a-z-A-Z]+), “ multi +``` + + From f63cbfc3b5e31bfd8936646948b62a1c4c0617eb Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Wed, 16 Apr 2025 19:00:14 +0530 Subject: [PATCH 2/9] minor fixes --- docs/search/optimize-search-performance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index f23ed5ae77..3c53fca9b8 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -96,7 +96,7 @@ _sourceCategory=foo ### Move terms from parse statement to source expression -Adding the parsing terms in the source expression will help you enhance the search performace. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse “completed * action“ as actionName` will remove logs that do not have **completed** and **action** terms. +Adding the parsing terms in the source expression will help you enhance the search performance. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse “completed * action“ as actionName` will remove logs that do not have **completed** and **action** terms. **Recommended approach:** @@ -138,7 +138,7 @@ _sourceCategory=Prod/User/Eventlog ### Remove redundant operators -Remove the search operators in the query that are not reffered or is not really required for the desired results. +Remove the search operators in the query that are not referred or is not really required for the desired results. For example, let’s say you have a `sort` operator before an aggregation and this sorting does not make any difference to the aggregated results, resulting in reducing the performance. From 2ec8206a5042d76c9b173a7244a30df6c6ea2c8f Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 11:51:59 +0530 Subject: [PATCH 3/9] Update docs/search/optimize-search-performance.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/search/optimize-search-performance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index 3c53fca9b8..0911584e5b 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -240,7 +240,7 @@ _sourceCategory=Prod/User/Eventlog ### Avoid multiple parse multi statements -A parse multi statement causes a single log to produce multiple logs in the results. But if a parse multi statement is followed by more parse multi statements, it can lead to data explosion and the query may never finish. Even if the query works the results may not be as expected. +A parse `multi` statement causes a single log to produce multiple logs in the results. But if a parse `multi` statement is followed by more parse `multi` statements, it can lead to data explosion and the query may never finish. Even if the query works the results may not be as expected. For example, consider the below query where the assumption is that a single log line contains multiple users and multiple event names. From 64703eb38db87219e9d756d2586b8ca4a64fea55 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 11:52:15 +0530 Subject: [PATCH 4/9] Update docs/search/optimize-search-performance.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/search/optimize-search-performance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index 0911584e5b..d910d97eb1 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -116,7 +116,7 @@ _sourceCategory=Prod/User/Eventlog ### Filter data before aggregation -While filtering the date, reduces the result set to the smallest possible size before performing aggregate operations such as sum, min, max, and average. Also, use subquery in source expression instead of using `if` or `where` search operators. +While filtering the date, reduce the result set to the smallest possible size before performing aggregate operations such as sum, min, max, and average. Also, use subquery in source expression instead of using `if` or `where` search operators. **Recommended approach:** From 2f2a7ccd872b0b9daa3ed5be1790752e97f4c5f3 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 11:52:21 +0530 Subject: [PATCH 5/9] Update docs/search/optimize-search-performance.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/search/optimize-search-performance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index d910d97eb1..4c8a7df7ac 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -138,9 +138,9 @@ _sourceCategory=Prod/User/Eventlog ### Remove redundant operators -Remove the search operators in the query that are not referred or is not really required for the desired results. +Remove the search operators in the query that are not required for the desired results. -For example, let’s say you have a `sort` operator before an aggregation and this sorting does not make any difference to the aggregated results, resulting in reducing the performance. +For example, let’s say you have a `sort` operator before an aggregation, but this sorting does not make any difference to the aggregated results. **Recommended approach:** From df43835dce79be6bc4713393889047a13d793f1e Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 11:52:43 +0530 Subject: [PATCH 6/9] Update docs/search/optimize-search-performance.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/search/optimize-search-performance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index 4c8a7df7ac..ba31477a4e 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -75,7 +75,7 @@ Data can be in both a Partition and a Scheduled View because the two tools are u ### Use the smallest Time Range -Always set the search time range to the minimum duration required for your use case. This reduces the data volume and improve the query efficiency. When working with long time ranges, start by building and testing your search on a shorter time range. Once the search is finalized and validated, extend it to cover the entire period needed for your analysis. +Always set the search time range to the minimum duration required for your use case. This reduces the data volume and improves the query efficiency. When working with long time ranges, start by building and testing your search on a shorter time range. Once the search is finalized and validated, extend it to cover the entire period needed for your analysis. ### Use fields extracted by FERs From b58a567ecae3730b00a480ada1088cc84a39ee1a Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 11:52:50 +0530 Subject: [PATCH 7/9] Update docs/search/optimize-search-performance.md Co-authored-by: John Pipkin (Sumo Logic) --- docs/search/optimize-search-performance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index ba31477a4e..e46ceba294 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -73,7 +73,7 @@ Data can be in both a Partition and a Scheduled View because the two tools are u ## Additional methods to optimize Search performance -### Use the smallest Time Range +### Use the smallest time range Always set the search time range to the minimum duration required for your use case. This reduces the data volume and improves the query efficiency. When working with long time ranges, start by building and testing your search on a shorter time range. Once the search is finalized and validated, extend it to cover the entire period needed for your analysis. From 8a784886024439c4a2120bd556a0156700677eb6 Mon Sep 17 00:00:00 2001 From: Jagadisha V <129049263+JV0812@users.noreply.github.com> Date: Thu, 17 Apr 2025 12:08:34 +0530 Subject: [PATCH 8/9] minor fixes --- docs/search/optimize-search-performance.md | 56 +++++++++++----------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index 3c53fca9b8..16275849df 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -81,35 +81,35 @@ Always set the search time range to the minimum duration required for your use c Instead of relying on the `where` operator, filter the data using fields that are already extracted through the Field Extraction Rules (FERs) in the source expression. This approach is more efficient and improves query performance. -**Recommended approach:** +**Not recommended approach:** ``` -sourceCategory=foo and field_a=value_a +_sourceCategory=foo +| where field_a="value_a" ``` -**Not recommended approach:** +**Recommended approach:** ``` -_sourceCategory=foo -| where field_a="value_a" +sourceCategory=foo and field_a=value_a ``` ### Move terms from parse statement to source expression Adding the parsing terms in the source expression will help you enhance the search performance. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse “completed * action“ as actionName` will remove logs that do not have **completed** and **action** terms. -**Recommended approach:** +**Not recommended approach:** ``` -_sourceCategory=Prod/User/Eventlog completed action +_sourceCategory=Prod/User/Eventlog | parse “completed * action“ as actionName | count by actionName ``` -**Not recommended approach:** +**Recommended approach:** ``` -_sourceCategory=Prod/User/Eventlog +_sourceCategory=Prod/User/Eventlog completed action | parse “completed * action“ as actionName | count by actionName ``` @@ -118,22 +118,22 @@ _sourceCategory=Prod/User/Eventlog While filtering the date, reduces the result set to the smallest possible size before performing aggregate operations such as sum, min, max, and average. Also, use subquery in source expression instead of using `if` or `where` search operators. -**Recommended approach:** +**Not recommended approach:** ``` -_sourceCategory=Prod/User/Eventlog userName +_sourceCategory=Prod/User/Eventlog | parse “userName: *, “ as user -| where user="john" | count by user +| where user="john" ``` -**Not recommended approach:** +**Recommended approach:** ``` -_sourceCategory=Prod/User/Eventlog +_sourceCategory=Prod/User/Eventlog userName | parse “userName: *, “ as user -| count by user | where user="john" +| count by user ``` ### Remove redundant operators @@ -142,20 +142,20 @@ Remove the search operators in the query that are not referred or is not really For example, let’s say you have a `sort` operator before an aggregation and this sorting does not make any difference to the aggregated results, resulting in reducing the performance. -**Recommended approach:** +**Not recommended approach:** ``` _sourceCategory=Prod/User/Eventlog | parse “userName: *, “ as user +| parse “evenName: *, “ as event | count by user ``` -**Not recommended approach:** +**Recommended approach:** ``` _sourceCategory=Prod/User/Eventlog | parse “userName: *, “ as user -| parse “evenName: *, “ as event | count by user ``` @@ -165,40 +165,40 @@ If the same operators are used multiple times in different levels of query, if p **Example 1:** - **Recommended approach:** + **Not recommended approach:** ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action in * ms“ as actionName, duration + | parse “completed * action“ as actionName + | parse “action in * ms“ as duration | pct(duration, 95) by actionName ``` - **Not recommended approach:** + **Recommended approach:** ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action“ as actionName - | parse “action in * ms“ as duration + | parse “completed * action in * ms“ as actionName, duration | pct(duration, 95) by actionName ``` **Example 2:** - **Recommended approach:** + **Not recommended approach:** ``` _sourceCategory=Prod/User/Eventlog | parse “completed * action“ as actionName - | toLowerCase(actionName) as actionNameLowered - | where actionNameLowered = “logIn” or actionNameLowered matches “abc*” or actionNameLowered contains “xyz” + | where toLowerCase(actionName) = “logIn” or toLowerCase(actionName) matches “abc*” or toLowerCase(actionName) contains “xyz" ``` - **Not recommended approach:** + **Recommended approach:** ``` _sourceCategory=Prod/User/Eventlog | parse “completed * action“ as actionName - | where toLowerCase(actionName) = “logIn” or toLowerCase(actionName) matches “abc*” or toLowerCase(actionName) contains “xyz" + | toLowerCase(actionName) as actionNameLowered + | where actionNameLowered = “logIn” or actionNameLowered matches “abc*” or actionNameLowered contains “xyz” ``` ### Use lookup on the lowest possible dataset From ee961491dfa1ee280213a82280d47b344d318a1e Mon Sep 17 00:00:00 2001 From: John Pipkin Date: Thu, 17 Apr 2025 11:26:41 -0500 Subject: [PATCH 9/9] Replace slanted quotation marks with straight ones --- docs/search/optimize-search-performance.md | 48 +++++++++++----------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/docs/search/optimize-search-performance.md b/docs/search/optimize-search-performance.md index 7f9f8eda28..7ff0f006bd 100644 --- a/docs/search/optimize-search-performance.md +++ b/docs/search/optimize-search-performance.md @@ -96,13 +96,13 @@ sourceCategory=foo and field_a=value_a ### Move terms from parse statement to source expression -Adding the parsing terms in the source expression will help you enhance the search performance. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse “completed * action“ as actionName` will remove logs that do not have **completed** and **action** terms. +Adding the parsing terms in the source expression will help you enhance the search performance. A parse statement without `nodrop` drops the logs that could not parse the desired field. For example, `parse "completed * action" as actionName` will remove logs that do not have **completed** and **action** terms. **Not recommended approach:** ``` _sourceCategory=Prod/User/Eventlog -| parse “completed * action“ as actionName +| parse "completed * action" as actionName | count by actionName ``` @@ -110,7 +110,7 @@ _sourceCategory=Prod/User/Eventlog ``` _sourceCategory=Prod/User/Eventlog completed action -| parse “completed * action“ as actionName +| parse "completed * action" as actionName | count by actionName ``` @@ -122,7 +122,7 @@ While filtering the date, reduce the result set to the smallest possible size be ``` _sourceCategory=Prod/User/Eventlog -| parse “userName: *, “ as user +| parse "userName: *, " as user | count by user | where user="john" ``` @@ -131,7 +131,7 @@ _sourceCategory=Prod/User/Eventlog ``` _sourceCategory=Prod/User/Eventlog userName -| parse “userName: *, “ as user +| parse "userName: *, " as user | where user="john" | count by user ``` @@ -146,8 +146,8 @@ For example, let’s say you have a `sort` operator before an aggregation, but t ``` _sourceCategory=Prod/User/Eventlog -| parse “userName: *, “ as user -| parse “evenName: *, “ as event +| parse "userName: *, " as user +| parse "evenName: *, " as event | count by user ``` @@ -155,7 +155,7 @@ _sourceCategory=Prod/User/Eventlog ``` _sourceCategory=Prod/User/Eventlog -| parse “userName: *, “ as user +| parse "userName: *, " as user | count by user ``` @@ -169,8 +169,8 @@ If the same operators are used multiple times in different levels of query, if p ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action“ as actionName - | parse “action in * ms“ as duration + | parse "completed * action" as actionName + | parse "action in * ms" as duration | pct(duration, 95) by actionName ``` @@ -178,7 +178,7 @@ If the same operators are used multiple times in different levels of query, if p ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action in * ms“ as actionName, duration + | parse "completed * action in * ms" as actionName, duration | pct(duration, 95) by actionName ``` @@ -188,17 +188,17 @@ If the same operators are used multiple times in different levels of query, if p ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action“ as actionName - | where toLowerCase(actionName) = “logIn” or toLowerCase(actionName) matches “abc*” or toLowerCase(actionName) contains “xyz" + | parse "completed * action" as actionName + | where toLowerCase(actionName) = "logIn” or toLowerCase(actionName) matches "abc*” or toLowerCase(actionName) contains "xyz" ``` **Recommended approach:** ``` _sourceCategory=Prod/User/Eventlog - | parse “completed * action“ as actionName + | parse "completed * action" as actionName | toLowerCase(actionName) as actionNameLowered - | where actionNameLowered = “logIn” or actionNameLowered matches “abc*” or actionNameLowered contains “xyz” + | where actionNameLowered = "logIn” or actionNameLowered matches "abc*” or actionNameLowered contains "xyz” ``` ### Use lookup on the lowest possible dataset @@ -212,9 +212,9 @@ Minimize the data processed by the `lookup` operator in the query, as lookup is ``` _sourceCategory=Prod/User/Eventlog -| parse “completed * action in * ms“ as actionName, duration +| parse "completed * action in * ms" as actionName, duration | lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName -| where actionName in (“login”, “logout”) +| where actionName in ("login”, "logout”) | count by actionName, actionType ``` @@ -222,8 +222,8 @@ _sourceCategory=Prod/User/Eventlog ``` _sourceCategory=Prod/User/Eventlog -| parse “completed * action in * ms“ as actionName, duration -| where actionName in (“login”, “logout”) +| parse "completed * action in * ms" as actionName, duration +| where actionName in ("login”, "logout”) | count by actionName | lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName ``` @@ -232,8 +232,8 @@ _sourceCategory=Prod/User/Eventlog ``` _sourceCategory=Prod/User/Eventlog -| parse “completed * action in * ms“ as actionName, duration -| where actionName in (“login”, “logout”) +| parse "completed * action in * ms" as actionName, duration +| where actionName in ("login”, "logout”) | lookup actionType from path://"/Library/Users/myusername@sumologic.com/actionTypes" on actionName | count by actionName, actionType ``` @@ -246,15 +246,15 @@ For example, consider the below query where the assumption is that a single log ``` _sourceCategory=Prod/User/Eventlog -| parse regex “userName: (?[a-z-A-Z]+), “ multi -| parse regex “eventName: (?[a-z-A-Z]+), “ multi +| parse regex "userName: (?[a-z-A-Z]+), " multi +| parse regex "eventName: (?[a-z-A-Z]+), " multi ``` But if you write the query like that, it will generate a result for every combination of `userName` and `eventName` values. Now suppose you want to count by `eventName`, it will not give you the desired result, since a single `eventName` has been duplicated for every `userName` in the same log. So, the better query would be: ``` _sourceCategory=Prod/User/Eventlog -| parse regex “userName: (?[a-z-A-Z]+), eventName: (?[a-z-A-Z]+), “ multi +| parse regex "userName: (?[a-z-A-Z]+), eventName: (?[a-z-A-Z]+), " multi ```