Skip to content

Commit f5b2546

Browse files
TrishaMcTrishaMc
authored
Merge pull request #49557 from bwren/la-language-cheatsheets
SQL and Splunk cheatsheets
2 parents 0d48b55 + d31a46e commit f5b2546

File tree

4 files changed

+322
-0
lines changed

4 files changed

+322
-0
lines changed

‎articles/log-analytics/query-language/splunk-cheatsheet.md

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
---
2+
title: Splunk to Azure Log Analytics | Microsoft Docs
3+
description: Assist for users who are familiar with Splunk in learning the Log Analytics query language.
4+
services: log-analytics
5+
documentationcenter: ''
6+
author: bwren
7+
manager: carmonm
8+
editor: ''
9+
ms.assetid:
10+
ms.service: log-analytics
11+
ms.workload: na
12+
ms.tgt_pltfrm: na
13+
ms.devlang: na
14+
ms.topic: conceptual
15+
ms.date: 08/21/2018
16+
ms.author: bwren
17+
ms.component: na
18+
---
19+
20+
# Splunk to Log Analytics
21+
22+
This article is intended to assist users who are familiar with Splunk in learning the Log Analytics query language. Direct comparisons are made between the two to understand key differences and also similarities where you can leverage your existing knowledge.
23+
24+
## Structure and concepts
25+
26+
The following table compares concepts and data structures between Splunk and Log Analytics.
27+
28+
| Concept | Splunk | Log Analytics | Comment
29+
| --- | --- | --- | ---
30+
| Deployment unit | cluster | cluster | Log Analytics allows arbitrary cross cluster queries. Splunk does not. |
31+
| Data caches | buckets | Caching and retention policies | Controls the period and caching level for the data. This setting directly impacts the performance of queries and cost of the deployment. |
32+
| Logical partition of data | index | database | Allows logical separation of the data. Both implementations allow unions and joining across these partitions. |
33+
| Structured event metadata | N/A | table | Splunk does not have the concept exposed to the search language of event metadata. Log Analytics has the concept of a table, which has columns. Each event instance is mapped to a row. |
34+
| Data record | event | row | Terminology change only. |
35+
| Data record attribute | field | column | In Log Analytics, this is predefined as part of the table structure. In Splunk, each event has its own set of fields. |
36+
| Types | datatype | datatype | Log Analytics datatypes are more explicit as they are set on the columns. Both have the ability to work dynamically with data types and roughly equivalent set of datatypes including JSON support. |
37+
| Query and search | search | query | Concepts are essentially the same between both Log Analytics and Splunk. |
38+
| Event ingestion time | System Time | ingestion_time() | In Splunk, each event gets a system timestamp of the time that the event was indexed. In Log Analytics, you can define a policy called ingestion_time that exposes a system column that can be referenced through the ingestion_time() function. |
39+
40+
## Functions
41+
42+
The following table specifies functions in Log Analytics that are equivalent to Splunk functions.
43+
44+
|Splunk | Log Analytics |Comment
45+
|---|---|---
46+
|strcat | strcat()| (1) |
47+
|split | split() | (1) |
48+
|if | iff() | (1) |
49+
|tonumber | todouble()<br>tolong()<br>toint() | (1) |
50+
|upper<br>lower |toupper()<br>tolower()|(1) |
51+
| replace | replace() | (1)<br> Also note that while `replace()` takes three parameters in both products, the parameters are different. |
52+
| substr | substring() | (1)<br>Also note that Splunk uses one-based indices. Log Analytics notes zero-based indices. |
53+
| tolower | tolower() | (1) |
54+
| toupper | toupper() | (1) |
55+
| match | matches regex | (2) |
56+
| regex | matches regex | In Splunk, `regex` is an operator. In Log Analytics, it's a relational operator. |
57+
| searchmatch | == | In Splunk, `searchmatch` allows searching for the exact string.
58+
| random | rand()<br>rand(n) | Splunk's function returns a number from zero to 2<sup>31</sup>-1. Log Analytics' returns a number between 0.0 and 1.0, or if a parameter provided, between 0 and n-1.
59+
| now | now() | (1)
60+
| relative_time | totimespan() | (1)<br>In Log Analytics, Splunk's equivalent of relative_time(datetimeVal, offsetVal) is datetimeVal + totimespan(offsetVal).<br>For example, <code>search &#124; eval n=relative_time(now(), "-1d@d")</code> becomes <code>... &#124; extend myTime = now() - totimespan("1d")</code>.
61+
62+
(1) In Splunk, the function is invoked with the `eval` operator. In Log Analytics, it is used as part of `extend` or `project`.<br>(2) In Splunk, the function is invoked with the `eval` operator. In Log Analytics, it can be used with the `where` operator.
63+
64+
65+
## Operators
66+
67+
The following sections give examples of using different operators between Splunk and Log Analytics.
68+
69+
> [!NOTE]
70+
> For the purpose of the following example, the Splunk field _rule_ maps to a table in Azure Log Analytics, and Splunk's default timestamp maps to the Logs Analytics _ingestion_time()_ column.
71+
72+
### Search
73+
In Splunk, you can omit the `search` keyword and specify an unquoted string. In Azure Log Analytics you must start each search with `find`, an unquoted string is a column name, and the lookup value must be a quoted string.
74+
75+
| | | |
76+
|:---|:---|:---|
77+
| Splunk | **search** | <code>search Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" earliest=-24h</code> |
78+
| Log Analytics | **find** | <code>find Session.Id=="c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time()> ago(24h)</code> |
79+
| | |
80+
81+
### Filter
82+
Azure Log Analytics queries start from a tabular result set where the filter. In Splunk, filtering is the default operation on the current index. You can also use `where` operator in Splunk, but it is not recommended.
83+
84+
| | | |
85+
|:---|:---|:---|
86+
| Splunk | **search** | <code>Event.Rule="330009.2" Session.Id="c8894ffd-e684-43c9-9125-42adc25cd3fc" _indextime>-24h</code> |
87+
| Log Analytics | **where** | <code>Office_Hub_OHubBGTaskError<br>&#124; where Session_Id == "c8894ffd-e684-43c9-9125-42adc25cd3fc" and ingestion_time() > ago(24h)</code> |
88+
| | |
89+
90+
91+
### Getting n events/rows for inspection
92+
Azure Log Analytics also supports `take` as an alias to `limit`. In Splunk, if the results are ordered, `head` will return the first n results. In Azure Log Analytics, limit is not ordered but returns the first n rows that are found.
93+
94+
| | | |
95+
|:---|:---|:---|
96+
| Splunk | **head** | <code>Event.Rule=330009.2<br>&#124; head 100</code> |
97+
| Log Analytics | **limit** | <code>Office_Hub_OHubBGTaskError<br>&#124; limit 100</code> |
98+
| | |
99+
100+
101+
102+
### Getting the first n events/rows ordered by a field/column
103+
For bottom results, in Splunk you use `tail`. In Azure Log Analytics you can specify the ordering direction with `asc`.
104+
105+
| | | |
106+
|:---|:---|:---|
107+
| Splunk | **head** | <code>Event.Rule="330009.2"<br>&#124; sort Event.Sequence<br>&#124; head 20</code> |
108+
| Log Analytics | **top** | <code>Office_Hub_OHubBGTaskError<br>&#124; top 20 by Event_Sequence</code> |
109+
| | |
110+
111+
112+
113+
114+
### Extending the result set with new fields/columns
115+
Splunk also has an `eval` function, which is not to be comparable with the `eval` operator. Both the `eval` operator in Splunk and the `extend` operator in Azure Log Analytics only support scalar functions and arithmetic operators.
116+
117+
| | | |
118+
|:---|:---|:---|
119+
| Splunk | **eval** | <code>Event.Rule=330009.2<br>&#124; eval state= if(Data.Exception = "0", "success", "error")</code> |
120+
| Log Analytics | **extend** | <code>Office_Hub_OHubBGTaskError<br>&#124; extend state = iif(Data_Exception == 0,"success" ,"error")</code> |
121+
| | |
122+
123+
124+
### Rename
125+
Azure Log Analytics uses the same operator to rename and to create a new field. Splunk has two separate operators, `eval` and `rename`.
126+
127+
| | | |
128+
|:---|:---|:---|
129+
| Splunk | **rename** | <code>Event.Rule=330009.2<br>&#124; rename Date.Exception as execption</code> |
130+
| Log Analytics | **extend** | <code>Office_Hub_OHubBGTaskError<br>&#124; extend exception = Date_Exception</code> |
131+
| | |
132+
133+
134+
135+
136+
### Format results/Projection
137+
Splunk does not seem to have an operator similar to `project-away`. You can use the UI to filter away fields.
138+
139+
| | | |
140+
|:---|:---|:---|
141+
| Splunk | **table** | <code>Event.Rule=330009.2<br>&#124; table rule, state</code> |
142+
| Log Analytics | **project**<br>**project-away** | <code>Office_Hub_OHubBGTaskError<br>&#124; project exception, state</code> |
143+
| | |
144+
145+
146+
147+
### Aggregation
148+
See the [Aggregations in Log Analytics queries](aggregations.md) for the different aggregation functions.
149+
150+
| | | |
151+
|:---|:---|:---|
152+
| Splunk | **stats** | <code>search (Rule=120502.*)<br>&#124; stats count by OSEnv, Audience</code> |
153+
| Log Analytics | **summarize** | <code>Office_Hub_OHubBGTaskError<br>&#124; summarize count() by App_Platform, Release_Audience</code> |
154+
| | |
155+
156+
157+
158+
### Join
159+
Join in Splunk has significant limitations. The subquery has a limit of 10000 results (set in the deployment configuration file), and there a limited number of join flavors.
160+
161+
| | | |
162+
|:---|:---|:---|
163+
| Splunk | **join** | <code>Event.Rule=120103* &#124; stats by Client.Id, Data.Alias | join Client.Id max=0 [search earliest=-24h Event.Rule="150310.0" Data.Hresult=-2147221040]</code> |
164+
| Log Analytics | **join** | <code>cluster("OAriaPPT").database("Office PowerPoint").Office_PowerPoint_PPT_Exceptions<br>&#124; where Data_Hresult== -2147221040<br>&#124; join kind = inner (Office_System_SystemHealthMetadata<br>&#124; summarize by Client_Id, Data_Alias)on Client_Id</code> |
165+
| | |
166+
167+
168+
169+
### Sort
170+
In Splunk, to sort in ascending order you must use the `reverse` operator. Azure Log Analytics also supports defining where to put nulls, at the beginning or at the end.
171+
172+
| | | |
173+
|:---|:---|:---|
174+
| Splunk | **sort** | <code>Event.Rule=120103<br>&#124; sort Data.Hresult<br>&#124; reverse</code> |
175+
| Log Analytics | **order by** | <code>Office_Hub_OHubBGTaskError<br>&#124; order by Data_Hresult, desc</code> |
176+
| | |
177+
178+
179+
180+
### Multivalue expand
181+
This is a similar operator in both Splunk and Log Analytics.
182+
183+
| | | |
184+
|:---|:---|:---|
185+
| Splunk | **mvexpand** | `mvexpand foo` |
186+
| Log Analytics | **mvexpand** | `mvexpand foo` |
187+
| | |
188+
189+
190+
191+
192+
### Results facets, interesting fields
193+
In the Log Analytics portal, only the first column is exposed. All columns are available through the API.
194+
195+
| | | |
196+
|:---|:---|:---|
197+
| Splunk | **fields** | <code>Event.Rule=330009.2<br>&#124; fields App.Version, App.Platform</code> |
198+
| Log Analytics | **facets** | <code>Office_Excel_BI_PivotTableCreate<br>&#124; facet by App_Branch, App_Version</code> |
199+
| | |
200+
201+
202+
203+
204+
### De-duplicate
205+
You can use `summarize arg_min()` instead to reverse the order of which record gets chosen.
206+
207+
| | | |
208+
|:---|:---|:---|
209+
| Splunk | **dedup** | <code>Event.Rule=330009.2<br>&#124; dedup device_id sortby -batterylife</code> |
210+
| Log Analytics | **summarize arg_max()** | <code>Office_Excel_BI_PivotTableCreate<br>&#124; summarize arg_max(batterylife, *) by device_id</code> |
211+
| | |
212+
213+
214+
215+
216+
## Next steps
217+
218+
- Go through a lesson on the [writing queries in Log Analytics](get-started-queries.md).

‎articles/log-analytics/query-language/sql-cheatsheet.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
title: SQL to Azure Log Analytics query language cheat sheet | Microsoft Docs
3+
description: Common functions to use for different scenarios in Log Analytics queries.
4+
services: log-analytics
5+
documentationcenter: ''
6+
author: bwren
7+
manager: carmonm
8+
editor: ''
9+
ms.assetid:
10+
ms.service: log-analytics
11+
ms.workload: na
12+
ms.tgt_pltfrm: na
13+
ms.devlang: na
14+
ms.topic: conceptual
15+
ms.date: 08/21/2018
16+
ms.author: bwren
17+
ms.component: na
18+
---
19+
20+
# SQL to Log Analytics query language cheat sheet
21+
22+
The table below helps users who are familiar with SQL to learn the Log Analytics query language. Have a look at the T-SQL command for solving a common scenarios and the equivalent using Log Analytics.
23+
24+
## SQL to Log Analytics
25+
26+
Description |SQL Query |Azure Log Analytics Query
27+
----------------------------------------|---------------------------------------------------------------------------------------------------|----------------------------------------
28+
Select all data from a table |`SELECT * FROM dependencies` |<code>dependencies</code>
29+
Select specific columns from a table |`SELECT name, resultCode FROM dependencies` |<code>dependencies <br>&#124; project name, resultCode</code>
30+
Select 100 records from a table |`SELECT TOP 100 * FROM dependencies` |<code>dependencies <br>&#124; take 100</code>
31+
Null evaluation |`SELECT * FROM dependencies WHERE resultCode IS NOT NULL` |<code>dependencies <br>&#124; where isnotnull(resultCode)</code>
32+
String comparison: equality |`SELECT * FROM dependencies WHERE name = "abcde"` |<code>dependencies <br>&#124; where name == "abcde"</code>
33+
String comparison: substring |`SELECT * FROM dependencies WHERE like "%bcd%"` |<code>dependencies <br>&#124; where name contains "bcd"</code>
34+
String comparison: wildcard |`SELECT * FROM dependencies WHERE name like "abc%"` |<code>dependencies <br>&#124; where name startswith "abc"</code>
35+
Date comparison: last 1 day |`SELECT * FROM dependencies WHERE timestamp > getdate()-1` |<code>dependencies <br>&#124; where timestamp > ago(1d)</code>
36+
Date comparison: date range |`SELECT * FROM dependencies WHERE timestamp BETWEEN '2016-10-01' AND '2016-11-01'` |<code>dependencies <br>&#124; where timestamp between (datetime(2016-10-01) .. datetime(2016-10-01))</code>
37+
Boolean comparison |`SELECT * FROM dependencies WHERE !(success)` |<code>dependencies <br>&#124; where success == "False" </code>
38+
Sort |`SELECT name, timestamp FROM dependencies ORDER BY timestamp asc` |<code>dependencies <br>&#124; order by timestamp asc </code>
39+
Distinct |`SELECT DISTINCT name, type FROM dependencies` |<code>dependencies <br>&#124; summarize by name, type </code>
40+
Grouping, Aggregation |`SELECT name, AVG(duration) FROM dependencies GROUP BY name` |<code>dependencies <br>&#124; summarize avg(duration) by name </code>
41+
Column aliases, Extend |`SELECT operation_Name as Name, AVG(duration) as AvgD FROM dependencies GROUP BY name` |<code>dependencies <br>&#124; summarize AvgD=avg(duration) by operation_Name <br>&#124; project Name=operation_Name, AvgD</code>
42+
Top n recrods by measure |`SELECT TOP 100 name, COUNT(*) as Count FROM dependencies GROUP BY name ORDER BY Count asc` |<code>dependencies <br>&#124; summarize Count=count() by name <br>&#124; top 100 by Count asc</code>
43+
Union |`SELECT * FROM dependencies UNION SELECT * FROM exceptions` |<code>union dependencies, exceptions</code>
44+
Union: with conditions |`SELECT * FROM dependencies WHERE value > 4 UNION SELECT * FROM exceptions value < 5` |<code>dependencies <br>&#124; where value > 4 <br>&#124; union (exceptions <br>&#124; where value < 5)</code>
45+
Join |`SELECT * FROM dependencies JOIN exceptions ON dependencies.operation_Id = exceptions.operation_Id`|<code>dependencies <br>&#124; join (exceptions) on operation_Id == operation_Id</code>
46+
47+
48+
## Next steps
49+
50+
- Go through a lesson on the [writing queries in Log Analytics](get-started-queries.md).

‎articles/log-analytics/query-language/useful-operators.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: Useful operators in Azure Log Analytics queries | Microsoft Docs
3+
description: Common functions to use for different scenarios in Log Analytics queries.
4+
services: log-analytics
5+
documentationcenter: ''
6+
author: bwren
7+
manager: carmonm
8+
editor: ''
9+
ms.assetid:
10+
ms.service: log-analytics
11+
ms.workload: na
12+
ms.tgt_pltfrm: na
13+
ms.devlang: na
14+
ms.topic: conceptual
15+
ms.date: 08/21/2018
16+
ms.author: bwren
17+
ms.component: na
18+
---
19+
20+
# Useful operators in Log Analytics queries
21+
22+
The table below provides some common functions to use for different scenarios in Log Analytics queries.
23+
24+
## Useful operators
25+
26+
Category |Relevant Analytics Function
27+
----------------------------------------|----------------------------------------
28+
Selection and Column aliases |`project`, `project-away`, `extend`
29+
Temporary tables and constants |`let scalar_alias_name = …;` <br> `let table_alias_name = (){ … &#124; … &#124; … };`
30+
Comparison and String Operators |`startswith`, `!startswith`, `has`, `!has` <br> `contains`, `!contains`, `containscs` <br> `hasprefix`, `!hasprefix`, `hassuffix`, `!hassuffix`, `in`, `!in` <br> `matches regex` <br> `==`, `=~`, `!=`, `!~`
31+
Common string functions |`strcat()`, `replace()`, `tolower()`, `toupper()`, `substring()`, `strlen()`
32+
Common math functions |`sqrt()`, `abs()` <br> `exp()`, `exp2()`, `exp10()`, `log()`, `log2()`, `log10()`, `pow()` <br> `gamma()`, `gammaln()`
33+
Parsing text |`extract()`, `extractjson()`, `parse`, `split()`
34+
Limiting output |`take`, `limit`, `top`, `sample`
35+
Date functions |`now()`, `ago()` <br> `datetime()`, `datepart()`, `timespan` <br> `startofday()`, `startofweek()`, `startofmonth()`, `startofyear()` <br> `endofday()`, `endofweek()`, `endofmonth()`, `endofyear()` <br> `dayofweek()`, `dayofmonth()`, `dayofyear()` <br> `getmonth()`, `getyear()`, `weekofyear()`, `monthofyear()`
36+
Grouping and aggregation |`summarize by` <br> `max()`, `min()`, `count()`, `dcount()`, `avg()`, `sum()` <br> `stddev()`, `countif()`, `dcountif()`, `argmax()`, `argmin()` <br> `percentiles()`, `percentile_array()`
37+
Joins and Unions |`join kind=leftouter`, `inner`, `rightouter`, `fullouter`, `leftanti` <br> `union`
38+
Sort, order |`sort`, `order`
39+
Dynamic object (JSON and array) |`parsejson()` <br> `makeset()`, `makelist()` <br> `split()`, `arraylength()` <br> `zip()`, `pack()`
40+
Logical operators |`and`, `or`, `iff(condition, value_t, value_f)` <br> `binary_and()`, `binary_or()`, `binary_not()`, `binary_xor()`
41+
Machine learning |`evaluate autocluster`, `basket`, `diffpatterns`, `extractcolumns`
42+
43+
44+
## Next steps
45+
46+
- Go through a lesson on the [writing queries in Log Analytics](get-started-queries.md).

‎articles/log-analytics/toc.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,8 @@
125125
href: query-language/get-started-queries.md
126126
- name: Search queries
127127
href: query-language/search-queries.md
128+
- name: Useful operators
129+
href: query-language/useful-operators.md
128130
- name: Lessons
129131
items:
130132
- name: String operations
@@ -143,6 +145,12 @@
143145
href: query-language/advanced-query-writing.md
144146
- name: Charts and diagrams
145147
href: query-language/charts.md
148+
- name: Cheatsheets
149+
items:
150+
- name: SQL
151+
href: query-language/sql-cheatsheet.md
152+
- name: Splunk
153+
href: query-language/splunk-cheatsheet.md
146154
- name: Cross-resource searches
147155
href: log-analytics-cross-workspace-search.md
148156
- name: Take action from search results

0 commit comments

Comments
 (0)