Skip to content
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
92be52d
First draft
jpipkin1 Oct 17, 2024
ba2b97c
Small updates
jpipkin1 Oct 17, 2024
b080c63
Add section on obtaining normalization data
jpipkin1 Oct 17, 2024
2c69ea9
Add links
jpipkin1 Oct 18, 2024
03d7f6e
Merge branch 'main' into docs-448-normalization
jpipkin1 Oct 23, 2024
fe6019d
Update docs/cse/records-signals-entities-insights/configure-entity-lo…
jpipkin1 Oct 29, 2024
40a9c51
Update vmware-vrealize-log-insight.md (#4681)
JV0812 Oct 24, 2024
ff2ac5b
Azure Flexible DB for MySQL Document (#4665)
sachin-sumologic Oct 25, 2024
4282de2
Usage management - Advanced tab information added (#4641)
JV0812 Oct 25, 2024
5740bb2
OpenTelemetry Remote Management V2 (#4482)
JV0812 Oct 25, 2024
254210a
Azure web app server farm doc changes (#4684)
ankurch627 Oct 25, 2024
1cd38d6
SUMO-248960 | We got to know that when collecting metrics via mongos …
sumoanema Oct 25, 2024
8a26e24
Scan Budgets Release notes_Pull back (#4696)
JV0812 Oct 25, 2024
be4b5b3
Update API intro reuse file (#4691)
jpipkin1 Oct 25, 2024
0b7c0b9
Zendesk App doc (#4689)
JV0812 Oct 25, 2024
35bd57d
LastPass app docs (#4690)
JV0812 Oct 25, 2024
d67277f
Change release note dates for LastPass and Zendesk (#4698)
jpipkin1 Oct 25, 2024
24f543d
DOCS-206 - Deprecate Root Cause Explorer (#4697)
jpipkin1 Oct 25, 2024
b001828
Update the app docs with version revert info (#4507)
JV0812 Oct 28, 2024
9ddbf05
Update minute-volume.md (#4692)
JV0812 Oct 28, 2024
142eca6
Changes in oracle otel app for addition of metric collection, metric …
sumoanema Oct 28, 2024
a3cf299
Added the new field (#4704)
rishav-sumo-dev Oct 28, 2024
759fcf9
CrowdStrike Spotlight app docs (#4695)
JV0812 Oct 29, 2024
9ddea3b
Update aws-cloudtrail.md (#4703)
JV0812 Oct 29, 2024
9f1b84b
Added "Show optimization tips" button information (#4683)
JV0812 Oct 29, 2024
318743b
Rename image file
jpipkin1 Oct 29, 2024
3eddc60
Test
jpipkin1 Oct 29, 2024
6382f8e
Cleanup
jpipkin1 Oct 29, 2024
72964a1
Add borders to images
jpipkin1 Oct 31, 2024
2801173
Merge branch 'main' into docs-448-normalization
jpipkin1 Oct 31, 2024
666da8a
Entity looup table cleanup
jpipkin1 Oct 31, 2024
0e2aa9e
Update docs/cse/administration/save-inventory-data-lookup-table.md
jpipkin1 Nov 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/cse/administration/save-inventory-data-lookup-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Although the Scheduled Search feature does support an **Alert Type** of “Save

## Prerequisites 

In order to create an inventory Lookup Table you need to have one or more sources of inventory data. One of the most common sources of inventory data in Sumo Logic is the Windows Active Directory Inventory source running on an Installed Collector. We recommend you collect AD logs every 12 hours, and that you do not collect logs more frequently than every 8 hours.
In order to create an inventory Lookup Table you need to have one or more sources of inventory data. One of the most common sources of inventory data in Sumo Logic is the [Windows Active Directory Inventory source](/docs/send-data/installed-collectors/sources/windows-active-directory-inventory-source/) running on an Installed Collector. We recommend you collect AD logs every 12 hours, and that you do not collect logs more frequently than every 8 hours.

Any inventory source–or any log source, for that matter–can be used to populate Lookup Tables. Sumo Logic also has a variety of inventory sources that run on Hosted Collectors, including the Okta and Carbon Black sources.

Expand All @@ -39,14 +39,14 @@ To create the Lookup Table schema:

1. Go to the Sumo Logic Library.
1. Navigate to the folder where you want to create the Lookup Table.
1. Click **Add New** and then select **New Lookup**. <br/><img src={useBaseUrl('img/cse/new-lookup.png')} alt="New lookup link" width="600"/>
1. The **Create Lookup Table** page appears. <br/><img src={useBaseUrl('img/cse/create-in-cip.png')} alt="Create lookup table" width="600"/>
1. Click **Add New** and then select **New Lookup**. <br/><img src={useBaseUrl('img/cse/new-lookup.png')} alt="New lookup link" style={{border: '1px solid gray'}} width="600"/>
1. The **Create Lookup Table** page appears. <br/><img src={useBaseUrl('img/cse/create-in-cip.png')} alt="Create lookup table" style={{border: '1px solid gray'}} width="600"/>
1. **Name**. Enter a name for the Lookup Table.
1. **Description**. (Optional)
1. **Set a TTL (Time to Live) for table entries**? Click **No**.
1. **Choose a size limit handling option**. This option controls how additions to the Lookup table will be handled when it reaches its size limit (100 MB). Click **Delete Old Data.**
1. **Create Lookup Table** Click **Create Schema only**.
1. The page displays a **Schema** section. (The screenshot below shows the schema settings for our example filled in.) <br/><img src={useBaseUrl('img/cse/schema.png')} alt="Schema settings" width="600"/>
1. The page displays a **Schema** section. (The screenshot below shows the schema settings for our example filled in.) <br/><img src={useBaseUrl('img/cse/schema.png')} alt="Schema settings" style={{border: '1px solid gray'}} width="600"/>
1. For the first column, enter:
* **Fields**. Enter *mail*.
* **Value Type**. Leave the default, *string*, selected.
Expand Down Expand Up @@ -75,7 +75,7 @@ Where:
* `_collector` identifies the collector where the Active Directory source runs. 
* `PATH` is the path of the lookup table, in this format: `path://"/Library/Admin Recommended/userIdToUsername"` You can copy the path to the Lookup Table in the Sumo Logic Library. Hover over the row for the table in the Library, and select **Copy path to clipboard** from the three-dot kebab menu.

<img src={useBaseUrl('img/cse/tree-dot.png')} alt="Kebab menu button" width="600"/>
<img src={useBaseUrl('img/cse/tree-dot.png')} alt="Kebab menu button" style={{border: '1px solid gray'}} width="600"/>

## Step 3: Save and schedule the search

Expand All @@ -85,21 +85,21 @@ Be sure to choose “Email” as the **Alert type**. (*Don’t* select **Save to

To save and schedule the search:

1. In the log search tab where you’ve run your query, choose **Save as** from the three-dot kebab menu in the query area. <br/><img src={useBaseUrl('img/cse/save-as.png')} alt="Save as on dropdown list" width="600"/>
1. In the log search tab where you’ve run your query, choose **Save as** from the three-dot kebab menu in the query area. <br/><img src={useBaseUrl('img/cse/save-as.png')} alt="Save as on dropdown list" style={{border: '1px solid gray'}} width="600"/>
1. On the **Save Item** popup:
* **Name**. Enter a name for the query.
* **Time range**. Select a time range for the query.
* **Search By**. Select *Receipt Time*.  
* **Location to save to**. Choose a folder location.
* Click **Schedule this search**. <br/><img src={useBaseUrl('img/cse/save-item.png')} alt="Save item dialog" width="400"/>
* Click **Schedule this search**. <br/><img src={useBaseUrl('img/cse/save-item.png')} alt="Save item dialog" style={{border: '1px solid gray'}} width="400"/>
1. On the **Save Item** popup:
* **Run frequency**. Select *Daily*, unless you have another preference.
* **Send Notification**. Choose *If the following condition is met*.
* **Alert condition**. Select *Less than \<*.
* **Alert type**. Select *Email*.
* **Number of results**. Enter *5*, or another value if you prefer.
* **Recipients.** Enter the email addresses of one or more users to receive email alerts.
* **Include in email**. Select *Search Query* and *Histogram*, unless you have another preference. <br/><img src={useBaseUrl('img/cse/save-item-2.png')} alt="Save item dialog" width="400"/>
* **Include in email**. Select *Search Query* and *Histogram*, unless you have another preference. <br/><img src={useBaseUrl('img/cse/save-item-2.png')} alt="Save item dialog" style={{border: '1px solid gray'}} width="400"/>
1. Click **Save.**

## Step 4: Configure the Lookup Table in Cloud SIEM
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,25 +26,34 @@ In addition, in some systems a user or a host has both a name and a unique ID, t
* `d8ece0f8-10a4-3c62-b8a3-2e636a3a0509`
* `testk-122.testlabs.local`

Multiple identifiers for the same user or host are a problem when it comes to correlating Signals around a common Entity: unless you allow for all permutations of a username or hostname, your rule or search won’t function as intended with all data sources.
Multiple identifiers for the same user or host are a problem when it comes to correlating Signals around a common Entity. Unless you allow for all permutations of a username or hostname, your rule or search won’t function as intended with all data sources.

### Examples of when you create Lookup Tables

Following are some examples of situations when you'd want to use Entity Lookup Tables:
* CrowdStrike FDR data uses an agent ID (AID) instead of a hostname for some messages.
* Mail Transfer Agent (MTA) systems report usernames in an email format.
* Your users have different login names on different systems (for example, Windows, Linux, and AWS).

### How does an Entity Lookup Table work?

An Entity Lookup Table defines two sets of values: a lookup value to look for in an incoming message and a substitution value. You can create Entity Lookup Tables to support the following types of normalization:

* **Host ID to Normalized Hostname**
* **User ID to Normalized Username**
* **Username to Normalized Username**

Entity Lookup Tables are based on Sumo Logic’s Lookup Tables feature. Here is an example of a **Host ID to Normalized Hostname** Lookup Table in the Sumo Logic Library:
Entity Lookup Tables are based on Sumo Logic’s [Lookup Tables](/docs/search/lookup-tables/) feature. Here is an example of a **Host ID to Normalized Hostname** Lookup Table in the Sumo Logic Library:

<img src={useBaseUrl('img/cse/example-table.png')} alt="Example Entity lookup table" style={{border: '1px solid gray'}} width="800"/>

## Limitations
## Creating a Lookup Table

You can configure a maximum of five Entity Lookup Tables
Before you configure a Lookup Table in Cloud SIEM, you must [create the Lookup Table](/docs/search/lookup-tables/create-lookup-table/) in the Sumo Logic platform. There are a variety of ways to create a Lookup Table

## Creating a Lookup Table
### Limitations

Before you configure a Lookup Table in Cloud SIEM, you must create the Lookup Table in the Sumo Logic platform. There are a variety of ways to create a Lookup Table
You can configure a maximum of five Entity Lookup Tables

### Populate table from inventory data

Expand All @@ -56,11 +65,11 @@ This method–the typical way to populate a Lookup Table for the purpose of Enti

If you already have a Lookup Table that contains normalization data, you can configure it in Cloud SIEM. Or, if you have existing normalization data that is not currently in a Lookup Table you can create a Lookup Table with that data. Note that your Lookup Table must contain a field that contains a lookup value and one that contains a substitution value. There is no requirement for particular column names.

For instructions, see the Create a Lookup Table topic. After creating the table, perform the steps in [Configure the Lookup Table in Cloud SIEM](#configure-the-lookup-table-in-cloud-siem), below.
For instructions, see [Create a Lookup Table](/docs/search/lookup-tables/create-lookup-table/). After creating the table, perform the steps in [Configure the Lookup Table in Cloud SIEM](#configure-the-lookup-table-in-cloud-siem), below.

### Configure the Lookup Table in Cloud SIEM

After youve created your Entity Lookup Table in the Sumo Logic Library, you can configure it in Cloud SIEM.
After you've [created your Entity Lookup Table](/docs/search/lookup-tables/create-lookup-table/) in the Sumo Logic Library, you can configure it in Cloud SIEM.

1. [**Classic UI**](/docs/cse/introduction-to-cloud-siem/#classic-ui). In the top menu select **Configuration**, and then under **Entities** select **Normalization**. <br/>[**New UI**](/docs/cse/introduction-to-cloud-siem/#new-ui). In the top menu select **Configuration**, and then under **Cloud SIEM Entities** select **Normalization**. You can also click the **Go To...** menu at the top of the screen and select **Normalization**.
1. On the **Entity Normalization** page, click **Lookup Tables**.
Expand Down
4 changes: 2 additions & 2 deletions docs/cse/schema/record-processing-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,15 @@ The key-value pairs are input to the next step of the process: mapping.

## Map message fields to schema attributes

The mapping process creates a Record from the key-value pairs that were extracted from a message, and maps a subset of the keys to Cloud SIEM schema attributes. 
The [mapping process](/docs/cse/schema/create-structured-log-mapping/) creates a Record from the key-value pairs that were extracted from a message, and maps a subset of the keys to Cloud SIEM schema attributes. 

Mapping solves a particular problem: messages from different products use different names to identify users, applications, devices and so on. For example, some messages may refer to a source IP address as `sourceIP`, while others use `sourceIpAddress`. We need a standard set of names for the data that most messages are likely to contain. The [Cloud SIEM schema](/docs/cse/schema) defines that standard set of names. 

What’s the benefit of mapping? It results in Records that use a common (standard) name for fields that hold the same sort of data, regardless of the source of the incoming message. The result: the same Cloud SIEM rule can be applied to all Records, regardless of the message source.

## Normalize usernames and hostnames

Username and hostname normalization is the process of transforming the value of Record attributes that contain usernames and hostnames into a standard format. The normalized value replaces the non-normalized value in a Record. The non-normalized values of hostname and usernames are retained in a `_raw` field in the Record.
[Username and hostname normalization](/docs/cse/schema/username-and-hostname-normalization/) is the process of transforming the value of Record attributes that contain usernames and hostnames into a standard format. The normalized value replaces the non-normalized value in a Record. The non-normalized values of hostname and usernames are retained in a `_raw` field in the Record.

Why normalize? Assume Cloud SIEM receives messages with an email-type field "[email protected]" and username-type field  "bob". We can use normalization to transform "[email protected]" to "bob", allowing the username and email to be correlated together.

Expand Down
Loading
Loading