Skip to content

Commit 9014b62

Browse files
committed
fully cleaning up steps
1 parent fc00968 commit 9014b62

File tree

1 file changed

+15
-144
lines changed

1 file changed

+15
-144
lines changed

articles/sentinel/notebook-get-started.md

Lines changed: 15 additions & 144 deletions
Original file line numberDiff line numberDiff line change
@@ -52,161 +52,32 @@ This procedure describes how to launch your notebook with Microsoft Sentinel.
5252
- *Markdown* cells contain text and graphics with instructions for using the notebook
5353
- *Code* cells contain executable code that performs the notebook functions
5454

55-
1. At the top of the page, select your **Compute**, and then read and run the code cells in order using the instructions in the notebook. Skipping cells or running them out of order might cause errors later in the notebook.
55+
1. At the top of the page, select your **Compute**.
5656

57-
Depending on the compute you selected, it might take a few minutes to start. When the compute is ready, it'll show as **Ready** at the top of the tab.
57+
1. Continue by reading markdown cells and running code cells in order, using the instructions in the notebook. Skipping cells or running them out of order might cause errors later in the notebook.
5858

59-
:::image type="content" source="media/notebook-get-started/compute-ready.png" alt-text="Screenshot of a machine learning environment ready to run code cells.":::
60-
61-
Depending on the function being performed, the code in the cell might run quickly, or it might take a few seconds to complete. When the cell is running, the play button changes to a loading spinner, and the status is displayed at the bottom of the cell, together with the elapsed time.
62-
63-
Run the code cells in the **Introduction** section to learn the basics of running code in notebooks, and then run the code in the **Initializing the notebook and MSTICPy** section to set up your initial environment.
64-
65-
When initializing the notebook, configuration warnings about missing settings are expected because you didn't configure anything yet.
66-
67-
## Create your configuration file
68-
69-
After the basic initialization, you're ready to create your configuration file with basic settings for working with MSTICPy.
70-
71-
Many Microsoft Sentinel notebooks connect to external services such as [VirusTotal](https://www.virustotal.com) (VT) to collect and enrich data. To connect to these services you need to set and store configuration details, such as authentication tokens. Having this data in your configuration file avoids you having to type in authentication tokens and workspace details each time you use a notebook.
72-
73-
MSTICPy uses a **msticpyconfig.yaml** for storing a wide range of configuration details. By default, a **msticpyconfig.yaml** file is generated by the notebook initialization function. If you [cloned this notebook from the Microsoft Sentinel portal](#run-and-initialize-the-getting-started-guide-notebook), the configuration file is populated with Microsoft Sentinel workspace data. This data is read from a **config.json** file, created in the Azure Machine Learning workspace when you launch your notebook. For more information, see the [MSTICPy Package Configuration documentation](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html).
74-
75-
The following sections describe how to add more configuration details to the **msticpyconfig.yaml** file.
76-
77-
If you run the *Getting Started Guide* notebook again, and already have a minimally configured **msticpyconfig.yaml** file, the `init_notebook` function doesn't overwrite or modify your existing file.
78-
79-
At any point in time, select the **-Help** drop-down menu in the MSTICPy configuration tool for more instructions and links to detailed documentation.
80-
81-
### Display the MSTICPy settings editor
82-
83-
1. Add a code cell to your notebook, and run the following code to import the `MpConfigEdit` tool and display a settings editor for your **msticpyconfig.yaml** file:
84-
85-
```python
86-
from msticpy.config import MpConfigEdit
87-
88-
mpedit = MpConfigEdit( "msticpyconfig.yaml")
89-
mpedit.set_tab("AzureSentinel")
90-
display(mpedit)
91-
```
92-
93-
The automatically created **msticpyconfig.yaml** file is shown in the notebook in a series of editable tabs. In the **Microsoft Sentinel** tab, two entries are already populated with details of the Microsoft Sentinel workspace that the notebook was cloned from. One entry has the name of your workspace and the other is named **Default**.
94-
95-
MSTICPy allows you to store configurations for multiple Microsoft Sentinel workspaces and switch between them. The **Default** entry allows you to authenticate to your "home" workspace by default, without having to name it explicitly. If you add another workspaces, you can configure any one of them to be the **Default** entry.
96-
97-
In the Azure Machine Learning environment, the settings editor might take 10-20 seconds to appear.
98-
99-
1. Verify your current settings and select **Save Settings**.
100-
101-
### Add threat intelligence provider settings
102-
103-
This procedure describes how to store your [VirusTotal API key](#prerequisites) in the **msticpyconfig.yaml** file. You can opt to upload the API key to Azure Key Vault, but you must configure the Key Vault settings first. For more information, see [Configure Key Vault settings](#configure-key-vault-settings).
104-
105-
To add VirusTotal details in the MSTICPy settings editor, complete the following steps.
106-
107-
1. Add a new code cell and run the following:
108-
109-
```python
110-
mpedit.set_tab("TI Providers")
111-
mpedit
112-
```
113-
114-
1. In the **TI Providers** tab, from the **Add prov** dropdown, select **VirusTotal** > **Add**.
115-
116-
1. Under **Auth Key**, select **Text** next to the **Storage** option.
117-
118-
1. In the **Value** field, paste your API key.
119-
120-
1. Select **Update**, and then select **Save Settings** at the bottom of the settings editor.
121-
122-
For more information about other supported threat intelligence providers, see [Threat intelligence providers](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html) in the MSTICPy documentation and [Threat intelligence integration in Microsoft Sentinel](threat-intelligence-integration.md).
59+
Depending on the function being performed, the code in the cell might run quickly, or it might take some time to complete. When the cell is running, the play button changes to a loading spinner, and the status is displayed at the bottom of the cell, together with the elapsed time.
12360

124-
### Add GeoIP provider settings
61+
The first time you run a code cell, it may several a few minutes to start the session, depending on your compute settings. A **Ready** indication is shown when the notebook is ready to run code cells. For example:
12562

126-
This procedure describes how to store a [MaxMind GeoLite2 license key](#prerequisites) in the **msticpyconfig.yaml** file, which allows your notebook to use geolocation lookup services for IP addresses.
127-
128-
To add GeoIP provider settings in the MSTICPy settings editor, complete the following steps.
129-
130-
1. Add a new code cell and run the following:
131-
132-
```python
133-
mpedit.set_tab("GeoIP Providers")
134-
mpedit
135-
```
136-
137-
1. In the **GeoIP Providers** tab, from the **Add prov** dropdown, select **GeoIPLite** > **Add**.
138-
139-
1. In the **Value** field, enter your MaxMind license key.
140-
141-
1. If needed, update the default **~/.msticpy** folder for storing the downloaded GeoIP database.
142-
143-
- On Windows, this folder is mapped to the **%USERPROFILE%/.msticpy**.
144-
- On Linux or macOS, this path is mapped to the **.msticpy** folder in your home folder.
145-
146-
For more information about other supported geolocation lookup services, see the [MSTICPy GeoIP Providers documentation](https://msticpy.readthedocs.io/en/latest/data_acquisition/GeoIPLookups.html).
147-
148-
### Configure Azure Cloud settings
149-
150-
If your organization doesn't use the Azure public cloud, you must specify this in your settings to successfully authenticate and use data from Microsoft Sentinel and Azure. For more information, see [Specify the Azure Cloud and default Azure Authentication methods](#specify-the-azure-cloud-and-azure-authentication-methods).
151-
152-
### Validate settings
153-
154-
1. On any tab in the MSTICPy settings editor, select **Validate settings**.
155-
156-
Warning messages about missing configurations are expected, but you shouldn't have any for threat intelligence provider or GeoIP provider settings.
157-
158-
1. Depending on your environment, you might also need to [Configure Key Vault settings](#configure-key-vault-settings) or [Specify the Azure cloud](#specify-the-azure-cloud-and-azure-authentication-methods).
159-
160-
1. If you need to make any changes because of the validation, make those changes and then select **Save Settings**.
161-
162-
1. When you're done, select the **Close** button to hide the validation output.
163-
164-
For more information, see: [Advanced configurations for Jupyter notebooks and MSTICPy in Microsoft Sentinel](notebooks-msticpy-advanced.md)
165-
166-
## find a space for this
167-
168-
169-
- **Querying data from Microsoft Sentinel.** Use this section of the notebook to verify your Microsoft Sentinel settings in MSTICPy, load a QueryProvider to query data from Microsoft Sentinel, authenticate to Microsoft Sentinel, and test your connection.
170-
171-
If you restart your Compute instance or switch to a different instance, you'll need to re-authenticate to Microsoft Sentinel. For more information, see [Caching credentials with Azure CLI](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/Caching-credentials-with-Azure-CLI).
172-
173-
- **Configure and test external data providers**. VirusTotal and Maxmind GeoLite2 are the external threat intelligence and geolocation data providers used as examples in this notebook. MSTICPy also supports other threat intelligence and geolocation providers. For more information, see:
174-
175-
- [Threat intelligence providers](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html) in the MSTICPy documentation and [Threat intelligence integration in Microsoft Sentinel](threat-intelligence-integration.md)
176-
- [MSTICPy GeoIP Providers documentation](https://msticpy.readthedocs.io/en/latest/data_acquisition/GeoIPLookups.html)
177-
178-
If your notebook doesn't seem to be working as described, restart the kernel and run the notebook from the beginning. For example, if any cell in the notebook takes longer than a minute to run, try restarting the kernel and re-running the notebook. The notebook includes instructions for the basic use of Jupyter notebooks, including restarting the Jupyter kernel.
179-
180-
181-
## Test Key Vault (optional)
182-
183-
This section is relevant if you're using an Azure key vault to store your secrets, and describes how to test your connection to the vault from the notebook. If you didn't add a secret, you don't see any details. If you need to, add a test secret from the Azure Key Vault portal to the vault, and check that it shows in Microsoft Sentinel.
184-
185-
For example:
186-
187-
```Python
188-
mpconfig = MpConfigFile()
189-
mpconfig.refresh_mp_config()
190-
mpconfig.show_kv_secrets()
191-
```
192-
193-
> [!CAUTION]
194-
> Do not leave the output displayed in your saved notebook. If there are real secrets in the output, use the notebook's **Clear output** command before saving the notebook.
195-
>
196-
> Also, delete cached copies of the notebook. For example, look in the **.ipynb_checkpoints** sub-folder of your notebook directory, and delete any copies of this notebook found. Saving the notebook with a cleared output should overwrite the checkpoint copy.
197-
>
63+
:::image type="content" source="media/notebook-get-started/compute-ready.png" alt-text="Screenshot of a machine learning environment ready to run code cells.":::
19864

199-
After you have your key vault configured, use the **Upload to KV** button in the Data Providers and TI Providers sections to move the selected setting to the vault. MSTICPy generates a default name for the secret based on the path of the setting, such as `TIProviders-VirusTotal-Args-AuthKey`.
65+
The **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook includes sections for the following activities:
20066

201-
If the value is successfully uploaded, the contents of the **Value** field in the settings editor is deleted and the underlying setting is replaced with a placeholder value. MSTICPy uses this value to indicate that it should automatically generate the Key Vault path when trying to retrieve the key.
67+
|Name |Description|
68+
|---------|---------|
69+
|**Introduction** | Describe notebook basics and provides sample code you can run to see how notebooks work. |
70+
|**Initializing the notebook and MSTICPy** | Helps you get your environment ready to run the rest of the notebook. When initializing the notebook, configuration warnings about missing settings are expected because you didn't configure anything yet. |
71+
|**Querying Data from Microsoft Sentinel** | Helps you verify, configure, and test Microsoft Sentinel settings. Use the code in this section to authenticate to Microsoft Sentinel and run a sample query to test the connection. |
72+
|**Configure and test external data providers (VirusTotal and Maxmind GeoLite2)** | Helps you configure settings for VirusTotal, as a sample threat intelligence service, and MaxMind GeoLite2, as a sample geo-location lookup service. Use the code in this section to run sample queries against these data providers to test them.|
20273

203-
If you already have the required secrets stored in a Key Vault, you can enter the secret name in the **Value** field. If the secret isn't stored in your default Vault (the values specified in the [Key Vault](https://msticpy.readthedocs.io/en/latest/getting_started/SettingsEditor.html#key-vault) section), you can specify a path of **VaultName/SecretName**.
74+
The code in the **Getting Started Guide For Microsoft Sentinel ML Notebooks** launches the **MpConfigEdit** tool, which has series of tabs for configuring your notebook environment. As you make changes in **MpConfigEdit** tool, make sure to save your changes before continuing. Settings for the notebook are stored in the **msticpyconfig.yaml** file, which is automatically populated with initial details for your workspace.
20475

205-
Fetching settings from a key vault in a different tenant isn't currently supported. For more information, see [Specifying secrets as Key Vault secrets](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html#specifying-secrets-as-key-vault-secrets).
76+
Make sure to read through the markdown cells carefully so that you understand the process completely, including each of the settings and the **msticpyconfig.yaml** file. Next steps, extra resources, and frequently asked questions from the [Azure Sentinel Notebooks wiki](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/) are linked from the end of the notebook.
20677

20778
## Customize your queries (optional)
20879

209-
The **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook provides sample queries for you to use when learning about notebooks. You can customize the built-in queries by adding more query logic, or run complete queries using the `exec_query` function. For example, most built-in queries support the `add_query_items` parameter, which you can use to append filters or other operations to the queries.
80+
The **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook provides sample queries for you to use when learning about notebooks. Customize the built-in queries by adding more query logic, or run complete queries using the `exec_query` function. For example, most built-in queries support the `add_query_items` parameter, which you can use to append filters or other operations to the queries.
21081

21182
1. Run the following code cell to add a data frame that summarizes the number of alerts by alert name:
21283

0 commit comments

Comments
 (0)