You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/notebook-get-started.md
+44-41Lines changed: 44 additions & 41 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,38 +17,29 @@ ms.collection: usx-security
17
17
18
18
# Get started with Jupyter notebooks and MSTICPy in Microsoft Sentinel
19
19
20
-
This article describes how to run the **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook, which sets up basic configurations for running Jupyter notebooks in Microsoft Sentinel and running simple data queries.
20
+
This article describes how to run the **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook, which sets up basic configurations for running Jupyter notebooks in Microsoft Sentinel and provides examples for running simple queries.
21
21
22
22
The **Getting Started Guide for Microsoft Sentinel ML Notebooks** notebook uses [MSTICPy](https://msticpy.readthedocs.io/en/latest/), a powerful Python library designed to enhance security investigations and threat hunting within Microsoft Sentinel notebooks. It provides built-in tools for data enrichment, visualization, anomaly detection, and automated queries, helping analysts streamline their workflow without extensive custom coding.
23
23
24
24
For more information, see [Use notebooks to power investigations](hunting.md#use-notebooks-to-power-investigations) and [Use Jupyter notebooks to hunt for security threats](notebooks.md).
25
25
26
-
The steps in this article describe how to run the **Getting Started Guide for Microsoft Sentinel ML Notebooks** notebook in your Azure Machine Learning workspace via Microsoft Sentinel. You can also use this article as guidance for performing similar steps to run notebooks in other environments, including locally.
27
-
28
-
Several Microsoft Sentinel notebooks don't use MSTICPy, such as the **Credential Scanner** notebooks, or the PowerShell and C# examples. Notebooks that don't use MSTICpy don't need the MSTICPy configuration described in this article.
Before you begin, make sure you have the required permissions and resources.
35
31
36
-
- To use notebooks in Microsoft Sentinel, make sure that you have the required permissions. For more information, see [Manage access to Microsoft Sentinel notebooks](notebooks.md#manage-access-to-microsoft-sentinel-notebooks).
37
-
38
-
- To perform the steps in this article, you need Python 3.6 or later. In Azure Machine Learning, you can use either a Python 3.8 kernel (recommended) or a Python 3.6 kernel. If you use the notebook described in this article in another Jupyter environment, you can use any kernel that supports Python 3.6 or later.
39
-
40
-
To use MSTICPy notebooks outside of Microsoft Sentinel and Azure Machine Learning (ML), you also need to configure your Python environment. Install Python 3.6 or later with the Anaconda distribution, which includes many of the required packages.
41
-
42
-
- This notebook uses the [MaxMind GeoLite2](https://www.maxmind.com) geolocation lookup service for IP addresses. To use the MaxMind GeoLite2 service, you need an account key. You can sign up for a free account and key at the [Maxmind signup page](https://www.maxmind.com/en/geolite2/signup).
43
-
44
-
- This notebook uses [VirusTotal](https://www.virustotal.com) (VT) as a threat intelligence source. To use VirusTotal threat intelligence lookup, you need a VirusTotal account and API key.
45
-
46
-
You can sign up for a free VT account at the [VirusTotal getting started page](https://developers.virustotal.com/v3.0/reference#getting-started). If you're already a VirusTotal user, you can use your existing key.
32
+
|Prerequisite |Description |
33
+
|---------|---------|
34
+
|**Permissions**| To use notebooks in Microsoft Sentinel, make sure that you have the required permissions. <br><br>For more information, see [Manage access to Microsoft Sentinel notebooks](notebooks.md#manage-access-to-microsoft-sentinel-notebooks). |
35
+
|**Python**| To perform the steps in this article, you need Python 3.6 or later. <br><br>In Azure Machine Learning, you can use either a Python 3.8 kernel (recommended) or a Python 3.6 kernel. If you use the notebook described in this article in another Jupyter environment, you can use any kernel that supports Python 3.6 or later.<br><br> To use MSTICPy notebooks outside of Microsoft Sentinel and Azure Machine Learning (ML), you also need to configure your Python environment. Install Python 3.6 or later with the Anaconda distribution, which includes many of the required packages. |
36
+
|[**MaxMind GeoLite2**](https://www.maxmind.com)| This notebook uses the MaxMind GeoLite2 geolocation lookup service for IP addresses. To use the MaxMind GeoLite2 service, you need an account key. You can sign up for a free account and key at the [Maxmind signup page](https://www.maxmind.com/en/geolite2/signup). |
37
+
|[**VirusTotal**](https://www.virustotal.com)| This notebook uses VirusTotal (VT) as a threat intelligence source. To use VirusTotal threat intelligence lookup, you need a VirusTotal account and API key. <br><br> |
47
38
48
-
> [!WARNING]
49
-
> If you're using a VT enterprise key, store it in Azure Key Vault instead of the **msticpyconfig.yaml** file. For more information, see [Specify secrets as Key Vault secrets](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html#specifying-secrets-as-key-vault-secrets) in the MSTICPY documentation.
50
-
>
51
-
> If you don’t want to set up an Azure Key Vault right now, sign up for and use a free account until you can set up Key Vault storage.
39
+
> [!WARNING]
40
+
> If you're using a VT enterprise key, store it in Azure Key Vault instead of the **msticpyconfig.yaml** file. For more information, see [Specify secrets as Key Vault secrets](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html#specifying-secrets-as-key-vault-secrets) in the MSTICPY documentation.
41
+
>
42
+
> If you don’t want to set up an Azure Key Vault right now, sign up for and use a free account until you can set up Key Vault storage.
52
43
53
44
## Install and run the Getting Started Guide notebook
54
45
@@ -68,28 +59,27 @@ This procedure describes how to launch your notebook with Microsoft Sentinel.
68
59
69
60
1. Read and run the code cells in order, using the directions in the notebook. Skipping cells or running them out of order might cause errors later in the notebook.
70
61
71
-
Depending on the function being performed, the code in the cell might run quickly, or it might take a few seconds to complete. When the cell is running, the play button changes to a loading spinner, and a status of `Executing` is displayed at the bottom of the cell, together with the elapsed time.
72
-
73
-
If your notebook doesn't seem to be working as described, restart the kernel and run the notebook from the beginning. For example, if any cell in the notebook takes longer than a minute to run, try restarting the kernel and re-running the notebook.
74
-
75
-
The notebook includes instructions for the basic use of Jupyter notebooks, including restarting the Jupyter kernel.
62
+
Depending on the function being performed, the code in the cell might run quickly, or it might take a few seconds to complete. When the cell is running, the play button changes to a loading spinner, and a status of `Executing` is displayed at the bottom of the cell, together with the elapsed time.
76
63
77
-
Continue by following the instructions in the notebook to do the following:
64
+
The notebook contains sections for you to run the following tasks:
78
65
79
-
-**Initialize the notebook and MSTICPy**. Use this section of the notebook to set up your environment and understand the basics of notebooks and MSYTICPy.
66
+
-**Initialize the notebook and MSTICPy**. Use this section of the notebook to set up your environment and understand the basics of notebooks and MSYTICPy. For more information, see the sample [`msticpyconfig.yaml](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/msticpyconfig.yaml) template, which has commented-out sections that might help you understand the settings.
80
67
81
68
-**Query data from Microsoft Sentinel.** Use this section of the notebook to verify your Microsoft Sentinel settings in MSTICPy, load a QueryProvider to query data from Microsoft Sentinel, authenticate to Microsoft Sentinel, and test your connection.
82
69
83
70
If you restart your Compute instance or switch to a different instance, you'll need to re-authenticate to Microsoft Sentinel. For more information, see [Caching credentials with Azure CLI](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/Caching-credentials-with-Azure-CLI).
84
71
85
-
-**Configure and test VirusTotal and Maxmind GeoLite2 as external data providers**. VirusTotal and Maxmind GeoLite2 are the external threat intelligence and geolocation data providers used as examples in this notebook. MSTICPy also supports other threat intelligence and geolocation providers. For more information, see:
72
+
-**Configure and test external data providers**. VirusTotal and Maxmind GeoLite2 are the external threat intelligence and geolocation data providers used as examples in this notebook. MSTICPy also supports other threat intelligence and geolocation providers. For more information, see:
86
73
87
74
-[Threat intelligence providers](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html) in the MSTICPy documentation and [Threat intelligence integration in Microsoft Sentinel](threat-intelligence-integration.md)
If your notebook doesn't seem to be working as described, restart the kernel and run the notebook from the beginning. For example, if any cell in the notebook takes longer than a minute to run, try restarting the kernel and re-running the notebook. The notebook includes instructions for the basic use of Jupyter notebooks, including restarting the Jupyter kernel.
91
78
92
-
This procedure is relevant if you're using an Azure key vault to store your secrets, and describes how to test your connection to the vault from the notebook. If you didn't add a secret, you don't see any details. If you need to, add a test secret from the Azure Key Vault portal to the vault, and check that it shows in Microsoft Sentinel.
79
+
80
+
## Test Key Vault (optional)
81
+
82
+
This section is relevant if you're using an Azure key vault to store your secrets, and describes how to test your connection to the vault from the notebook. If you didn't add a secret, you don't see any details. If you need to, add a test secret from the Azure Key Vault portal to the vault, and check that it shows in Microsoft Sentinel.
93
83
94
84
For example:
95
85
@@ -113,7 +103,7 @@ If you already have the required secrets stored in a Key Vault, you can enter th
113
103
114
104
Fetching settings from a key vault in a different tenant isn't currently supported. For more information, see [Specifying secrets as Key Vault secrets](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html#specifying-secrets-as-key-vault-secrets).
115
105
116
-
## Customize your queries
106
+
## Customize your queries (optional)
117
107
118
108
The **Getting Started Guide For Microsoft Sentinel ML Notebooks** notebook provides sample queries for you to use when learning about notebooks. You can customize the built-in queries by adding more query logic, or run complete queries using the `exec_query` function. For example, most built-in queries support the `add_query_items` parameter, which you can use to append filters or other operations to the queries.
119
109
@@ -150,20 +140,33 @@ For more information, see:
150
140
- The [MSTICPy query reference](https://msticpy.readthedocs.io/en/latest/data_acquisition/DataQueries.html)
The steps in this article describe how to run the **Getting Started Guide for Microsoft Sentinel ML Notebooks** notebook in your Azure Machine Learning workspace via Microsoft Sentinel. You can also use this article as guidance for performing similar steps to run notebooks in other environments, including locally.
146
+
147
+
Several Microsoft Sentinel notebooks don't use MSTICPy, such as the **Credential Scanner** notebooks, or the PowerShell and C# examples. Notebooks that don't use MSTICpy don't need the MSTICPy configuration described in this article.
148
+
149
+
Try out other Microsoft Sentinel notebooks, such as:
150
+
151
+
-**Configuring your Notebook Environment**
152
+
-**A Tour of Cybersec notebook features**
153
+
-**Machine Learning in Notebooks Examples**
154
+
- The **Entity Explorer** series, including variations for accounts, domains and URLs, IP addresses, and Linux or Windows hosts.
155
155
156
156
For more information, see:
157
157
158
-
|Subject |More references |
159
-
|---------|---------|
160
-
|**MSTICPy**|- [MSTICPy Package Configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)<br>- [MSTICPy Settings Editor](https://msticpy.readthedocs.io/en/latest/getting_started/SettingsEditor.html)<br>- [Configuring Your Notebook Environment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb).<br>- [MPSettingsEditor notebook](https://github.com/microsoft/msticpy/blob/master/docs/notebooks/MPSettingsEditor.ipynb). <br><br>**Note**: The `Azure-Sentinel-Notebooks` GitHub repository also contains a template *msticpyconfig.yaml*filewith commented-out sections, which might help you understand the settings. |
161
-
|**Microsoft Sentinel and Jupyter notebooks** | - [Advanced configurations for Jupyter notebooks and MSTICPy in Microsoft Sentinel](notebooks-msticpy-advanced.md)<br> - [Create your first Microsoft Sentinel notebook](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/creating-your-first-microsoft-sentinel-notebook/ba-p/2977745) (Blog series)<br> - [Jupyter Notebooks: An Introduction](https://realpython.com/jupyter-notebook-introduction/)<br> - [MSTICPy documentation](https://msticpy.readthedocs.io/)<br> - [Microsoft Sentinel Notebooks documentation](notebooks.md)<br> - [The Infosec Jupyterbook](https://infosecjupyterbook.com/introduction.html)<br> - [Linux Host Explorer Notebook walkthrough](https://techcommunity.microsoft.com/t5/azure-sentinel/explorer-notebook-series-the-linux-host-explorer/ba-p/1138273)<br> - [Why use Jupyter for Security Investigations](https://techcommunity.microsoft.com/t5/azure-sentinel/why-use-jupyter-for-security-investigations/ba-p/475729)<br> - [Security Investigations with Microsoft Sentinel & Notebooks](https://techcommunity.microsoft.com/t5/azure-sentinel/security-investigation-with-azure-sentinel-and-jupyter-notebooks/ba-p/432921)<br> - [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/index.html)<br> - [Bokeh Documentation](https://docs.bokeh.org/en/latest/) |
158
+
- [Jupyter notebooks with Microsoft Sentinel hunting capabilities](notebooks.md)
159
+
- [Advanced configurations for Jupyter notebooks and MSTICPy in Microsoft Sentinel](notebooks-msticpy-advanced.md)
160
+
- [Create your first Microsoft Sentinel notebook](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/creating-your-first-microsoft-sentinel-notebook/ba-p/2977745) (Blog series)
161
+
- [Linux Host Explorer Notebook walkthrough](https://techcommunity.microsoft.com/t5/azure-sentinel/explorer-notebook-series-the-linux-host-explorer/ba-p/1138273) (Blog)
162
162
163
-
## Next steps
164
163
165
-
Try out other notebooks stored in the [Microsoft Sentinel Notebooks GitHub repository](https://github.com/Azure/Azure-Sentinel-Notebooks), such as:
164
+
## Related content
165
+
166
+
For more information, see:
166
167
167
-
- [Tour of the Cybersec features](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/9bba6bb9007212fca76169c3d9a29df2da95582d/A%20Tour%20of%20Cybersec%20notebook%20features.ipynb)
- The [Entity Explorer series](https://github.com/Azure/Azure-Sentinel-Notebooks/) of notebooks, which allow for a deep drill-down into details about a host, account, IP address, and other entities.
- [Why use Jupyter for Security Investigations](https://techcommunity.microsoft.com/t5/azure-sentinel/why-use-jupyter-for-security-investigations/ba-p/475729)
172
+
- [Security Investigations with Microsoft Sentinel & Notebooks](https://techcommunity.microsoft.com/t5/azure-sentinel/security-investigation-with-azure-sentinel-and-jupyter-notebooks/ba-p/432921)
0 commit comments