Skip to content

Commit 2eb9033

Browse files
committed
update README
1 parent a0c96a3 commit 2eb9033

File tree

1 file changed

+102
-9
lines changed

1 file changed

+102
-9
lines changed

README.md

Lines changed: 102 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,110 @@
1-
# REPO NAME
1+
# Databricks Workspace Detection App
22

3-
```
4-
Placeholder
3+
A collection of security detection notebooks for Databricks workspaces that analyze the `system.access.audit` table to identify potential security threats and suspicious activities.
54

6-
Fill here a description at a functional level - what is this content doing
7-
```
5+
## Overview
6+
7+
This detection app provides 25+ pre-built security detection notebooks designed for security operations teams to monitor Databricks workspace activities. The detections cover various security scenarios including:
8+
9+
- **Authentication & Access Control**: Token creation/deletion, MFA changes, SSO configuration changes
10+
- **User Management**: Account creation/deletion, role modifications, group changes
11+
- **Session Security**: Session hijacking detection, multi-device login patterns
12+
- **Administrative Activity**: Privilege escalation, admin activity spikes
13+
- **Audit & Compliance**: Verbose logging changes, audit configuration tampering
14+
15+
## Features
16+
17+
- **Coverage**: 25+ detection scenarios covering major security use cases
18+
- **Production Ready**: Designed for batch execution via Databricks workflows
19+
- **Configurable**: Customizable time ranges and detection parameters
20+
- **Audit Table Focus**: Leverages Databricks `system.access.audit` table for comprehensive visibility
21+
- **Unity Catalog Compatible**: Designed for Unity Catalog enabled accounts
822

9-
## Video Overview
23+
## Detection Categories
1024

11-
Include a GIF overview of what your project does. Use a service like Quicktime, Zoom or Loom to create the video, then convert to a GIF.
25+
### Authentication & Identity
26+
- Access Token Created/Deleted
27+
- MFA Key Added/Deleted
28+
- Non-SSO Login Detection
29+
- User Password Changes
30+
- SSO Configuration Changes
1231

32+
### User & Group Management
33+
- User Account Created/Deleted
34+
- Group Created/Deleted
35+
- Principal Added/Removed from Groups
36+
- User Role Modifications
37+
38+
### Session Security
39+
- Session Hijacking Detection (Multiple IPs/Devices)
40+
- High Session Count Detection
41+
- Frequent Login Patterns
42+
- Multi-Device Session Reuse
43+
44+
### Administrative Monitoring
45+
- Spike in Table Admin Activity
46+
- Databricks Employee Logon Detection
47+
- Verbose Audit Logging Disabled
48+
49+
### Network & Access Control
50+
- Attempted Logon from Denied IP
51+
- Token Scanning Activity Detection
1352

1453
## Installation
1554

16-
Include details on how to use and install this content.
55+
### Prerequisites
56+
- Databricks workspace with Unity Catalog enabled
57+
- Access to `system.access.audit` table
58+
- Appropriate permissions to create and run workflows
59+
60+
### Setup
61+
1. **Import the App**: Add the detection notebooks to your Databricks workspace
62+
2. **Configure Workflows**: Set up Databricks workflows for each detection
63+
3. **Adjust Parameters**: Modify start/end times and detection parameters as needed
64+
4. **Schedule Execution**: Configure trigger schedules matching your lookback periods
65+
66+
### Configuration Notes
67+
- Detection searches rely on access to the audit table
68+
- Designed for batch mode execution using workflows
69+
- Ensure trigger schedules match lookback periods for full coverage
70+
- Avoid duplicate events by properly configuring execution intervals
71+
72+
## Usage
73+
74+
### Running Individual Detections
75+
Each detection notebook can be run independently with configurable time parameters:
76+
77+
```python
78+
# Example: Run access token detection for last 24 hours
79+
result = access_token_created(
80+
earliest="2025-01-01T00:00:00",
81+
latest="2025-01-02T00:00:00"
82+
)
83+
```
84+
85+
### Workflow Integration
86+
Detections are designed to be integrated into Databricks workflows for automated security monitoring:
87+
88+
1. **Batch Processing**: Run detections on scheduled intervals
89+
2. **Alert Generation**: Output results to detection or alerts tables
90+
3. **Ad-hoc Analysis**: Generate dataframes for manual investigation
91+
92+
### Output Formats
93+
- **DataFrame Output**: Structured data for further analysis
94+
- **Standardized Schema**: Consistent column naming across all detections
95+
- **Audit Trail**: Complete event details with timestamps and metadata
96+
97+
## Architecture
98+
99+
### Core Components
100+
- **Detection Notebooks**: Individual security detection logic
101+
- **Common Library**: Shared utilities and enrichment functions
102+
- **Audit Table Integration**: Direct queries against `system.access.audit`
103+
104+
### Dependencies
105+
- **PySpark**: Core data processing framework
106+
- **GeoIP2**: IP address geolocation capabilities
107+
- **NetAddr**: IP address manipulation utilities
17108

18109
## How to get help
19110

@@ -22,7 +113,9 @@ Databricks support doesn't cover this content. For questions or bugs, please ope
22113

23114
## License
24115

25-
© 2025 Databricks, Inc. All rights reserved. The source in this notebook is provided subject to the Databricks License [https://databricks.com/db-license-source]. All included or referenced third party libraries are subject to the licenses set forth below.
116+
© 2025 Databricks, Inc. All rights reserved. The source in this notebook is provided subject to the Databricks License [https://databricks.com/db-license-source]. All included or referenced third party libraries are subject to the licenses set forth below.
26117

27118
| library | description | license | source |
28119
|----------------------------------------|-------------------------|------------|-----------------------------------------------------|
120+
| geoip2 | IP address geolocation | Apache 2.0 | https://github.com/maxmind/GeoIP2-python |
121+
| netaddr | IP address manipulation| BSD | https://github.com/netaddr/netaddr |

0 commit comments

Comments
 (0)