Skip to content

Commit b95b15d

Browse files
committed
Solutions Analyzer V3
1 parent af5f8da commit b95b15d

File tree

940 files changed

+52168
-1375
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

940 files changed

+52168
-1375
lines changed

Tools/Solutions Analyzer/README.md

Lines changed: 178 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,27 @@
11
# Azure Sentinel Solutions Analyzer
22

3-
This tool analyzes Azure Sentinel Solutions to extract and map data connector definitions to their ingestion tables, producing comprehensive CSV reports for solution metadata analysis.
3+
This directory contains two complementary tools for analyzing Microsoft Sentinel Solutions:
4+
5+
1. **`solution_connector_tables.py`** - Extracts and maps data connector definitions to their ingestion tables, producing CSV reports with solution metadata
6+
2. **`generate_connector_docs.py`** - Generates browsable markdown documentation from the CSV data with AI-rendered setup instructions
47

58
## Quick Start
69

7-
**Pre-generated CSV files and documentation are already available in this directory:**
8-
- `solutions_connectors_tables_mapping.csv` - Main mapping of connectors to tables with full metadata
9-
- `solutions_connectors_tables_issues_and_exceptions_report.csv` - Issues and exceptions report
10-
- [`connector-docs/`](connector-docs/) - [Microsoft Sentinel Data Connector Reference](connector-docs/README.md) with browsable indexes by solutions, connectors, and tables
10+
**Pre-generated files are already available in this directory:**
11+
- [`solutions_connectors_tables_mapping.csv`](solutions_connectors_tables_mapping.csv) - Main mapping of connectors to tables with full metadata
12+
- [`solutions_connectors_tables_issues_and_exceptions_report.csv`](solutions_connectors_tables_issues_and_exceptions_report.csv) - Issues and exceptions report
1113

12-
You can use these files directly without running the script. They are kept up-to-date with the Solutions directory.
14+
You can use these files directly without running the scripts. They are kept up-to-date with the Solutions directory.
1315

14-
To regenerate the files with the latest data:
15-
```bash
16-
python solution_connector_tables.py
17-
```
16+
---
1817

19-
To regenerate the markdown documentation:
20-
```bash
21-
python generate_connector_docs.py
22-
```
18+
# 1. Solution Connector Tables Analyzer
19+
20+
**Script:** `solution_connector_tables.py`
2321

2422
## Overview
2523

26-
The analyzer scans the Solutions directory to:
24+
Scans the Solutions directory to:
2725
- Extract table references from connector JSON files (queries, sample queries, data types)
2826
- Resolve parser function references to actual tables
2927
- Flatten solution metadata from SolutionMetadata.json files
@@ -33,9 +31,8 @@ The analyzer scans the Solutions directory to:
3331

3432
**Note:** Solutions without data connectors are included in the CSV output with empty `connector_id`, `connector_title`, `connector_description`, `connector_publisher`, `connector_files`, and `Table` fields. This ensures complete solution coverage in the documentation while clearly indicating which solutions do not include data ingestion components.
3533

36-
## Installation and Requirements
34+
## Prerequisites
3735

38-
### Prerequisites
3936
- Python 3.7 or higher
4037
- No external dependencies required (optional: json5 for enhanced JSON parsing)
4138

@@ -44,7 +41,8 @@ The analyzer scans the Solutions directory to:
4441
pip install json5 # For improved JSON parsing with comments and trailing commas
4542
```
4643

47-
### Running the Script
44+
## Running the Script
45+
4846
From the `Tools/Solutions Analyzer` directory:
4947
```bash
5048
python solution_connector_tables.py
@@ -78,23 +76,11 @@ python solution_connector_tables.py --output custom_output.csv --report custom_r
7876

7977
## Output Files
8078

81-
### 1. Microsoft Sentinel Data Connector Reference (connector-docs/)
82-
83-
Browsable markdown documentation generated from the CSV data, providing:
84-
85-
- **[Solutions Index](connector-docs/solutions-index.md)** - All solutions organized alphabetically (with and without connectors)
86-
- **[Connectors Index](connector-docs/connectors-index.md)** - All unique connectors with metadata
87-
- **[Tables Index](connector-docs/tables-index.md)** - All unique tables with solution references
88-
- **Individual Solution Pages** - Detailed pages for each solution with connector and table information (in `solutions/` directory)
89-
- **Individual Connector Pages** - Detailed pages for each connector with usage information (in `connectors/` directory)
90-
91-
See the [connector-docs README](connector-docs/README.md) for full documentation.
92-
93-
### 2. solutions_connectors_tables_mapping.csv (Primary Output)
79+
### 1. solutions_connectors_tables_mapping.csv (Primary Output)
9480

9581
The main CSV file containing one row per unique combination of solution, connector, and table.
9682

97-
**Note:** Newlines in the `connector_description` field are replaced with `<br>` tags to ensure proper rendering in GitHub's CSV viewer while preserving formatting information.
83+
**Note:** Newlines in the `connector_description` and `connector_permissions` fields are replaced with `<br>` tags to ensure proper rendering in GitHub's CSV viewer. The `connector_instruction_steps` field uses standard JSON encoding with `\n` for newlines as it contains JSON-formatted data.
9884

9985
#### Column Descriptions
10086

@@ -117,6 +103,8 @@ The main CSV file containing one row per unique combination of solution, connect
117103
| `connector_publisher` | Connector publisher name. Empty for solutions without data connectors. |
118104
| `connector_title` | Connector display title. Empty for solutions without data connectors. |
119105
| `connector_description` | Connector description (newlines replaced with `<br>` for GitHub CSV rendering). Empty for solutions without data connectors. |
106+
| `connector_instruction_steps` | Setup and configuration instructions rendered from connector UI definitions using AI, stored as JSON-encoded string with standard JSON `\n` encoding. May contain inaccuracies - verify in portal. Empty for solutions without data connectors. |
107+
| `connector_permissions` | Required permissions and prerequisites in markdown format (newlines replaced with `<br>`). Empty for solutions without data connectors. |
120108
| `connector_files` | Semicolon-separated list of GitHub URLs to connector definition files. Empty for solutions without data connectors. |
121109
| `is_unique` | `true` if table appears in only one connector file, `false` otherwise |
122110
| `table_detection_methods` | (Optional, with --show-detection-methods) Semicolon-separated list of methods used to detect this table |
@@ -134,7 +122,7 @@ https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/{solution_name}
134122
https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/{solution_name}/Data Connectors/{file_path}
135123
```
136124

137-
### 3. solutions_connectors_tables_issues_and_exceptions_report.csv (Issues Report)
125+
### 2. solutions_connectors_tables_issues_and_exceptions_report.csv (Issues Report)
138126

139127
Contains exceptions and issues encountered during analysis.
140128

@@ -163,8 +151,6 @@ Contains exceptions and issues encountered during analysis.
163151
| `missing_connector_json` | Data Connectors folder exists but contains no valid JSON | Solution has no connector entries |
164152
| `missing_solution_metadata` | Solution has connectors but no SolutionMetadata.json | Solution appears with empty metadata fields |
165153

166-
**Note:** `parser_tables_resolved` entries are automatically filtered from the report as they represent successful parser-to-table resolution.
167-
168154
## Detection Logic
169155

170156
### Table Detection Methods
@@ -257,28 +243,169 @@ python solution_connector_tables.py --show-detection-methods
257243

258244
This will include the `table_detection_methods` column showing exactly how each table was detected.
259245

260-
## Updating the Script
246+
---
261247

262-
The script is located at:
263-
```
264-
Tools/Solutions Analyzer/solution_connector_tables.py
265-
```
248+
# 2. Connector Documentation Generator
249+
250+
**Script:** `generate_connector_docs.py`
251+
252+
## Overview
253+
254+
Generates browsable markdown documentation from the CSV data produced by `solution_connector_tables.py`. The documentation includes:
255+
256+
- Three index pages (solutions, connectors, tables)
257+
- Individual pages for each solution with connector details
258+
- Individual pages for each connector with usage information
259+
- **AI-rendered setup instructions** extracted from connector UI definitions
260+
261+
## Output
262+
263+
The script generates the **Microsoft Sentinel Data Connector Reference** documentation in the `connector-docs/` directory:
264+
265+
- **[Solutions Index](connector-docs/solutions-index.md)** - All solutions organized alphabetically (with and without connectors)
266+
- **[Connectors Index](connector-docs/connectors-index.md)** - All unique connectors with metadata
267+
- **[Tables Index](connector-docs/tables-index.md)** - All unique tables with solution references
268+
- **Individual Solution Pages** - Detailed pages for each solution with connector and table information (in [`solutions/`](connector-docs/solutions/) directory)
269+
- **Individual Connector Pages** - Detailed pages for each connector with usage information (in [`connectors/`](connector-docs/connectors/) directory)
270+
271+
See the [connector-docs README](connector-docs/README.md) for full documentation.
272+
273+
## Prerequisites
274+
275+
- Python 3.7 or higher
276+
- Pre-generated CSV file from `solution_connector_tables.py`
277+
- No external dependencies required
278+
279+
## Running the Script
280+
281+
From the `Tools/Solutions Analyzer` directory:
266282

267-
After modifications, test with:
268283
```bash
269-
cd "Tools/Solutions Analyzer"
270-
python solution_connector_tables.py
284+
python generate_connector_docs.py
285+
```
286+
287+
The script reads `solutions_connectors_tables_mapping.csv` and generates all documentation in the `connector-docs/` directory.
288+
289+
## Output Structure
290+
291+
The generated documentation is organized as:
292+
293+
```
294+
connector-docs/
295+
├── README.md # Documentation guide
296+
├── solutions-index.md # Alphabetical list of all solutions
297+
├── connectors-index.md # Alphabetical list of all connectors
298+
├── tables-index.md # Alphabetical list of all tables
299+
├── solutions/ # Individual solution pages (477 files)
300+
│ ├── 1password.md
301+
│ ├── aws-cloudfront.md
302+
│ └── ...
303+
└── connectors/ # Individual connector pages (503 files)
304+
├── 1passwordeventreporter.md
305+
├── awscloudfront.md
306+
└── ...
271307
```
272308

273-
## Contributing
309+
### Generated Content
310+
311+
**Solution Pages** include:
312+
- Solution metadata (publisher, support, categories)
313+
- List of connectors in the solution
314+
- Setup instructions for each connector (AI-rendered)
315+
- Required permissions and prerequisites
316+
- Tables ingested by each connector
317+
- Links to connector definition files
318+
319+
**Connector Pages** include:
320+
- Connector description and metadata
321+
- **AI-rendered setup instructions** from connector UI definitions with step-by-step guidance
322+
- Required permissions and prerequisites
323+
- List of solutions using this connector
324+
- Tables ingested by the connector
325+
- Links to GitHub connector definition files
326+
327+
**Index Pages** provide:
328+
- Alphabetical navigation
329+
- Quick statistics
330+
- Cross-references between solutions, connectors, and tables
331+
332+
## AI-Rendered Setup Instructions
333+
334+
The "Setup Instructions" sections in the generated connector documentation are **automatically rendered from connector UI definition files using AI**. These instructions interpret the UI-centric JSON structures that define the Azure Portal configuration interface and convert them into readable step-by-step guidance.
335+
336+
### ⚠️ Important Disclaimer
337+
338+
**These AI-rendered instructions may not be fully accurate.** They are generated by interpreting UI definition metadata and should always be verified against the actual Microsoft Sentinel portal before implementation. The instructions provide a helpful starting point but are not a substitute for official documentation or hands-on portal verification.
274339

275-
When adding new detection methods or modifying the logic:
276-
1. Update the `table_detection_methods` tracking in `record_table()` function
277-
2. Test with `--show-detection-methods` flag to verify detection sources
278-
3. Update this README with new detection methods or column descriptions
279-
4. Validate output doesn't introduce false positives (field names detected as tables)
340+
### How It Works
341+
342+
The rendering process involves several steps:
343+
344+
1. **JSON Parsing**: The script extracts `instructionSteps` arrays from connector definition files in the Solutions directory
345+
2. **UI Type Detection**: Each instruction step has a `type` property (e.g., `DataConnectorsGrid`, `ContextPane`, `GCPGrid`) that determines how it should be interpreted
346+
3. **AI-Powered Rendering**: Specialized handlers for each UI type convert the JSON structure into descriptive markdown:
347+
- Form fields (textboxes, dropdowns) are described with their purposes and validation requirements
348+
- Management grids and data selectors are explained with their configuration options
349+
- Portal-only interfaces are identified and marked with clear indicators
350+
4. **Markdown Formatting**: The rendered content is formatted with emoji indicators, step numbers, and disclaimers
351+
352+
### UI Types Supported
353+
354+
The script includes specialized handlers for connector UI configuration types based on the [official Microsoft Sentinel data connector UI definitions reference](https://learn.microsoft.com/en-us/azure/sentinel/data-connector-ui-definitions-reference#instructionsteps):
355+
356+
**Standard Instruction Types:**
357+
358+
- **OAuthForm**: OAuth authentication forms with client credentials
359+
- **Textbox**: Input fields for text, passwords, numbers, and email addresses
360+
- **Dropdown**: Selection lists with single or multi-select options
361+
- **Markdown**: Formatted text content with links and formatting
362+
- **CopyableLabel**: Text fields with copy-to-clipboard functionality
363+
- **InfoMessage**: Inline information messages with contextual help
364+
- **ConnectionToggleButton**: Connect/disconnect toggle controls
365+
- **InstructionStepsGroup**: Collapsible groups of nested instructions
366+
- **InstallAgent**: Links to Azure portal sections for agent installation (18 link types supported)
367+
368+
**UI-Centric Configuration Types:**
369+
370+
- **DataConnectorsGrid**: Interactive data connector management interface with enable/disable controls
371+
- **ContextPane**: Sidebar configuration panels with detailed settings
372+
- **GCPGrid** / **GCPContextPane**: Google Cloud Platform specific configuration interfaces
373+
- **AADDataTypes**: Azure Active Directory data type selectors
374+
- **MCasDataTypes**: Microsoft Defender for Cloud Apps data type selectors
375+
- **OfficeDataTypes**: Microsoft 365 data type selectors
376+
377+
Instructions for 74 connectors using these UI-centric configuration interfaces have been enhanced with AI-rendered setup guidance.
378+
379+
### Example Output
380+
381+
Instructions are formatted with:
382+
383+
- 📋 Portal-only interfaces clearly marked
384+
- 📝 Form fields with descriptions and placeholders
385+
- ⚠️ Disclaimers about AI generation and accuracy
386+
- 🔗 Links to GitHub connector definition files
387+
388+
---
280389

281390
## Version History
282391

283-
- **v1.0** - Initial release with basic table detection
284-
- **v2.0** - Added parser resolution, context-aware detection, enhanced JSON parsing, flattened metadata, GitHub URLs
392+
### v3.0
393+
394+
- Added `connector_instruction_steps` and `connector_permissions` fields to CSV output
395+
- Added AI-rendered connector setup instructions from UI definitions
396+
- Added individual table detail pages for tables with multiple solutions or connectors
397+
- Improved tables index with limited inline display and clickable "+X more" links
398+
399+
### v2.0
400+
401+
- Added parser resolution and context-aware table detection
402+
- Enhanced JSON parsing tolerance for malformed connector definitions
403+
- Flattened metadata extraction from nested solution structures
404+
- Added GitHub URLs for all file references
405+
- Improved error handling and validation
406+
407+
## v1.0
408+
409+
- Initial release with basic table detection from connector JSON files
410+
- CSV output with solution, connector, and table mappings
411+
- Issues and exceptions reporting

Tools/Solutions Analyzer/connector-docs/README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,23 @@ Individual solution pages are organized in the [`solutions/`](solutions/) direct
1717
- Solution metadata (title, publisher, description)
1818
- List of data connectors included in the solution
1919
- Detailed connector information (ID, title, description)
20+
- Setup instructions (AI-generated from UI definitions - verify in portal)
21+
- Required permissions and prerequisites
2022
- Tables associated with each connector
2123
- Table uniqueness indicators (whether a table is used by only one connector)
2224

25+
### Connector Pages
26+
27+
Individual connector pages are organized in the [`connectors/`](connectors/) directory. Each connector page includes:
28+
29+
- Connector metadata (ID, publisher, tables ingested)
30+
- Full connector description
31+
- Required permissions and prerequisites
32+
- **Setup Instructions** - Step-by-step configuration guidance rendered from connector UI definitions using AI
33+
- ⚠️ **Note**: Instructions are automatically rendered from the user interface definition files using AI and may not be fully accurate. Always verify configuration steps in the Microsoft Sentinel portal.
34+
- Solutions that include this connector
35+
- Links to connector definition files on GitHub
36+
2337
## 📊 Quick Statistics
2438

2539
For current statistics, see the [Solutions Index](solutions-index.md) which displays up-to-date counts of solutions (with and without connectors), connectors, and tables.
@@ -64,6 +78,17 @@ This documentation is automatically generated from the Solutions Analyzer tool,
6478

6579
The analyzer identifies table references in connector configurations and parser logic to create comprehensive mappings.
6680

81+
### AI-Generated Instructions
82+
83+
**Setup Instructions** in connector documentation are automatically extracted from connector UI definition files using AI:
84+
- Interprets UI-centric instruction types (DataConnectorsGrid, ContextPane, GCPGrid, AADDataTypes, etc.)
85+
- Converts JSON UI definitions to readable markdown format
86+
- Generates step-by-step configuration guidance
87+
- Describes form fields, dropdowns, and management interfaces
88+
- Marks portal-only features with visual indicators
89+
90+
⚠️ **Important**: AI-generated instructions may not be fully accurate. Always verify all configuration steps in the Microsoft Sentinel portal before implementation.
91+
6792
## 📝 Data Source
6893

6994
The documentation is based on analysis of the `solutions_connectors_tables_mapping.csv` file, which contains:

0 commit comments

Comments
 (0)