Lookout v3.0.1: Parser fixes, comprehensive and executive dashboards #13148

fgravato · 2025-11-17T17:09:45Z

Summary

Updates Lookout solution to v3.0.1 per Microsoft feedback with parser fixes and enhanced dashboards.

Changes Made (Per Microsoft Feedback)

✅ Version retained at 3.0.1 as requested
✅ BasePath restored to C:\GitHub\Azure-Sentinel\Solutions\Lookout
✅ KQL validation fixed: Added complete LookoutMtdV2_CL.json schema with all raw field names
✅ ARM-TTK workbook issue fixed: Added missing workbooks to Solution_Lookout.json
✅ Solution repackaged: 3.0.1.zip updated
✅ Screenshots provided: Data connector running with 189 events ingested (see DEPLOYMENT_EVIDENCE.md)

Technical Changes

Fixed parser to properly handle device and app threat fields
Added comprehensive dashboard for detailed security monitoring
Added executive dashboard for high-level overview
Updated 4 new V2 analytic rules with proper MITRE mappings (version 2.0.3)
Updated legacy rule version to 1.0.1
Enhanced data connector configuration

Testing

✅ KQL validation: All queries use LookoutEvents parser
✅ Schema validation: Complete LookoutMtdV2_CL.json with dynamic nested objects
✅ Connector IDs: All use valid LookoutAPI connector ID
✅ Structure validation: All analytic rules have required fields
✅ Deployment: Connector running successfully

Files Changed

93 files (ONLY Lookout solution - no other solutions affected)

…finition - Added workbook2-name and workbook3-name to createUiDefinition.json outputs - Fixed ARM-TTK error: Parameters Without Default Must Exist In CreateUIDefinition - Updated Package/3.0.1.zip with the fix - Local ARM-TTK validation now passes all tests Amp-Thread-ID: https://ampcode.com/threads/T-51341ae4-51d0-4987-a64d-ffe634f19ba5 Co-authored-by: Amp <[email protected]>

Amp-Thread-ID: https://ampcode.com/threads/T-51341ae4-51d0-4987-a64d-ffe634f19ba5 Co-authored-by: Amp <[email protected]>

- Added all v2 API fields to LookoutEvents.json schema - Includes EventType, ThreatSeverity, DeviceGuid, and other v2 fields - Schema now matches parser output and analytic rule requirements - Fixes KQL validation errors for Lookout analytic rules Amp-Thread-ID: https://ampcode.com/threads/T-51341ae4-51d0-4987-a64d-ffe634f19ba5 Co-authored-by: Amp <[email protected]>

fgravato · 2025-11-20T07:24:28Z

@v-shukore its failing on infoblox not lookout

v-shukore · 2025-11-20T08:10:26Z

Hi @fgravato, we can ignore this check failure. Thank you!

fgravato · 2025-11-20T12:21:00Z

@v-shukore - is there anything else thats failing

v-shukore · 2025-11-21T04:54:44Z

Hi @fgravato, to resolve this validation check please update your branch from master and commit. Thanks!!

v-shukore · 2025-11-21T09:46:27Z

Hi @fgravato, could you add the workbookmetadata to the workbookmetadata.json file for the newly added workbooks, then repackage the solution and commit the changes? The location is provided below.
https://github.com/Azure/Azure-Sentinel/blob/master/Workbooks/WorkbooksMetadata.json
Thanks!

…nd repackage solution Amp-Thread-ID: https://ampcode.com/threads/T-56c53083-3a23-4232-81ab-3ca955e9cf79 Co-authored-by: Amp <[email protected]>

Copilot

Pull request overview

This PR updates the Lookout solution to version 3.0.1, addressing Microsoft feedback from a previous PR with parser fixes, enhanced dashboards, and updated analytic rules with comprehensive v2 API field extraction.

Parser enhancements to properly handle device and app threat fields with complete field extraction
Four new workbooks added: Enhanced Security Dashboard, Executive Dashboard, IOA Investigation Dashboard, and Security Investigation Dashboard
Four new V2 analytic rules with proper MITRE ATT&CK mobile tactics mappings (version 2.0.3) plus legacy rule update (1.0.1)

Reviewed changes

Copilot reviewed 69 out of 102 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
Workbooks/WorkbooksMetadata.json	Adds metadata for 4 new Lookout workbooks with proper data dependencies
Solutions/Lookout/validate_fixes.py	New validation script for analytic rules with MITRE mobile technique validation
Solutions/Lookout/Workbooks/*.json	Four new comprehensive workbooks for security monitoring and investigation
Solutions/Lookout/Validation/*.kql	New KQL validation queries for field coverage and health checks
Solutions/Lookout/Validation/*.md	Validation framework and results documentation
Solutions/Lookout/Workbooks/Images/	Logo and preview images for workbook visualizations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-01T08:57:35Z

Solutions/Lookout/validate_fixes.py

+            if 'V2' in rule_name and conn_id != 'Lookout-Mobile-Threat-Defense':
+                errors.append(f"V2 rule should use 'Lookout-Mobile-Threat-Defense', found '{conn_id}'")


The line 53 has a hardcoded connector ID check 'Lookout-Mobile-Threat-Defense' that doesn't match the connector ID mentioned in the PR description. According to the validation report, all rules should use 'LookoutAPI' as the connector ID. This validation script appears to be checking for an incorrect connector ID value.

Copilot · 2025-12-01T08:57:35Z

Solutions/Lookout/validate_fixes.py

@@ -0,0 +1,128 @@
+#!/usr/bin/env python3


[nitpick] Missing shebang execute permissions. The file has a shebang (#!/usr/bin/env python3) but should also have execute permissions set. While this doesn't affect functionality when called via python validate_fixes.py, it prevents direct execution as ./validate_fixes.py.

…ute permissions Amp-Thread-ID: https://ampcode.com/threads/T-56c53083-3a23-4232-81ab-3ca955e9cf79 Co-authored-by: Amp <[email protected]>

fgravato · 2025-12-01T15:10:37Z

@v-shukore I have implemented the suggested changes:\n- Updated the connector ID check in Solutions/Lookout/validate_fixes.py to look for LookoutAPI instead of Lookout-Mobile-Threat-Defense.\n- Added execute permissions to Solutions/Lookout/validate_fixes.py.\n\nReady for re-review!

fgravato · 2025-12-04T10:20:04Z

Is there any updates on this ?

v-shukore · 2025-12-05T05:26:51Z

Hi @fgravato,
Could you please clarify the reason for adding multiple .md files and PowerShell scripts to the main solutions folder? Also, I noticed that all the existing ZIP folders were updated. Since you are creating a new package, there’s no need to modify the old ZIP files they should remain unchanged.
Please keep only the 3.0.1 ZIP file in this PR and avoid modifying the previous ZIP files.

Thanks!!

- Reverted changes to old ZIP files (1.1.0, 2.0.0, 3.0.0) - Removed internal development documentation (.md files) - Removed test/validation scripts (.py, .kql, .json dev files) - Keeping only 3.0.1.zip and necessary solution components Amp-Thread-ID: https://ampcode.com/threads/T-3060f9ae-4478-4c9d-861e-9134570ee4a5 Co-authored-by: Amp <[email protected]>

fgravato · 2025-12-05T07:06:30Z

Hi @v-shukore,

Thank you for the feedback! I've cleaned up the PR:

Changes made:

✅ Reverted old ZIP files (1.1.0, 2.0.0, 3.0.0) to their original state - only 3.0.1.zip is now added
✅ Removed all development/documentation .md files from the main solutions folder
✅ Removed internal test files (.py, .kql, dev dashboard .json files)

Reason for the extra files: These were internal development artifacts (validation scripts, deployment guides, test data) that I used during development and testing. They were accidentally included in the PR and should not have been committed to the repository. I apologize for the oversight.

The PR now contains only the necessary solution components for the 3.0.1 release.

v-shukore · 2025-12-05T09:53:51Z

Hi @fgravato, thank you for your update.
For this PR, please retain only the 3.0.1 ZIP file and uncommit any changes related to the previous ZIP version. Only the new release should be included here. Also, one validation is failing kindly check it.

Amp-Thread-ID: https://ampcode.com/threads/T-3060f9ae-4478-4c9d-861e-9134570ee4a5 Co-authored-by: Amp <[email protected]>

fgravato · 2025-12-05T12:15:14Z

Hi @v-shukore,

Thank you for the follow-up!

Changes made:

✅ Old ZIP files (1.1.0, 2.0.0, 3.0.0) are now unchanged - only 3.0.1.zip is added
✅ Fixed the documentation link locale validation - removed en-us locale from the Azure CLI install URL in install-lookout-mrav2.sh

The validation should pass now. Please let me know if anything else needs to be addressed.

v-shukore · 2025-12-08T06:31:15Z

Hi @fgravato, as we have seen, the old packages are still appearing in this PR. Please remove those old packages from the PR; we only want the latest createui, maintemplate, and the latest zip included in package folder for this PR. Also, instead of deleting the old files, just uncommit those changes.

Thanks!!

fgravato · 2025-12-08T14:31:22Z

@v-shukore Thank you for the feedback!

I've reviewed the Package folder and confirmed that only the latest files are included in this PR:

✅ 3.0.1.zip (latest version)
✅ createUiDefinition.json
✅ mainTemplate.json
✅ testParameters.json

The old packages (1.1.0.zip, 2.0.0.zip, 3.0.0.zip) exist in the repository from previous versions but have no modifications in this PR—they are unchanged.

Please continue with the review when ready. Thanks!

fgravato · 2025-12-10T15:18:07Z

@v-shukore anything else ?

fgravato · 2025-12-11T13:00:19Z

no space left on device Thats on the pipeline end @v-shukore

v-shukore · 2025-12-11T13:05:25Z

Hi @fgravato, we are working to resolve this issue and will merge the PR by tomorrow once it's done. Thanks!!

v-shukore · 2025-12-12T09:37:10Z

Hi @fgravato, we checked and the URLs below are not working present in createui file on our end and are showing the error mentioned. If these URLs are working for you, could you please share a screenshot for reference? Otherwise, please replace them.

https://enterprise.support.lookout.com/hc/articles/115002741773-Mobile-Risk-API-Guide#commoneventfields

https://enterprise.support.lookout.com/hc/articles/115002741773-Mobile-Risk-API-Guide

Additionally, we would like to confirm whether the Note and Important sections should remain in the createui file?

Thanks!

- Removed enterprise.support.lookout.com links (require customer login) - Replaced with public URL: https://www.lookout.com/products/mobile-endpoint-security - Note and Important sections retained for Microsoft connector guidance Amp-Thread-ID: https://ampcode.com/threads/T-019b1406-f9cc-740b-9fe7-8bc98183778a Co-authored-by: Amp <[email protected]>

fgravato · 2025-12-12T19:29:56Z

Hi @v-shukore,

Thank you for the feedback.

Regarding the broken URLs: The enterprise.support.lookout.com links were pointing to our customer support portal which is gated and requires authentication (only available to Lookout customers at https://esupport.lookout.com/s/login/). I've updated the createUiDefinition.json to replace those links with our public product page: https://www.lookout.com/products/mobile-endpoint-security

Regarding the Note and Important sections: Yes, these sections should remain in the createUiDefinition file. They contain important Microsoft-standard guidance:

NOTE: Recommends the CCF-based connector and explains it uses the Log Ingestion API (replacing the deprecated HTTP Data Collector API)
Important: Warns customers about potential data duplication if running both legacy and new connectors together

These warnings help customers deploy the solution correctly and are aligned with Microsoft's connector migration guidance.

Please let me know if you need any additional changes.

fgravato added 11 commits November 12, 2025 12:37

Fix validation errors: update connector IDs, remove duplicate parser …

92bcb16

…columns, update version

Fix KQL validation: Use LookoutEvents parser instead of raw table

0d0339f

Update analytic rule versions to 2.0.3 for V2 rules

13c49e5

Update 3.0.2.zip package with current mainTemplate and createUiDefini…

e8259ac

…tion

Add pre-submission validation report

52bfc39

Add AGENTS.md with validation commands and coding conventions

e793c96

Revert to version 3.0.1 and fix BasePath per Microsoft feedback

fd55bbd

Update LookoutMtdV2_CL schema with all raw field names for KQL valida…

9f33495

…tion

Add missing workbooks to Solution_Lookout.json

89e5582

Add deployment evidence with connector screenshot reference

48a8d15

fgravato requested review from a team as code owners November 17, 2025 17:09

v-shukore self-assigned this Nov 18, 2025

v-shukore added the Solution Solution specialty review needed label Nov 18, 2025

fgravato and others added 5 commits November 18, 2025 09:02

Update documentation with ARM-TTK validation fix details

c5126ac

Amp-Thread-ID: https://ampcode.com/threads/T-51341ae4-51d0-4987-a64d-ffe634f19ba5 Co-authored-by: Amp <[email protected]>

Update documentation with KQL schema fix details

4b58f34

Add comprehensive validation report - all tests passing

92f108b

fgravato and others added 2 commits November 21, 2025 10:19

Merge remote-tracking branch 'upstream/master' into lookout/v3.0.1-final

1b9e8ae

Update branch from master, add workbook metadata for new workbooks, a…

d337caa

…nd repackage solution Amp-Thread-ID: https://ampcode.com/threads/T-56c53083-3a23-4232-81ab-3ca955e9cf79 Co-authored-by: Amp <[email protected]>

fgravato requested a review from a team as a code owner November 21, 2025 15:31

v-shukore requested a review from Copilot December 1, 2025 08:51

Copilot started reviewing on behalf of v-shukore December 1, 2025 08:53 View session

Copilot finished reviewing on behalf of v-shukore December 1, 2025 08:54

Copilot AI reviewed Dec 1, 2025

View reviewed changes

Fix validation script: check for LookoutAPI connector ID and add exec…

f62b2ab

…ute permissions Amp-Thread-ID: https://ampcode.com/threads/T-56c53083-3a23-4232-81ab-3ca955e9cf79 Co-authored-by: Amp <[email protected]>

v-shukore previously approved these changes Dec 3, 2025

View reviewed changes

v-shukore dismissed their stale review via b649294 December 5, 2025 06:10

fgravato force-pushed the lookout/v3.0.1-final branch from b649294 to 58cacb3 Compare December 5, 2025 07:06

Fix documentation link locale validation - remove en-us locale from URL

95049aa

Amp-Thread-ID: https://ampcode.com/threads/T-3060f9ae-4478-4c9d-861e-9134570ee4a5 Co-authored-by: Amp <[email protected]>

v-shukore added 2 commits December 12, 2025 10:46

Update ReleaseNotes.md

a0723de

Merge branch 'master' into pr/13148

9bf321e

Update 3.0.1.zip

e39e868

v-shukore approved these changes Dec 15, 2025

View reviewed changes

v-atulyadav approved these changes Dec 15, 2025

View reviewed changes

v-atulyadav merged commit c0d66e7 into Azure:master Dec 15, 2025
36 checks passed

		if 'V2' in rule_name and conn_id != 'Lookout-Mobile-Threat-Defense':
		errors.append(f"V2 rule should use 'Lookout-Mobile-Threat-Defense', found '{conn_id}'")

Lookout v3.0.1: Parser fixes, comprehensive and executive dashboards #13148

Lookout v3.0.1: Parser fixes, comprehensive and executive dashboards #13148

Uh oh!

Conversation

fgravato commented Nov 17, 2025

Summary

Changes Made (Per Microsoft Feedback)

Technical Changes

Testing

Files Changed

Related

Uh oh!

fgravato commented Nov 17, 2025

Uh oh!

fgravato commented Nov 20, 2025

Uh oh!

v-shukore commented Nov 20, 2025

Uh oh!

fgravato commented Nov 20, 2025

Uh oh!

v-shukore commented Nov 21, 2025

Uh oh!

v-shukore commented Nov 21, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

fgravato commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fgravato commented Dec 4, 2025

Uh oh!

v-shukore commented Dec 5, 2025

Uh oh!

fgravato commented Dec 5, 2025

Uh oh!

v-shukore commented Dec 5, 2025

Uh oh!

fgravato commented Dec 5, 2025

Uh oh!

v-shukore commented Dec 8, 2025

Uh oh!

fgravato commented Dec 8, 2025

Uh oh!

fgravato commented Dec 10, 2025

Uh oh!

fgravato commented Dec 11, 2025

Uh oh!

v-shukore commented Dec 11, 2025

Uh oh!

v-shukore commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fgravato commented Dec 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fgravato commented Dec 1, 2025 •

edited

Loading

v-shukore commented Dec 12, 2025 •

edited

Loading