Skip to content

Microsoft 365 Secure Score Foundry Script

Jump to bottom
directorcia edited this page Jan 21, 2026 · 1 revision

Overview

The o365-secure-score-foundry.ps1 script is a comprehensive PowerShell tool that collects Microsoft 365 security posture data and leverages Azure OpenAI Foundry agents to provide AI-powered security analysis and recommendations. The script generates a detailed HTML report with actionable insights to improve your tenant's security posture.

Key Features

  • Automated Data Collection: Gathers security data from Microsoft Graph API including:

    • Secure Score metrics and trends
    • Security control implementations
    • Conditional Access policies
    • MFA adoption statistics
    • Security defaults configuration

  • AI-Powered Analysis: Sends collected data to Azure OpenAI Foundry agents for intelligent analysis

    • Identifies security gaps and vulnerabilities
    • Provides prioritized recommendations
    • Generates implementation roadmaps
    • Explains business impact and threat protection

  • Professional HTML Reports: Creates interactive, visually appealing reports with:

    • Executive summaries
    • Detailed security findings
    • Interactive sections with collapsible content
    • Mermaid diagrams for visual representations
    • Markdown-rendered AI analysis
    • Dark/light theme support

  • Flexible Operation Modes: Three modes to suit different workflows

    • Collect: Gather data and save to JSON file
    • Analyze: Use existing JSON data for analysis
    • Both: Collect data and analyze in one run (default)

  • Token Caching: Intelligent authentication caching to minimize re-authentication

    • Caches both Microsoft Graph and Azure AD tokens
    • Automatic token expiry handling
    • Reduces authentication prompts for better user experience

Prerequisites

Required Software

  • PowerShell 5.1 or later (PowerShell 7+ recommended)
  • Microsoft.Graph PowerShell Module 
    Install-Module Microsoft.Graph -Scope CurrentUser

Required Azure Resources

  1. Azure AD App Registration (for Foundry authentication)

    • Application (Client) ID
    • Tenant ID where Foundry is deployed
    • Device code flow enabled

  2. Azure OpenAI Foundry Instance

    • Foundry deployment URL
    • Foundry Agent ID

Required Permissions

The script requires the following Microsoft Graph API permissions:

  • SecurityEvents.Read.All - Read security events and Secure Score
  • Policy.Read.All - Read Conditional Access policies
  • Reports.Read.All - Read usage reports and MFA statistics
  • Directory.Read.All - Read directory data

These permissions can be granted through:

  • Interactive consent during first connection
  • Pre-consented admin permissions in your tenant

Parameters

Required Parameters

-TenantDomain (string)

The primary domain of your Microsoft 365 tenant (e.g., "contoso.onmicrosoft.com").

Example:

-TenantDomain "contoso.onmicrosoft.com"

Optional Parameters

-Mode (ValidateSet)

Operation mode for the script.

Valid Values:

  • Collect - Only collect data and save to JSON file
  • Analyze - Use existing JSON data for analysis (requires -DataFile)
  • Both - Collect data and analyze (default)

Default: Both

Example:

-Mode "Collect"  # Only gather data
-Mode "Analyze"  # Only analyze existing data
-Mode "Both"     # Do both in one run

-DataFile (string)

Path to save/load the JSON data file.

Default: Auto-generated filename with timestamp in script directory

Example:

-DataFile "C:\Reports\tenant-data.json"

-FoundryInstanceUrl (string)

The Azure OpenAI Foundry endpoint URL.

Default: https://ciafoundry1-resource.openai.azure.com/openai/deployments/model-router/chat/completions

Example:

-FoundryInstanceUrl "https://your-foundry.openai.azure.com/openai/deployments/model-router/chat/completions"

-FoundryAgentId (string)

The Foundry Agent ID to use for analysis.

Default: Agent508new:47

Example:

-FoundryAgentId "Agent123:45"

-AzureADClientId (string)

Application (Client) ID of your Azure AD app registration.

Required for Foundry authentication.

Example:

-AzureADClientId "12345678-1234-1234-1234-123456789abc"

-AzureADTenantId (string)

Tenant ID where your Foundry instance is deployed.

Required for Foundry authentication.

Example:

-AzureADTenantId "87654321-4321-4321-4321-cba987654321"

-Detailed (switch)

Enable detailed analysis mode for comprehensive reports.

When enabled:

  • AI provides more thorough analysis
  • More detailed recommendations
  • Implementation roadmaps
  • Business impact explanations
  • Longer analysis time (60-120 seconds vs 30-60 seconds)
  • Higher token usage

Example:

-Detailed

Usage Examples

Example 1: Basic Usage (Collect and Analyze)

Collect data from tenant and generate AI-powered report in one run.

.\o365-secure-score-foundry.ps1 `
    -TenantDomain "contoso.onmicrosoft.com" `
    -AzureADClientId "12345678-1234-1234-1234-123456789abc" `
    -AzureADTenantId "87654321-4321-4321-4321-cba987654321"

Example 2: Detailed Analysis

Generate comprehensive report with detailed recommendations.

.\o365-secure-score-foundry.ps1 `
    -TenantDomain "contoso.onmicrosoft.com" `
    -AzureADClientId "12345678-1234-1234-1234-123456789abc" `
    -AzureADTenantId "87654321-4321-4321-4321-cba987654321" `
    -Detailed

Example 3: Data Collection Only

Collect data and save to JSON for later analysis.

.\o365-secure-score-foundry.ps1 `
    -TenantDomain "contoso.onmicrosoft.com" `
    -Mode "Collect" `
    -DataFile "C:\Reports\tenant-2026-01-22.json"

Example 4: Analyze Existing Data

Analyze previously collected data without re-collecting.

.\o365-secure-score-foundry.ps1 `
    -TenantDomain "contoso.onmicrosoft.com" `
    -Mode "Analyze" `
    -DataFile "C:\Reports\tenant-2026-01-22.json" `
    -AzureADClientId "12345678-1234-1234-1234-123456789abc" `
    -AzureADTenantId "87654321-4321-4321-4321-cba987654321" `
    -Detailed

Example 5: Custom Foundry Instance

Use a custom Foundry deployment.

.\o365-secure-score-foundry.ps1 `
    -TenantDomain "contoso.onmicrosoft.com" `
    -FoundryInstanceUrl "https://my-foundry.openai.azure.com/openai/deployments/model-router/chat/completions" `
    -FoundryAgentId "MyAgent:10" `
    -AzureADClientId "12345678-1234-1234-1234-123456789abc" `
    -AzureADTenantId "87654321-4321-4321-4321-cba987654321"

Workflow

Phase 1: Data Collection (if Mode = "Collect" or "Both")

  1. Authentication

    • Checks for existing Microsoft Graph session
    • Connects to Microsoft Graph with required scopes
    • Tests Graph API connectivity

  2. Data Collection (5 sequential steps)

    • Step 1: Secure Score data (current score, trends, max possible)
    • Step 2: Security controls (recommendations, implementation status)
    • Step 3: Conditional Access policies (configurations, states, conditions)
    • Step 4: MFA adoption metrics (registration, usage statistics)
    • Step 5: Security defaults status (enabled/disabled)

  3. Data Persistence

    • Saves collected data to JSON file
    • Creates timestamped filename if not specified
    • Compresses data for efficient storage

Phase 2: AI Analysis (if Mode = "Analyze" or "Both")

  1. Foundry Authentication

    • Checks for cached token
    • Initiates device code flow if needed
    • Displays code for browser authentication
    • Polls for authentication completion
    • Caches token for future use

  2. Data Preparation

    • Loads JSON data (from file or memory)
    • Summarizes large datasets to optimize AI analysis
    • Validates data structure

  3. AI Processing

    • Sends security data to Foundry agent
    • AI analyzes security posture
    • Generates recommendations and insights
    • Model router automatically selects best AI model
    • Typical processing time: 30-90 seconds

  4. Report Generation

    • Parses AI response (Markdown + JSON)
    • Extracts structured recommendations
    • Builds interactive HTML report
    • Includes:
      • Executive summary
      • Security score trends
      • Control recommendations
      • Conditional Access policies
      • MFA statistics
      • AI analysis with diagrams
      • Implementation guides

Phase 3: Output

  1. HTML Report

    • Saved to script directory
    • Timestamped filename
    • Automatically opens in default browser
    • Features:
      • Responsive design
      • Collapsible sections
      • Syntax highlighting
      • Mermaid diagram rendering
      • Dark/light theme support

  2. Console Summary

    • Displays execution time
    • Shows file locations
    • Provides quick statistics

Authentication Flow

Microsoft Graph Authentication

  • Method: Interactive browser-based OAuth
  • Caching: Session-based (survives for duration of PowerShell session)
  • Scopes: Dynamically requested based on required permissions
  • Re-authentication: Only when session expires or different tenant

Azure AD (Foundry) Authentication

  • Method: Device code flow
  • Caching: File-based token cache (expires after token lifetime - 5 min buffer)
  • Flow:
    1. Script requests device code
    2. Displays code and URL to user
    3. User opens browser and enters code
    4. Script polls for completion
    5. Token cached for future runs
  • Cache Location: %TEMP%\O365SecureScore\foundry-token-{TenantId}.json

Output Files

JSON Data File

Location: Script directory (or specified path)
Naming: SecureScoreData-YYYY-MM-DD-HHmmss.json
Contents:

  • Secure Score metrics
  • Security controls
  • Conditional Access policies
  • MFA statistics
  • Security defaults status
  • Collection timestamp

Example:

SecureScoreData-2026-01-22-153045.json

HTML Report

Location: Script directory
Naming: M365-SecureScore-Analysis-YYYY-MM-DD-HHmmss.html
Contents:

  • Interactive security report
  • AI-powered analysis
  • Recommendations and implementation guides
  • Visual diagrams and charts

Example:

M365-SecureScore-Analysis-2026-01-22-153245.html

Report Sections

1. Header Section

  • Tenant information
  • Report metadata (agent, mode, timestamp)
  • AI-Powered Security Posture Assessment subtitle

2. Secure Score Section

  • Latest score (current/max/percentage)
  • Trend table (if historical data available)
  • Active and licensed user counts

3. Top Open Secure Score Controls

  • Collapsible table of unimplemented controls
  • Ranked by priority
  • Impact, score potential, threats mitigated
  • Implementation state

4. Conditional Access Policies

  • Collapsible policy listing
  • Policy state (enabled/disabled/report-only)
  • Scope (users, groups, applications)
  • Grant and session controls
  • Last modified dates

5. MFA Registration Summary

  • User registration statistics
  • MFA-capable vs registered users
  • Registration percentages

6. AI Analysis Section

  • Executive summary
  • Critical findings
  • Detailed security assessment
  • Risk analysis
  • Recommendations with:
    • Why critical
    • Business impact
    • Implementation steps
    • Testing procedures
    • References and documentation

7. Implementation Guide (Detailed mode)

  • Step-by-step implementation roadmap
  • Prerequisites and dependencies
  • Testing and validation procedures
  • Rollback procedures
  • Additional resources

Troubleshooting

Common Issues

Issue: "Microsoft.Graph.Authentication module not found"

Solution:

Install-Module Microsoft.Graph -Scope CurrentUser -Force

Issue: "Authentication timeout - please try again"

Cause: Device code authentication not completed in time
Solution:

  • Ensure you entered the code in browser within 15 minutes
  • Check internet connectivity
  • Try running script again

Issue: "Graph API connectivity test failed"

Cause: Network issues or insufficient permissions
Solution:

  • Check internet connection
  • Verify you have required Graph permissions
  • Ensure you're logged in as appropriate user

Issue: "Data payload is very large"

Cause: Tenant has extensive security configurations
Solution:

  • Script automatically summarizes data
  • Consider using -Mode Collect to save data for review
  • Large payloads may still work but take longer

Issue: Token cache errors

Solution:

# Clear token cache manually
Remove-Item "$env:TEMP\O365SecureScore\*" -Force

Issue: "Could not parse token expiry time"

Cause: Corrupted token cache
Solution: Script automatically removes corrupted cache and re-authenticates

Debug Mode

Enable verbose output for troubleshooting:

.\o365-secure-score-foundry.ps1 -TenantDomain "contoso.onmicrosoft.com" -Verbose

Performance Considerations

Typical Execution Times

Phase Standard Mode Detailed Mode
Data Collection 2-5 minutes 2-5 minutes
AI Analysis 30-60 seconds 60-120 seconds
Report Generation 2-5 seconds 2-5 seconds
Total 3-6 minutes 4-8 minutes

Times vary based on tenant size, network speed, and AI model load

Optimization Tips

  1. Use token caching: Run script multiple times without re-authenticating
  2. Collect once, analyze multiple times: Use -Mode Collect then multiple -Mode Analyze runs
  3. Standard mode for quick insights: Use detailed mode only when needed
  4. Check network speed: Slow connections impact Graph API calls

Security Best Practices

Credential Management

  • Never hardcode credentials in scripts
  • Use Azure AD app registrations with minimal required permissions
  • Regularly rotate app secrets
  • Enable Conditional Access for admin accounts

Data Handling

  • JSON files contain sensitive security data
  • Store data files securely
  • Delete old data files when no longer needed
  • Restrict file permissions appropriately

Report Distribution

  • HTML reports contain sensitive information
  • Share only with authorized personnel
  • Consider using password-protected archives
  • Remove old reports after review

Advanced Scenarios

Automated Scheduled Runs

Create a scheduled task to run weekly reports:

# Create scheduled script
$scriptPath = "C:\Scripts\o365-secure-score-foundry.ps1"
$outputPath = "C:\Reports"

# Script content with parameters
$scriptContent = @"
.\o365-secure-score-foundry.ps1 ``
    -TenantDomain "contoso.onmicrosoft.com" ``
    -AzureADClientId "12345678-1234-1234-1234-123456789abc" ``
    -AzureADTenantId "87654321-4321-4321-4321-cba987654321" ``
    -DataFile "$outputPath\weekly-$(Get-Date -Format 'yyyy-MM-dd').json" ``
    -Detailed
"@

# Save to file and create scheduled task
$scriptContent | Out-File "C:\Scripts\weekly-security-report.ps1"

Multi-Tenant Analysis

Analyze multiple tenants in sequence:

$tenants = @(
    @{ Domain = "contoso.onmicrosoft.com"; ClientId = "guid1"; TenantId = "guid1" }
    @{ Domain = "fabrikam.onmicrosoft.com"; ClientId = "guid2"; TenantId = "guid2" }
)

foreach ($tenant in $tenants) {
    Write-Host "Processing $($tenant.Domain)..." -ForegroundColor Cyan
    .\o365-secure-score-foundry.ps1 `
        -TenantDomain $tenant.Domain `
        -AzureADClientId $tenant.ClientId `
        -AzureADTenantId $tenant.TenantId `
        -Detailed
}

Custom Data Processing

Use collected JSON for custom analysis:

# Collect data only
.\o365-secure-score-foundry.ps1 -TenantDomain "contoso.onmicrosoft.com" -Mode Collect

# Load and process JSON
$data = Get-Content "SecureScoreData-*.json" -Raw | ConvertFrom-Json

# Custom analysis
$openControls = $data.securityControls | Where-Object { $_.implementationStatus -ne "Implemented" }
Write-Host "Open controls: $($openControls.Count)"

# Export to CSV for Excel analysis
$openControls | Export-Csv "open-controls.csv" -NoTypeInformation

Version History

Current Version: Foundry Agent Version
Last Updated: January 2026

Recent Enhancements

  • Token caching for improved user experience
  • Automatic token expiry handling
  • Enhanced error reporting
  • Improved data summarization
  • Better HTML report styling with high-contrast headings
  • Mermaid diagram support
  • Collapsible report sections
  • Responsive design improvements

Support and Resources

Microsoft Documentation

PowerShell Resources

Azure OpenAI

License

This script is provided as-is for security assessment purposes. Review and test in non-production environments before production use.

Contributing

For issues, enhancements, or questions, please contact your Microsoft 365 security team or Azure OpenAI Foundry administrators.

Script: o365-secure-score-foundry.ps1
Documentation Version: 1.0
Last Updated: January 22, 2026

Clone this wiki locally