Skip to content

Comments

Tools/solutions analyzer/telemetry#13666

Merged
oshezaf merged 9 commits intomasterfrom
tools/solutions-analyzer/telemetry
Feb 20, 2026
Merged

Tools/solutions analyzer/telemetry#13666
oshezaf merged 9 commits intomasterfrom
tools/solutions-analyzer/telemetry

Conversation

@oshezaf
Copy link
Contributor

@oshezaf oshezaf commented Feb 20, 2026 •
edited
Loading

v7.9.2 - CCF Legacy, Capabilities Statistics, and ASIM Parser Fixes

CCF (Legacy) Collection Method:

  • New CCF (Legacy) collection method for connectors with embedded pollingConfig in their primary ARM template and no separate CCF config file
  • Capabilities (auth type, paging, POST) extracted from embedded pollingConfig for legacy CCF connectors
  • Improved find_ccf_config_file() detection: now finds connectors.json (Bitwarden-style) and searches sibling *_ccp/ subdirectories (GCP-style)

CCF Capabilities Statistics:

  • New CCF Capabilities subsection on the statistics page with breakdown of connector kind, authentication methods, and request features across all CCF/CCF Push/CCF Legacy connectors
  • New Nested capability: detects stepType: Nested in CCF config files
  • Improved MvExpand detection: now uses nestedTransformName containing MvExpandTransformer instead of text-based search

Custom Log Table Rules (collect_table_info.py):

  • All _CL tables now correctly marked as supporting Ingestion API
  • _CL tables with lake-only support now also marked as supporting transformations

Bug Fixes:

  • Fixed empty Product column in ASIM union parser pages (e.g., imDns): sub-parsers listed with _Im_ prefix now correctly resolve product names and page links from _ASim_ source parser data
  • Excluded empty parsers (e.g., _Im_Dns_Empty, _Im_AlertEvent_Empty) from the Products table on union parser pages
  • Fixed broken links to sub-parser pages on union parser pages: _Im_ prefixed sub-parsers now correctly link to their _ASim_ parser page files

v7.9.1 - CCF Push and CCF Capabilities

CCF Push Collection Method:

  • New collection method CCF Push for connectors using CCF in push mode (partner pushes data via DCR/DCE)
  • DeployPushConnectorButton + HasDataConnectors pattern now classified as "CCF Push" instead of "CCF"
  • Separate documentation page and metadata for CCF Push in generated docs

CCF Configuration and Capabilities:

  • New ccf_config_file column in connectors.csv: GitHub URL to the CCF configuration file (polling/poller/push config)
  • New ccf_capabilities column in connectors.csv: semicolon-separated capabilities extracted from the config JSON
  • Capabilities include: connector kind (GCP, Push, etc.), auth type (APIKey, OAuth2, Basic, JwtToken), Paging, POST, MvExpand
  • Connector detail pages in generated docs now display CCF Configuration link and CCF Capabilities for CCF/CCF Push connectors

v7.9 - Kusto Uploader

New Tool: upload_to_kusto.py

  • Upload CSV files to Azure Data Explorer (Kusto) clusters
  • Solution Analyzer mode (--solution-analyzer): uploads all 10 Solution Analyzer CSVs with predefined table names
  • Custom CSV mode: upload any CSV files with automatic schema detection
  • Local source directory (--source-dir): read Solution Analyzer CSVs from a local folder instead of downloading from GitHub
  • Dry run mode (--dry-run): preview operations before executing
  • Uses Azure CLI authentication and Kusto queued ingestion
  • All columns created as string type with automatic CSV mapping

@oshezaf
Add Kusto uploader tool to Solutions Analyzer (v7.9)
af80102 
- New upload_to_kusto.py script for uploading CSV files to Azure Data Explorer
- Solution Analyzer mode uploads all 10 CSVs with predefined table names
- Supports local source directory (--source-dir) or GitHub download
- Added script documentation in script-docs/upload_to_kusto.md
- Updated README.md with new tool entry and v7.9 changelog
@oshezaf
Fix: skip CSV header row during Kusto ingestion
c4316a4 
Add ignore_first_record=True to both ingestion paths to prevent
the CSV header line from being uploaded as a data row.
@oshezaf oshezaf requested a review from a team as a code owner February 20, 2026 07:43
@oshezaf
Add CCF Push collection method, CCF config file discovery, and CCF ca…
c00a677 
…pabilities extraction

- New 'CCF Push' collection method for connectors using DCR/DCE push ingestion
  (11 connectors, detected via DeployPushConnectorButton pattern)
- New ccf_config_file column in connectors.csv with GitHub URL to CCF config
  (122 out of 133 CCF/CCF Push connectors have config files identified)
- New ccf_capabilities column extracting auth type, paging, POST, MvExpand
  from CCF config JSON (103 connectors with capabilities)
- CCF Push entry in COLLECTION_METHODS_METADATA for doc generation
- Connector detail pages now show CCF Configuration link and CCF Capabilities
- Updated script-docs and README with v7.9.1 changelog
- Added --skip-input-generation to copilot-instructions.md
@oshezaf
Add CCF (Legacy) collection method and fix config file detection\n\n-…
984fe14 
… New CCF (Legacy) type for 9 connectors with embedded pollingConfig and no\n separate CCF config file (Dynatrace x4, Egress, Island x2, LastPass, Seraphic)\n- extract_legacy_ccf_capabilities() extracts auth/paging/POST from pollingConfig\n- Improved find_ccf_config_file(): finds connectors.json (Bitwarden) and\n searches sibling *_ccp/ directories (GCP)\n- CCF (Legacy) added to doc generator COLLECTION_METHODS_METADATA and stats\n- Updated README changelog, mapper and doc generator script docs
@oshezaf
Fix CCF config detection: skip azuredeploy ARM templates\n\nThe _poll…
8fdd036 
…er pattern in CCF_CONFIG_PATTERNS was matching azuredeploy ARM\ntemplate files (e.g., azuredeploy_Okta_native_poller_connector_v2.json)\ninstead of actual CCF config files (e.g., OktaSSOv2_PollingConfig.json).\n\nAdded azuredeploy and mainTemplate to skip patterns in find_ccf_config_file().\nFixes 16 connectors that had wrong config files, including OktaSSOv2.\nOktaSSO_Polling (v1) correctly reclassified as CCF (Legacy) since its\nonly file was the azuredeploy ARM template with embedded pollingConfig.
@oshezaf
Replace em-dash with ? for missing info, update copilot instructions\…
bd0139a 
…n\n- Replace misleading em-dash (—) with ? for 'no information available' in\n generated docs (table features, tables count, solution links, etc.)\n- Copilot instructions: add isBackground:false and timeout:0 for both\n mapper and doc generator scripts to ensure full output visibility
@oshezaf
Fix CCF config self-referencing and add child _ccp dir search\n\nfind…
e1f5d82 
…_ccf_config_file() now skips the connector JSON file itself, preventing\nself-referencing where the primary file was returned as the config file.\nAlso searches child *_ccp/ directories (not just siblings), fixing\nCortexXDR which has CortexXDR_ccp/ as a child of Data Connectors/.\n\nFixes: Phosphorus_Polling, EgressSiemPolling -> CCF (Legacy)\n CortexXDRIncidents -> CCF (finds PollingConfig.json in child _ccp/)
@oshezaf
fix: _CL tables always support Ingestion API; lake-only _CL tables su…
1b208f6 
…pport transformations

Custom log tables (_CL suffix) are not listed in Microsoft's reference docs,
so their capabilities were incorrectly reported. Applied rules:
- All _CL tables: ingestion_api_supported = Yes
- _CL tables with lake-only support: supports_transformations = Yes

Fix applied in both collect_table_info.py (for tables_reference.csv) and
map_solutions_connectors_tables.py (for tables.csv, covering _CL tables
not in the reference).
@oshezaf
Finalize CCF analysis
83b5d54
@oshezaf oshezaf merged commit ba4d29f into master Feb 20, 2026
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yummyblabla yummyblabla yummyblabla approved these changes

Assignees

@yummyblabla yummyblabla

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@oshezaf @yummyblabla