SUMO-242752 | Changes in IIS otel app to add metric collection and dashboards (#4579)

* SUMO-242752 | Changes in IIS otel app to add metric collection and dashboards

* Update iis-10-opentelemetry.md

* Update docs/integrations/web-servers/opentelemetry/iis-10-opentelemetry.md

* Apply suggestions from code review

Co-authored-by: Kim (Sumo Logic) <[email protected]>
Co-authored-by: Jagadisha V <[email protected]>

---------

Co-authored-by: Jagadisha V <[email protected]>
Co-authored-by: Kim (Sumo Logic) <[email protected]>

Author: 3 people

docs/integrations/web-servers/opentelemetry/iis-10-opentelemetry.md

@@ -11,19 +11,23 @@ import TabItem from '@theme/TabItem';
 
 <img src={useBaseUrl('img/integrations/microsoft-azure/microsoft_iis_10.png')} alt="thumbnail icon" width="130"/> <img src={useBaseUrl('img/send-data/otel-color.svg')} alt="Thumbnail icon" width="45"/>
 
-The [Internet Information Services](https://learn.microsoft.com/en-gb/iis/get-started/introduction-to-iis/introduction-to-iis-architecture) (IIS) 10 app is a logs app that helps you monitor your IIS web servers' availability. Preconfigured dashboards provide insight into application pools, ASP.NET applications, requests, latency, visitor locations, visitor access types, traffic patterns, errors, web server operations, and access from known malicious sources.
+The [Internet Information Services](https://learn.microsoft.com/en-gb/iis/get-started/introduction-to-iis/introduction-to-iis-architecture) (IIS) 10 app is a logs and metrics app designed to monitor the availability and performance of your IIS web servers. Preconfigured dashboards and searches provide insight into application pools, ASP.NET applications, requests, latency, visitor locations, visitor access types, traffic patterns, errors, web server operations, and access from known malicious sources.
 
 IIS logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver).
 
+IIS metrics are sent to Sumo Logic through OpenTelemetry through [windowsperfcountersreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/windowsperfcountersreceiver). This receiver captures the configured system, application, or custom performance counter data from the Windows registry using the [PDH interface](https://docs.microsoft.com/en-us/windows/win32/perfctrs/using-the-pdh-functions-to-consume-counter-data). It is based on the [Telegraf Windows Performance Counters Input Plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters).
+
 <img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Schematics.png' alt="Schematics" />
 
 ## Fields creation in Sumo Logic for IIS
 
 Following are the [Fields](/docs/manage/fields/) which will be created as part of IIS App install if not already present.
 
-- `webengine.cluster.name`. User configured.Enter a name to uniquely identify your IIS web server cluster. This web server cluster name will be shown in the Sumo Logic dashboards.
-- `webengine.system`. Has fixed value of **iis**
 - `sumo.datasource`. Has fixed value of **iis**.
+- `deployment.environment`. User configured. This is the deployment environment where the IIS cluster resides. For example: `dev`, `prod`, or `qa`.
+- `webengine.cluster.name`. User configured. Enter a name to uniquely identify your IIS web server cluster. This web server cluster name will be shown in the Sumo Logic dashboards.
+- `webengine.system`. Has fixed value of **iis**.
+- `webengine.node.name`. Holds the value of the Fully Qualified Domain Name (FQDN) of the machine from which the OpenTelemetry collector is collecting logs and metrics.
 
 ## Prerequisites
 
@@ -104,9 +108,11 @@ import SetupColl from '../../../reuse/apps/opentelemetry/set-up-collector.md';
 
 In this step, you will configure the yaml required for IIS Collection.
 
-Path of the different log file configured to capture IIS logs is needed to be given here (see [Prerequisites](#prerequisites)).
+The path of the log file configured to capture IIS logs is needed to be given here, refer to [Prerequisites](#prerequisites).
+
+Metrics for IIS app are collected through windows perf counters. You can specify the **Collection Interval** to specify at what interval should the metrics be scrapped. You can add any custom fields which you want to tag along with the data ingested in Sumo Logic. 
 
-You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the **Download YAML File** button to get the yaml file.
+Click on the **Download YAML File** button to get the yaml file.
 
 <img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-YAML.png' style={{border:'1px solid gray'}} alt="YAML" />
 
@@ -166,25 +172,71 @@ import LogsOutro from '../../../reuse/apps/opentelemetry/send-logs-outro.md';
 
 <LogsOutro/>
 
+:::note
+A warning message will be printed if any one of the specified performance counters cannot be loaded on startup. The application will not fail fast with this warning. It is expected that some performance counters may not exist on some systems due to different OS configuration.
+:::
+
 ## Sample log messages
 
 ```sql title="Sample Log Message - Non-Kubernetes environments"
 2023-01-13 10:56:55 10.0.0.111 GET / ProgramID=236 443 - 207.235.176.5 Mozilla/5.0+(compatible;+Nimbostratus-Bot/v1.3.2;+http://cloudsystemnetworks.com) http://www.google.com/url?sa=t&rct=j&q=anomaly%20detection&source=web&cd=4 304 11 1236 70
 ```
 
-## Sample queries
+## Sample metrics
+```
+{
+    "queryId": "A",
+    "_source": "iis/windowsperfcounters",
+    "_metricId": "UHWuWs-fdl2-SSVRh6Yfww",
+    "webengine.node.name": "EC2AMAZ-ENUFFVK",
+    "_sourceName": "iis",
+    "host.group": "anemawiniis",
+    "os.type": "windows",
+    "webengine.cluster": "test",
+    "sumo.datasource": "iis",
+    "instance": "*",
+    "_sourceCategory": "OTC Metric Input",
+    "deployment.environment": "anemawiniis",
+    "_contentType": "win_password",
+    "host.name": "EC2AMAZ-ENUFFVK",
+    "metric": "win.aspnet.Request.Execution.Time",
+    "_collectorId": "00005AF310C7F19E",
+    "_sourceId": "0000000000000000",
+    "webengine.system": "iis",
+    "_sourceHost": "EC2AMAZ-ENUFFVK",
+    "_collector": "EC2AMAZ-ENUFFVK",
+    "max": 0,
+    "min": 0,
+    "avg": 0,
+    "sum": 0,
+    "latest": 0,
+    "count": 1
+}
+```
+
+## Sample log query
 
 This sample Query is from the **IIS - Overview** > **Visitor Location** panel.
 
 ```sql title="Query String"
 " %\"sumo.datasource\"=iis %\"webengine.cluster.name\"=* | json \"log\" as _rawlog nodrop \n| if (isEmpty(_rawlog), _raw, _rawlog) as iis_log_message\n| parse regex field=iis_log_message \"(?<server_ip>\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}) (?<method>\\S+?) (?<cs_uri_stem>\\S+?) (?<cs_uri_query>\\S+?) (?<s_port>\\S+?) (?<cs_username>\\S+?) (?<c_ip>\\S+?) (?<cs_User_Agent>\\S+?) (?<cs_referer>\\S+?) (?<sc_status>\\S+?) (?<sc_substatus>\\S+?) (?<sc_win32_status>\\S+?) (?<time_taken>\\S+?)$\"\n| count by c_ip\n| lookup latitude, longitude, country_name from geo://location on ip=c_ip\n| where !isNull(latitude)"
 ```
 
+## Sample metric query
+
+```sql title="Running Application pool"
+sumo.datasource=iis  deployment.environment=* webengine.cluster.name=* webengine.node.name=* instance=*  metric=win.app.pool.was.Current.Application.Pool.State  | filter latest = 3 | count
+```
+
 ## Viewing IIS dashboards
 
 ### Overview
 
-The **IIS - Overview** dashboard provides a high-level view of the integrity of your Microsoft Internet Information Services (IIS) infrastructure. Dashboard panels display visual graphs and detailed information on IIS versions, platforms, and log formats. Panels also show visitor geographic locations, top app requests. OS platforms, response status, response times, and client and server errors.
+The **IIS - Overview** dashboard provides a high-level view of the performance and integrity of your Microsoft Internet Information Services (IIS) infrastructure. Dashboard panels display visual graphs and detailed information on IIS versions, platforms, and log formats. Panels also show visitor geographic locations, top app requests. OS platforms, response status, response times, and client and server errors.
+
+Use this dashboard to:
+* Get a high-level overview of sites, requests, connect, cache, data received and sent, queue, application pool, client location, client platforms, error and threats identified.
+* Drill Down to specific use cases by clicking on specific panels of interest.
 
 <img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Overview.png' alt="Overview" />
 
@@ -235,3 +287,34 @@ The **IIS - Visitor Locations** dashboard provides a high-level view of Nginx vi
 The **IIS - Visitor Traffic Insight** Dashboard provides detailed information on the top documents accessed, top referrers, top search terms from popular search engines, and the media types served.
 
 <img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Visitor-Traffic-Insights.png' alt="Visitor Traffic Insights" />
+
+### Application Pool
+
+The **IIS - Application Pool** dashboard provides a high-level view of Application Pool State, Information and Worker Process Metrics.
+
+<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Application-Pool.png' alt="IIS-Application-Pool" />
+
+### ASP.NET
+
+The **IIS - ASP.NET** dashboard provides a high-level view of the ASP.NET global performance counters. This dashboard helps you to analyse the state server sessions, monitor applications performance, and understand the request execution and wait time.
+
+<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-ASP.NET.png' alt="IIS-ASP.NET" />
+
+### ASP.NET Applications
+
+The **IIS - ASP.NET Applications** dashboard provides a high-level view of the ASP.NET application performance counters. This dashboard helps you to monitor compilations, errors, cache, requests executing, requests in application queue, pipeline instance count, and output cache.
+
+<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-ASP.NET-Applications.png' alt="IIS-ASP.NET-Applications" />
+
+### Cache Performance
+
+The **IIS - Cache Performance** dashboard provides a high-level view of the the Web Service Cache Counters object includes cache counters specific to the World Wide Web Publishing Service. This dashboard helps you to monitor the output cache, cache memory, file cache, and URI cache.
+
+<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Cache-Performance.png' alt="IIS-Cache-Performance" />
+
+### Web Service
+
+The **IIS - Web Service** dashboard provides a high-level view of the Web Service object includes counters specific to the World Wide Web Publishing Service. This dashboard helps you to monitor the total site, connections, site uptime, method, and miscellaneous.
+
+<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/IIS-OpenTelemetry/IIS-Web-Service.png' alt="IIS-Web-Service" />
+