Skip to content

Commit 4c9915c

Browse files
committed
Final Updates
1 parent 215d288 commit 4c9915c

File tree

6 files changed

+73
-41
lines changed

6 files changed

+73
-41
lines changed

content/en/ninja-workshops/12-alerting_monitoring_with_itsi/2-creating-basic-alerts/_index.md

Lines changed: 5 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -42,35 +42,19 @@ Splunk alerts are triggered by search results that match specific criteria. We'l
4242
**Time Ranges and Frequency:** Since everything in Splunk core is a search, you need to consider the search timespan and frequency so that you are not a) searching the same data multiple times with an overlap timespan, b) missing events because of a gap between timespan and frequency, c) running too frequently and adding overhead or d) running too infrequently and experiencing delays in alerting.
4343

4444

45-
## 2. AppDynamics Alerts (Health Rule Violations)
45+
## 2. Splunk Observability Cloud Alerts (Detectors)
4646

47-
**2. Create a Health Rule (or modify an existing one):**
48-
* Click "Create Rule" (or edit an existing rule that applies to your application).
49-
* Give the health rule a descriptive name (e.g., "Order Service Response Time Alert").
50-
* **Scope:** Select the application and tier (e.g., "OrderService").
51-
* **Conditions:**
52-
* Choose the metric "Average Response Time."
53-
* Set the threshold: "is greater than" "500" "milliseconds."
54-
* Configure the evaluation frequency (how often AppDynamics checks the metric).
55-
* **Actions:**
56-
* For this basic example, choose "Log to console." In a real-world scenario, you would configure email, SMS, or other notification channels.
57-
* **Save:** Save the health rule.
58-
59-
**Explanation:** This health rule continuously monitors the average response time of the "OrderService." If the response time exceeds 500ms, the health rule is violated, triggering the alert and the configured actions.
60-
61-
62-
## 3. Splunk Observability Cloud Alerts (Detectors)
63-
64-
**2. Create a Detector:**
47+
**Create a Detector:**
6548
* Click "Detectors & SLOs" in the lefthand menu
6649
* Click "Create Detector -> Custom Detector"
6750
* Give the detector a descriptive name (e.g., "High CPU Utilization Alert - INITIALS").
6851
* **Signal:**
6952
* Select the metric you want to monitor ("cpu.utilization").
7053
* Add any necessary filters to specify the host (`service.name:otelshop-loadgenerator`).
54+
* Click "Proceed to Alert Condition"
7155
* **Condition:**
72-
* Set the threshold: "is above" "90" "%."
73-
* Configure the evaluation frequency and the "for" duration (how long the condition must be true before the alert triggers).
56+
* Select Static Threshold
57+
* Set the threshold: "is above" "90"
7458
* **Notifications:**
7559
* For this example, choose a simple notification method (e.g., a test webhook). In a real-world scenario, you would configure integrations with PagerDuty, Slack, or other notification systems.
7660
* **Save:** Save the detector.
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
title: Creating an O11y Based Service
3+
linkTitle: 3.1 Creating an O11y Based Service
4+
weight: 2
5+
---
6+
7+
# Starting with an Observability Cloud Based Service
8+
9+
1. **Access Services:** In ITSI click "Configuration", click on "Services".
10+
11+
2. **Create New Service: PaymentService2:** Click "Create New Service".
12+
13+
3. **Service Details (PaymentService2):**
14+
* **Title:** "PaymentService2"
15+
* **Description (Optional):** e.g., "Payment Service for Hipster Shop - version 2"
16+
17+
4. **Select Template:** Choose "Link service to a service template" and search for "Splunk APM Business Workflow KPIs" from the template dropdown. Click **Create** to save the new service.
18+
19+
6. **Entity Assignment:**
20+
* The page will load and display the new Service and you will be on the Entities page. This demo defaults to selecting the *paymentservice:grpc.hipstershop.PaymentService/Charge* entity. In a real world situation you would need to match the workflow to the entity name manually.
21+
* **Direct Entity Selection (If Available):** Search for the entity using `sf_workflow="paymentservice:grpc.hipstershop.PaymentService/Charge"` and select it.
22+
23+
7. **Save Service (PaymentService2):** Click "Save" to create "PaymentService2".
24+
25+
8. **Settings:** Click the "Settings" tab, enable *Backfill* and keep that standard 7 days. Enable the Service, and click "Save"
26+
27+
## Setting PaymentService2's Service Health as a Dependency for Online-Boutique-US
28+
29+
1. **Locate Online-Boutique-US:** Find the "Online-Boutique-US" service in the service list.
30+
31+
2. **Edit Online-Boutique-US:** Click "Edit".
32+
33+
3. **Service Dependencies:** Look for the "Service Dependencies" section.
34+
35+
4. **Add Dependency:** There should be an option to add a dependent service. Search for "PaymentService2".
36+
37+
5. **Select KPI:** Check the box next to ServiceHealthScore for PaymentService2.
38+
39+
6. **Save Changes:** Save the changes to the "Online-Boutique-US" service.
40+
41+
## Verification
42+
43+
* Click on "Service Analyzer" and select the "Default Analyzer"
44+
* Filter the service to just "Buttercup Business Health"
45+
* Verify that *PaymentService2* is now present below *Online-Boutique-US* and should be in a grey status.
46+
47+
![show-entry](../images/service_tree_o11y.png?classes=inline)

content/en/ninja-workshops/12-alerting_monitoring_with_itsi/3-creating-services-in-itsi/2-creating-appd-service.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,35 +8,35 @@ weight: 3
88

99
1. **Access Services:** In ITSI click "Configuration", click on "Services".
1010

11-
2. **Create New Service: AD-Ecommerce2:** Click "Create New Service".
11+
2. **Create Service: AD-Ecommerce2:** Click "Create Service -> Create Service".
1212

1313
3. **Service Details (AD-Ecommerce2):**
1414
* **Title:** "AD-Ecommerce2"
1515
* **Description (Optional):** e.g., "Ecommerce Service - version 2"
1616

1717
4. **Select Template:** Choose "Link service to a service template" and search for "AppDynamics App Performance Monitoring" from the template dropdown. Click **Create** to save the new service.
1818

19-
6. **Entity Assignment:**
19+
5. **Entity Assignment:**
2020
* The page will load and display the new Service and you will be on the Entities page. This demo defaults to selecting the *AD-Ecommerce:18112:demo1.saas.appdynamics.com* entity. In a real world situation you would need to match the entity_name to the entity name manually.
2121
* **Direct Entity Selection (If Available):** Search for the entity using `entity_name="AD-Ecommerce:18112:demo1.saas.appdynamics.com"` and select it.
2222

23-
7. **Save Service (AD-Ecommerce2):** Click "Save" to create "AD-Ecommerce2".
24-
25-
8. **Settings:** Click the "Settings" tab, enable *Backfill* and keep that standard 7 days. Enable the Service, and click "Save"
23+
7. **Settings:** Click the "Settings" tab, enable *Backfill* and keep that standard 7 days. Enable the Service, and click "Save"
2624

2725
## Setting AD-Ecommerce2's Service Health as a Dependency for AD.Ecommerce
2826

29-
1. **Locate AD.Ecommerce:** Find the "AD.Ecommerce" service in the service list.
27+
1. **Navigate back to Services page:** Click "Configuration -> Services"
28+
29+
2. **Locate AD.Ecommerce:** Find the "AD.Ecommerce" service in the service list.
3030

31-
2. **Edit AD.Ecommerce:** Click "Edit".
31+
3. **Edit AD.Ecommerce:** Click "Edit".
3232

33-
3. **Service Dependencies:** Look for the "Service Dependencies" section.
33+
4. **Service Dependencies:** Look for the "Service Dependencies" section.
3434

35-
4. **Add Dependency:** There should be an option to add a dependent service. Search for "AD-Ecommerce2".
35+
5. **Add Dependency:** There should be an option to add a dependent service. Search for "AD-Ecommerce2".
3636

37-
5. **Select KPI:** Check the box next to ServiceHealthScore for AD-Ecommerce2.
37+
6. **Select KPI:** Check the box next to ServiceHealthScore for AD-Ecommerce2.
3838

39-
6. **Save Changes:** Save the changes to the "AD.Ecommerce" service.
39+
7. **Save Changes:** Save the changes to the "AD.Ecommerce" service.
4040

4141
## Verification
4242

content/en/ninja-workshops/12-alerting_monitoring_with_itsi/3-creating-services-in-itsi/_index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,4 @@ We have two existing services: "Online-Boutique-US" (representing an application
1616

1717
**Return to your Splunk Environment and under Apps, select IT Service Intelligence**
1818

19+
In the Default Analyzer update the Filter to "Buttercup Business Health"

content/en/ninja-workshops/12-alerting_monitoring_with_itsi/4-creating-alerts-in-itsi/_index.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,20 +26,20 @@ This section guides you through configuring a basic alert in Splunk IT Service I
2626
```
2727
index=itsi_summary kpi="*KPI*" alert_severity=critical
2828
```
29-
* **Time Range:** Last 5 minutes
29+
* **Time Range:** Last 15 minutes
3030
* **Service:** *Service Name*
3131
* **Entity Lookup Field:** itsi_service_id
32-
* **Run Every:** 5 minutes
32+
* **Run Every:** minutes
3333
* **Notable Event Title:** *Service Name* *KPI* Critical
34+
* **Severity:** Critical
3435
* **Notable Event Identified Fields:** source
3536

36-
![show-entry](../images/alerts.png?classes=inline)
37-
3837
**After Creating the Alert:**
3938

4039
* You will need to wait 5-10 minutes for the alert to run
4140
* The alert will be listed in the "Alerts and Episodes" Pane in ITSI.
4241

42+
![show-entry](../images/alerts.png?classes=inline)
4343

4444
**Important Considerations:**
4545

content/en/ninja-workshops/12-alerting_monitoring_with_itsi/5-episodes-in-itsi/_index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,15 @@ This section outlines the steps to create an aggregation policy in Splunk ITSI t
1515

1616
## Steps
1717

18-
1. **Navigate to Notable Event Aggregation Policies:** In Splunk, go to the Notable Event Aggregation Policies section (usually under "Configuration" -> "Notable Event Aggregation Policies").
18+
1. **Navigate to Notable Event Aggregation Policies:** In Splunk, go to "Configuration" -> "Notable Event Aggregation Policies".
1919

2020
2. **Create New Policy:** click the green "Create Notable Event Aggregation Policy" button in the upper right corner.
2121

2222
3. **Filtering Criteria:** This is the most important part. You'll define the criteria for alerts to be grouped by this policy. Click "Add Rule (OR)"
2323

2424
* **Field:** Select "title" from the dropdown menu.
2525
* **Operator:** Choose "matches".
26-
* **Value:** Enter the string "*Service Name**".
26+
* **Value:** Enter the string "*Service Name**". (make sure to include the *)
2727

2828
4. **Splitting Events:** Remove the "hosts" field that is provided by default and update it to use the "service" field. We want this generating new episodes for each Service that is found. In our example, it should only be 1.
2929

@@ -35,15 +35,15 @@ This section outlines the steps to create an aggregation policy in Splunk ITSI t
3535

3636
8. **Click Next**
3737

38-
9. **Policy Name and Description:**
38+
9. **Policy Title and Description:**
3939
* **Policy Title:** *Service Name* Alert Grouping
4040
* **Description:** Grouping *Service Name* alerts together.
4141

42-
8. **Save Policy:** Click the "Save" button to create the aggregation policy.
42+
8. **Save Policy:** Click the "Next" button to create the aggregation policy.
4343

4444
## Verification
4545

46-
After saving the policy, navigate to the "Alerts and Episodes" page and filter alerts for last 15 minutes and add a filter to status=New and search for our Service Name in the search box.
46+
After saving the policy, navigate to the "Go to Episode Review" page and filter alerts for last 15 minutes and add a filter to status=New and search for our Service Name in the search box.
4747

4848
There may already be an episode named after our specific alert already, if so, close it out and wait for a new one to be generated with our new Title.
4949

0 commit comments

Comments
 (0)