Skip to content

Commit 5d0ab5c

Browse files
himsharma01jpipkin1amee-sumo
authored
Added docs for ec2 auto-scaling (#5199)
* Added docs for ec2 auto-scaling * Updates from review * Updated amazon autoscaling doc --------- Co-authored-by: John Pipkin <[email protected]> Co-authored-by: Amee Lepcha <[email protected]>
1 parent 33d45f7 commit 5d0ab5c

File tree

6 files changed

+206
-11
lines changed

6 files changed

+206
-11
lines changed

‎docs/integrations/amazon-aws/amazon-ec2-auto-scaling.md‎

Lines changed: 206 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,219 @@
11
---
22
id: amazon-ec2-auto-scaling
33
title: Amazon EC2 Auto Scaling
4-
description: Learn about the collection process for the Amazon EC2 Auto Scaling service.
4+
description: The Sumo Logic App for Amazon EC2 Auto Scaling provides comprehensive insights into the performance and health of your EC2 Auto Scaling groups within your AWS environment. This unified logs and metrics (ULM) App offers a centralized view to monitor and analyze the scaling activities and behavior of your EC2 instances. Leverage the prebuilt dashboards to gain visibility into key metrics such as instance launches, terminations, scaling events, and overall group health.
55
---
66

77
import useBaseUrl from '@docusaurus/useBaseUrl';
88

99
<img src={useBaseUrl('img/integrations/amazon-aws/amazon-ec2-auto-scaling-logo.png')} alt="Thumbnail icon" width="50"/>
1010

11-
Amazon EC2 Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. You create collections of EC2 instances, called Auto Scaling groups. You can specify the minimum number of instances in each Auto Scaling group, and Amazon EC2 Auto Scaling ensures that your group never goes below this size. You can specify the maximum number of instances in each Auto Scaling group, and Amazon EC2 Auto Scaling ensures that your group never goes above this size. If you specify the desired capacity, either when you create the group or at any time thereafter, Amazon EC2 Auto Scaling ensures that your group has this many instances. If you specify scaling policies, then Amazon EC2 Auto Scaling can launch or terminate instances as demand on your application increases or decreases. For more details, refer to the [AWS documentation](https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-is-amazon-ec2-auto-scaling.html).
11+
Amazon EC2 Auto Scaling helps you maintain application availability and lets you automatically add or remove EC2 instances using scaling policies that you define. Dynamic or predictive scaling policies let you add or remove EC2 instance capacity to service established or real-time demand patterns.
1212

13-
## Log and metric types
14-
* [CloudWatch Metrics](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-metrics.html)
15-
* [CloudTrail Logs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/logging-using-cloudtrail.html)
13+
The Sumo Logic App for Amazon EC2 Auto Scaling provides comprehensive insights into the performance and health of your EC2 Auto Scaling groups within your AWS environment. This App offers a centralized view to monitor and analyze the scaling activities and behavior of your EC2 instances. Leverage the prebuilt dashboards to gain visibility into key metrics such as instance launches, terminations, scaling events, and overall group health.
1614

15+
## Log and metrics types
1716

18-
## Setup
19-
You can collect the logs and metrics for Sumo Logic's Amazon EC2 Auto Scaling integration by following the below steps.
17+
The Amazon EC2 Auto Scaling app uses the following logs and metrics:
18+
* [Monitor Amazon EC2 Auto Scaling API calls using CloudTrail](https://docs.aws.amazon.com/autoscaling/ec2/userguide/logging-using-cloudtrail.html).
19+
* [Amazon EC2 Auto Scaling metrics](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-metrics.html).
2020

21-
### Configure metrics collection
22-
* Collect **CloudWatch Metrics** with namespace `AWS/AutoScaling` using the [AWS Kinesis Firehose for Metrics](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source/) source. For `AWS/AutoScaling` metrics and dimensions, refer to [Amazon EC2 Auto Scaling CloudWatch metrics](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-metrics.html).
23-
### Configure logs collection
24-
* Collect [AWS CloudTrail Logs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/logging-using-cloudtrail.html) using [AWS CloudTrail](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/) source. Amazon EC2 Auto Scaling is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or a service using Amazon EC2 Auto Scaling. CloudTrail captures all API calls for Amazon EC2 Auto Scaling as events, which includes calls from the Amazon EC2 Auto Scaling console and code calls to the Amazon EC2 Auto Scaling API. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon EC2 Auto Scaling. Using the information collected by CloudTrail, you can determine the request that was made to Amazon EC2 Auto Scaling, the IP address from which the request was made, who made the request, when it was made, and additional details.
21+
### Sample CloudTrail log message
22+
<details>
23+
<summary>Click to expand</summary>
24+
25+
```json title="CloudTrail"
26+
{
27+
"eventVersion":"1.08",
28+
"userIdentity":{
29+
"type":"IAMUser",
30+
"principalId":"AIDAIHL7V6WZEXAMPLEVU",
31+
"arn":"arn:aws:iam::123456789981:user/greg",
32+
"accountId":"123456789981",
33+
"accessKeyId":"AKIA12345EXAMPLE67890",
34+
"userName":"greg",
35+
},
36+
"eventTime":"2025-03-10T15:05:37Z",
37+
"eventSource":"autoscaling.amazonaws.com",
38+
"eventName":"CreateAutoScalingGroup",
39+
"awsRegion":"us-east-1",
40+
"sourceIPAddress":"223.233.233.233",
41+
"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36",
42+
"requestParameters":{
43+
"autoScalingGroupName":"catalogService",
44+
"launchTemplate":{
45+
"launchTemplateId":"lt-02c96752a8d5a34d9",
46+
"version":"$Default"
47+
},
48+
"minSize":1,
49+
"maxSize":3,
50+
"desiredCapacity":2,
51+
"healthCheckType":"EBS",
52+
"healthCheckGracePeriod":300,
53+
"vPCZoneIdentifier":"subnet-010d7504b1d1a23f,subnet-02aff57192da8785",
54+
"newInstancesProtectedFromScaleIn":false,
55+
"defaultInstanceWarmup":-1,
56+
"capacityReservationSpecification":{"capacityReservationPreference":"default"},
57+
"availabilityZoneDistribution":{"capacityDistributionStrategy":"balanced-best-effort"}
58+
},
59+
"responseElements":null,
60+
"requestID":"61a1e435-d8cc-4a7e-a314-6f96c0f016fe",
61+
"eventID":"fcb4f52d-bf15-4732-8d0b-25688be73a47",
62+
"readOnly":false,
63+
"eventType":"AwsApiCall",
64+
"managementEvent":true,
65+
"recipientAccountId":"123456789981",
66+
"eventCategory":"Management",
67+
"tlsDetails":{
68+
"tlsVersion":"TLSv1.3",
69+
"cipherSuite":"TLS_AES_128_GCM_SHA256",
70+
"clientProvidedHostHeader":"autoscaling.us-east-1.amazonaws.com"
71+
},
72+
"sessionCredentialFromConsole":"true"
73+
}
74+
```
75+
</details>
76+
77+
### Sample queries
78+
79+
```sql title="Events by status (CloudTrail logs)"
80+
account="account" region="region" "\"eventsource\":\"autoscaling.amazonaws.com\""
81+
| json "userIdentity", "eventSource", "eventName", "awsRegion", "sourceIPAddress", "userAgent", "eventType", "recipientAccountId", "requestParameters", "responseElements", "requestID", "errorCode", "errorMessage", "apiVersion" as userIdentity, event_source, event_name, region, src_ip, user_agent, event_type, recipient_account_id, requestParameters, responseElements, request_id, error_code, error_message, api_version nodrop
82+
| where event_source = "autoscaling.amazonaws.com"
83+
| where namespace matches "aws/autoscaling" or isEmpty(namespace)
84+
| json field=userIdentity "accountId", "type", "arn", "userName" as accountid, type, arn, username nodrop
85+
| parse field=arn ":assumed-role/*" as user nodrop
86+
| parse field=arn "arn:aws:iam::*:*" as accountid, user nodrop
87+
| json field=requestParameters "autoScalingGroupName" as asgname nodrop
88+
| if (isBlank(accountid), recipient_account_id, accountid) as accountid
89+
| where (tolowercase(asgname) matches tolowercase("{{autoscalinggroup}}")) or isBlank(asgname)
90+
| if (isEmpty(error_code), "Success", "Failure") as event_status
91+
| if (isEmpty(username), user, username) as user
92+
| count by event_status
93+
| sort by _count, event_status asc
94+
```
95+
96+
```bash title="Group Desired Capacity (Cloud watch Metrics)"
97+
account=account region=region autoscalinggroupname=cartService namespace=aws/autoscaling metric=GroupDesiredCapacity Statistic=average
98+
| avg by account, region, autoscalinggroupname
99+
```
100+
101+
## Collecting logs and metrics for the Amazon EC2 Auto Scaling
102+
When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see [Configure a Hosted Collector and Source](/docs/send-data/hosted-collectors/configure-hosted-collector).
103+
104+
### Collect Cloudtrail logs
105+
106+
1. Configure a Amazon EC2 Auto Scaling [Cloudtrail Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-cloudtrail-source/).
107+
1. **Metadata**. Click the **+Add Field** link to add custom log metadata [Fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
108+
1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
109+
1. Keep in mind:
110+
* ![green check circle.png](/img/reuse/green-check-circle.png) A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
111+
* ![orange exclamation point.png](/img/reuse/orange-exclamation-point.png) An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled, in the Fields table schema. In this case, an option to automatically add or enable the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema or is disabled it is ignored, known as dropped.
112+
113+
### Collect metrics
114+
115+
1. Sumo Logic supports collecting metrics using two source types:
116+
* Configure an [AWS Kinesis Firehose for Metrics Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-metrics-source) (recommended); or
117+
* Configure an [Amazon CloudWatch Source for Metrics](/docs/send-data/hosted-collectors/amazon-aws/amazon-cloudwatch-source-metrics)
118+
1. **Metadata**. Click the **+Add Field** link to add custom log metadata [fields](/docs/manage/fields). Define the fields you want to associate, each field needs a name (key) and value.
119+
1. Add an **account** field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.<br/><img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/AWS-Lambda/Metadata.png')} alt="Metadata" style={{border: '1px solid gray'}} width="500" />
120+
1. Keep in mind:
121+
* ![green check circle.png](/img/reuse/green-check-circle.png) A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
122+
* ![orange exclamation point.png](/img/reuse/orange-exclamation-point.png) An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled, in the Fields table schema. In this case, an option to automatically add or enable the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema or is disabled it is ignored, known as dropped.
123+
:::note
124+
Namespace for Amazon EC2 Auto-scaling Service is AWS/AutoScaling.
125+
:::
126+
127+
## Field in field schema
128+
129+
1. [**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Logs > Fields**. <br/>[**New UI**](/docs/get-started/sumo-logic-ui). In the top menu select **Configuration**, and then under **Logs** select **Fields**. You can also click the **Go To...** menu at the top of the screen and select **Fields**.
130+
1. Search for the `autoscalinggroup` field.
131+
1. If not present, create it. Learn how to create and manage fields [here](/docs/manage/fields.md#manage-fields).
132+
133+
## Field Extraction Rule(s)
134+
135+
Create a Field Extraction Rule (FER) for Amazon EC2 Auto-scaling access logs and Cloudtrail logs. Learn how to create a Field Extraction Rule [here](/docs/manage/field-extractions/create-field-extraction-rule).
136+
137+
**Amazon EC2 Auto Scaling CloudTrail logs**
138+
139+
```sql
140+
Rule Name: AwsObservabilityEC2ASGCloudTrailLogsFER
141+
Applied at: Ingest Time
142+
Scope (Specific Data): account=* eventSource eventName
143+
```
144+
145+
```sql title="Parse Expression"
146+
json "eventSource", "awsRegion", "requestParameters", "recipientAccountId" as eventSource, region, requestParameters, accountid nodrop
147+
| json field=requestParameters "autoScalingGroupName" as autoscalinggroup nodrop
148+
| where eventSource = "autoscaling.amazonaws.com"
149+
| "aws/autoscaling" as namespace
150+
| tolowercase(autoscalinggroup) as autoscalinggroup
151+
| fields region, namespace, autoscalinggroup, accountid
152+
```
153+
154+
## Installing the Amazon EC2 Auto Scaling app
155+
156+
Now that you have set up collection for Amazon EC2 Auto Scaling, install the Sumo Logic App to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.
157+
158+
import AppInstallNoDataSourceV2 from '../../reuse/apps/app-install-index-apps-v2.md';
159+
160+
<AppInstallNoDataSourceV2/>
161+
162+
## Viewing Amazon EC2 Auto Scaling dashboards
163+
164+
### Amazon EC2 Autoscaling - Overview
165+
The **Amazon EC2 Autoscaling - Overview** dashboard provides an overview of Amazon EC2 Autoscaling operations, offering insights into instance capacity, error messages, user agents, and disruptive events. It allows users to monitor and analyze the performance and behavior of their autoscaling groups in real-time.
166+
167+
Use this dashboard for:
168+
* Monitoring the current capacity of EC2 instances, including group and warm pool instances
169+
* Tracking the desired capacity of autoscaling groups
170+
* Identifying and troubleshooting common error messages related to autoscaling operations
171+
* Monitoring disruptive events such as group updates, instance detachments, and policy changes
172+
173+
<img src={useBaseUrl('img/integrations/01.-Amazon-EC2-Auto-Scaling-Overview.png')} alt="Amazon EC2 Auto Scaling dashboard" style={{border: '1px solid gray'}} width="800"/>
174+
175+
### Amazon EC2 Auto Scaling - CloudTrail Audit
176+
The **Amazon EC2 Auto Scaling - CloudTrail Audit** dashboard provides a comprehensive overview of Amazon EC2 Auto Scaling activities and CloudTrail audit logs. The dashboard displays information on event locations, top error codes, users with highest failure rates, disruptive events, error details, successful events, event status, and active users/roles.
177+
178+
Use this dashboard for:
179+
* Monitoring the overall health and performance of your Amazon EC2 Auto Scaling groups
180+
* Identifying and troubleshooting common errors and failures in auto-scaling operations
181+
* Tracking user activities and potential security concerns related to auto-scaling events
182+
* Analyzing trends in event types, success rates, and failure patterns over time
183+
184+
<img src={useBaseUrl('img/integrations/02.-Amazon-EC2-Auto-Scaling-CloudTrail-Audit.png')} alt="Amazon EC2 Auto Scaling dashboard" style={{border: '1px solid gray'}} width="800"/>
185+
186+
### Amazon EC2 Auto Scaling - Instances
187+
The **Amazon EC2 Auto Scaling - Instances** dashboard provides a detailed overview of Amazon EC2 Auto Scaling instance metrics, allowing users to monitor and analyze the behavior of their auto scaling groups across different autoscaling groups. It offers real-time insights into instance capacities, states, and group configurations.
188+
189+
Use this dashboard for:
190+
* Tracking the desired capacity and actual in-service instances for each auto scaling group
191+
* Monitoring the minimum and maximum group size limits to ensure proper scaling boundaries
192+
* Observing the total number of instances in each group, including their various states (pending, standby, and terminating)
193+
* Comparing instance metrics across different auto scaling groups
194+
195+
<img src={useBaseUrl('img/integrations/03.-Amazon-EC2-Auto-Scaling-Instances.png')} alt="Amazon EC2 Auto Scaling dashboard" style={{border: '1px solid gray'}} width="800"/>
196+
197+
### Amazon EC2 Auto Scaling - Capacity
198+
The **Amazon EC2 Auto Scaling - Capacity** dashboard provides a comprehensive view of Amazon EC2 Auto Scaling group capacity metrics, offering insights into the total, in-service, pending, standby, and terminating capacities for different autoscaling groups. It allows users to monitor and analyze the capacity fluctuations and states of their auto scaling groups over time.
199+
200+
Use this dashboard for:
201+
* Tracking the total capacity of auto scaling groups across different auto scaling groups
202+
* Monitoring the in-service capacity to ensure adequate resources are available to handle current workloads
203+
* Analyzing standby capacity to understand the reserve resources available for quick scaling
204+
* Tracking terminating capacity to monitor the scale-in process and resource optimization
205+
* Detecting potential issues in the scaling process, such as instances stuck in pending or terminating states
206+
207+
<img src={useBaseUrl('img/integrations/04.-Amazon-EC2-Auto-Scaling-Capacity.png')} alt="Amazon EC2 Auto Scaling dashboard" style={{border: '1px solid gray'}} width="800"/>
208+
209+
### Amazon EC2 Auto Scaling - Warm Pool
210+
The **Amazon EC2 Auto Scaling - Warm Pool** dashboard provides a detailed view of Amazon EC2 Auto Scaling Warm Pool metrics, offering insights into the capacity and state of pre-initialized instances ready to quickly respond to sudden traffic spikes or workload demands. It allows users to monitor and analyze the warm pool behavior across different autoscaling groups.
211+
212+
Use this dashboard for:
213+
* Tracking the total and desired capacity of warm pools for each autoscaling group
214+
* Monitoring the minimum size of warm pools to ensure adequate reserve capacity
215+
* Observing pending capacity in warm pools to identify potential delays in instance preparation
216+
* Monitoring the terminating capacity in warm pools to understand instance lifecycle
217+
* Optimizing warm pool size and capacity to improve application responsiveness during sudden load increases
218+
219+
<img src={useBaseUrl('img/integrations/05.-Amazon-EC2-Auto-Scaling-Warm-Pool.png')} alt="Amazon EC2 Auto Scaling dashboard" style={{border: '1px solid gray'}} width="800"/>

‎static/img/integrations/01.-Amazon-EC2-Auto-Scaling-Overview.png‎

205 KB
Loading

‎static/img/integrations/02.-Amazon-EC2-Auto-Scaling-CloudTrail-Audit.png‎

451 KB
Loading

‎static/img/integrations/03.-Amazon-EC2-Auto-Scaling-Instances.png‎

130 KB
Loading

‎static/img/integrations/04.-Amazon-EC2-Auto-Scaling-Capacity.png‎

143 KB
Loading

‎static/img/integrations/05.-Amazon-EC2-Auto-Scaling-Warm-Pool.png‎

157 KB
Loading

0 commit comments

Comments
 (0)