diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 45d11e72fa1..70d1d330b24 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -38,6 +38,7 @@ /packages/aws/data_stream/cloudtrail @elastic/obs-infraobs-integrations /packages/aws/data_stream/cloudwatch_logs @elastic/obs-ds-hosted-services /packages/aws/data_stream/cloudwatch_metrics @elastic/obs-ds-hosted-services +/packages/aws/data_stream/config @elastic/security-service-integrations /packages/aws/data_stream/dynamodb @elastic/obs-infraobs-integrations /packages/aws/data_stream/ebs @elastic/obs-ds-hosted-services /packages/aws/data_stream/ec2_logs @elastic/obs-ds-hosted-services diff --git a/packages/aws/_dev/build/docs/config.md b/packages/aws/_dev/build/docs/config.md new file mode 100644 index 00000000000..d5fd01b1be1 --- /dev/null +++ b/packages/aws/_dev/build/docs/config.md @@ -0,0 +1,81 @@ +# Config + +[AWS Config](https://docs.aws.amazon.com/config/) provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time. + +Use this integration to collect and parse data from your AWS Config APIs. Visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference data when troubleshooting an issue. + +**IMPORTANT: Extra AWS charges on API requests will be generated by this integration. Check [API Requests](https://www.elastic.co/docs/current/integrations/aws#api-requests) for more details.** + +## Data streams + +The AWS Config integration collects one type of data: logs. + +**Logs** help you keep a record of the findings in AWS Config, allowing you to track and audit compliance status of your resources. + +The AWS Config integration works by first retrieving all config rules using the [DescribeConfigRules](https://docs.aws.amazon.com/config/latest/APIReference/API_DescribeConfigRules.html) API. Then, for each specific config rule, the integration fetches its evaluation results using the [GetComplianceDetailsByConfigRule](https://docs.aws.amazon.com/config/latest/APIReference/API_GetComplianceDetailsByConfigRule.html) API. These evaluation results enrich their respective config rules, ultimately producing a finding log. + +See more details in the [Logs reference](#logs-reference). + +## Requirements + +### Agentless Enabled Integration +Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html). +Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features. + +### Agent Based Installation +- Elastic Agent must be installed +- You can install only one Elastic Agent per host. +- Elastic Agent is required to stream data from the REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines. + +Before using any AWS integration you will need: + +* **AWS Credentials** to connect with your AWS account. +* **AWS Permissions** to make sure the user you're using to connect has permission to share the relevant data. + +For more details about these requirements, please take a look at the [AWS integration documentation](https://docs.elastic.co/integrations/aws#requirements). + +## Setup + +Use this integration if you only need to collect data from the AWS Config service. + +### To collect data from AWS Config APIs, users must have an Access Key and a Secret Key. To create API token follow below steps: + +1. Login to https://console.aws.amazon.com/. +2. Go to https://console.aws.amazon.com/iam/ to access the IAM console. +3. On the navigation menu, choose Users. +4. Choose your IAM user name. +5. Select Create access key from the Security Credentials tab. +6. To see the new access key, choose Show. + +### Enabling the integration in Elastic: + +1. In Kibana navigate to Management > Integrations. +2. In "Search for integrations" top bar, search for `AWS Config`. +3. Select the "AWS Config" integration from the search results. +4. Select "Add AWS Config" to add the integration. +5. Add all the required integration configuration parameters, including the aws_region to enable data collection. +6. Select "Save and continue" to save the integration. + +**Note** +1. For the current integration package, it is compulsory to add Secret Access Key and Access Key ID. +2. The AWS Config integration performs a full ingestion of all findings during each interval. + +## Logs reference + +### Config + +This is the `config` dataset. + +#### Example + +An example event for `config` looks as following: + +{{event "config"}} + +**ECS Field Reference** + +Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields. + +#### Exported fields + +{{fields "config"}} diff --git a/packages/aws/changelog.yml b/packages/aws/changelog.yml index 3e768179fd7..77f911b964d 100644 --- a/packages/aws/changelog.yml +++ b/packages/aws/changelog.yml @@ -1,4 +1,9 @@ # newer versions go on top +- version: "3.4.0" + changes: + - description: Add new AWS Config datastream. + type: enhancement + link: https://github.com/elastic/integrations/pull/13830 - version: "3.3.3" changes: - description: Update README - Document ingested log types of AWS Network Firewall. diff --git a/packages/aws/data_stream/config/_dev/deploy/docker/docker-compose.yml b/packages/aws/data_stream/config/_dev/deploy/docker/docker-compose.yml new file mode 100644 index 00000000000..a3debfddd9d --- /dev/null +++ b/packages/aws/data_stream/config/_dev/deploy/docker/docker-compose.yml @@ -0,0 +1,17 @@ +version: '2.3' +services: + config: + image: docker.elastic.co/observability/stream:v0.17.0 + hostname: config.xxxx.amazonaws.com + ports: + - 443 + volumes: + - ./files:/files:ro + environment: + PORT: "443" + command: + - http-server + - --addr=:443 + - --config=/files/config.yml + - --tls-cert=/files/certificate.crt + - --tls-key=/files/private.key diff --git a/packages/aws/data_stream/config/_dev/deploy/docker/files/certificate.crt b/packages/aws/data_stream/config/_dev/deploy/docker/files/certificate.crt new file mode 100644 index 00000000000..a84bcdf0cbb --- /dev/null +++ b/packages/aws/data_stream/config/_dev/deploy/docker/files/certificate.crt @@ -0,0 +1,23 @@ +-----BEGIN CERTIFICATE----- +MIID4TCCAsmgAwIBAgIUBdbnNWnUUMxH4YR2GEfqbZN60m8wDQYJKoZIhvcNAQEL +BQAwgYUxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQH +DA1TYW4gRnJhbmNpc2NvMRMwEQYDVQQKDApNeSBDb21wYW55MRAwDgYDVQQLDAdN +eSBVbml0MSIwIAYDVQQDDBljb25maWcueHh4eC5hbWF6b25hd3MuY29tMB4XDTI1 +MDUwNjA2Mjc0M1oXDTI2MDUwNjA2Mjc0M1owgYUxCzAJBgNVBAYTAlVTMRMwEQYD +VQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRMwEQYDVQQK +DApNeSBDb21wYW55MRAwDgYDVQQLDAdNeSBVbml0MSIwIAYDVQQDDBljb25maWcu +eHh4eC5hbWF6b25hd3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC +AQEAwzflKp5qNhAy07KZDXq0cZ0w6HEPWGuPh+1qK+ZxDqbyPBYXtNJv4XXOKmJw +nVH+XriwL9PA6T/R96zIr5GR7mT3lKa4QGdlOLAqFQjDs8HGNpePDLJImPp4Kktw +svRXrvfgsNVGy7qejT2ufK0OgszpVDSH4NaXXdwpGOuXF0e5qLox1DFiUj4N9ntA +Zqw/A9VLDvwuLveO1X4aI3a9xfTuSrLiRvED57rqW+3YJvOEru0SZn7F1pY9+V9j +kPgyTlzK6sv3xbXkt17lK4wzUvPzi0wxDRYuBNmlhZ4oq2ysMNAVbcEMzebOQ1jf +u9LTKeBn4cmltXTH9y9RfMsOkwIDAQABo0cwRTAkBgNVHREEHTAbghljb25maWcu +eHh4eC5hbWF6b25hd3MuY29tMB0GA1UdDgQWBBSc5srLUGJ3wzkdpurLxqRL8dQQ +RDANBgkqhkiG9w0BAQsFAAOCAQEAAz/TkbmDvstJg6Fc0AUWdR4YDN9N4pQXBCJ/ +C4aB+JVHoJfWD3tmXZ1y7or9/q/UXxfutUzSpXzFOq5gG3mlduaDDfgz54tr7Fzf +FjMJMjNwuIxBILi2e6uJAwxuJRn7SmMtNv46PswR8N3XvM4kyTt/11nEB1YE2yr6 +46XFW+1db4ds8lnwmdRYM0j6gCe3jswZ6M3mhF5SNCrp+LCb70LUnsLSnh7LdPp+ +xR+OxIwWBtgT2iL5ArdWJr219Ey40G0bSVPZmtlED4Hi2oue5KIt3MnVzpxIsu8p +UrA2ofnvUjhhk6CKjFBTE7BnkH9u6NAZseQLA42vtHvgm8tu5g== +-----END CERTIFICATE----- diff --git a/packages/aws/data_stream/config/_dev/deploy/docker/files/config.yml b/packages/aws/data_stream/config/_dev/deploy/docker/files/config.yml new file mode 100644 index 00000000000..ff31f285aa6 --- /dev/null +++ b/packages/aws/data_stream/config/_dev/deploy/docker/files/config.yml @@ -0,0 +1,217 @@ +rules: + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.DescribeConfigRules" + request_body: '{"NextToken":"page2"}' + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "ConfigRules": [ + { + "ConfigRuleArn": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id3", + "ConfigRuleId": "config-rule-id3", + "ConfigRuleName": "required-tags", + "ConfigRuleState": "ACTIVE", + "Description": "Checks whether your resources have the tags that you specify.", + "EvaluationModes": [ + { + "Mode": "DETECTIVE" + } + ], + "InputParameters": "{\"tag1Key\":\"k1\",\"tag1Value\":\"v1\"}", + "Scope": { + "ComplianceResourceTypes": [ + "AWS::EC2::Instance" + ] + }, + "Source": { + "Owner": "AWS", + "SourceIdentifier": "REQUIRED_TAGS" + } + } + ] + } + `}} + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.DescribeConfigRules" + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "ConfigRules": [ + { + "ConfigRuleArn": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1", + "ConfigRuleId": "config-rule-id1", + "ConfigRuleName": "access-keys-rotated", + "ConfigRuleState": "ACTIVE", + "Description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "EvaluationModes": [ + { + "Mode": "DETECTIVE" + } + ], + "InputParameters": "{\"maxAccessKeyAge\":\"90\"}", + "MaximumExecutionFrequency": "TwentyFour_Hours", + "Source": { + "Owner": "AWS", + "SourceIdentifier": "ACCESS_KEYS_ROTATED" + } + }, + { + "ConfigRuleArn": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id2", + "ConfigRuleId": "config-rule-id2", + "ConfigRuleName": "account-part-of-organizations", + "ConfigRuleState": "ACTIVE", + "Description": "Rule checks whether AWS account is part of AWS Organizations. The rule is NON_COMPLIANT if the AWS account is not part of AWS Organizations or AWS Organizations master account ID does not match rule parameter MasterAccountId.", + "EvaluationModes": [ + { + "Mode": "DETECTIVE" + } + ], + "InputParameters": "{}", + "MaximumExecutionFrequency": "TwentyFour_Hours", + "Source": { + "Owner": "AWS", + "SourceIdentifier": "ACCOUNT_PART_OF_ORGANIZATIONS" + } + } + ], + "NextToken": "page2" + } + `}} + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.GetComplianceDetailsByConfigRule" + request_body: '{"ConfigRuleName":"access-keys-rotated","Limit":2}' + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "EvaluationResults": [ + { + "ComplianceType": "COMPLIANT", + "ConfigRuleInvokedTime": 1444799479.852, + "EvaluationResultIdentifier": { + "EvaluationResultQualifier": { + "ConfigRuleName": "access-keys-rotated", + "EvaluationMode": "DETECTIVE", + "ResourceId": "i-0a4468fbfafeeg20h", + "ResourceType": "AWS::EC2::Instance" + }, + "OrderingTimestamp": 1443541951.883 + }, + "ResultRecordedTime": 1444799480.061 + }, + { + "ComplianceType": "COMPLIANT", + "ConfigRuleInvokedTime": 1544799479.852, + "EvaluationResultIdentifier": { + "EvaluationResultQualifier": { + "ConfigRuleName": "access-keys-rotated", + "EvaluationMode": "DETECTIVE", + "ResourceId": "i-0a4468fbfafeeg30h", + "ResourceType": "AWS::EC2::Instance" + }, + "OrderingTimestamp": 1543541951.883 + }, + "ResultRecordedTime": 1544799480.061 + } + ], + "NextToken": "page2" + } + `}} + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.GetComplianceDetailsByConfigRule" + request_body: '{"ConfigRuleName":"access-keys-rotated","Limit":2,"NextToken":"page2"}' + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "EvaluationResults": [ + { + "ComplianceType": "NON_COMPLIANT", + "ConfigRuleInvokedTime": 1644799479.852, + "EvaluationResultIdentifier": { + "EvaluationResultQualifier": { + "ConfigRuleName": "access-keys-rotated", + "EvaluationMode": "DETECTIVE", + "ResourceId": "i-0a4468fbfafeeg30h", + "ResourceType": "AWS::EC2::Instance" + }, + "OrderingTimestamp": 1643541951.883 + }, + "ResultRecordedTime": 1644799480.061 + } + ] + } + `}} + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.GetComplianceDetailsByConfigRule" + request_body: '{"ConfigRuleName":"account-part-of-organizations","Limit":2}' + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "EvaluationResults": [] + } + `}} + - path: / + methods: ["POST"] + request_headers: + Content-Type: + - "application/x-amz-json-1.1" + X-Amz-Target: + - "StarlingDoveService.GetComplianceDetailsByConfigRule" + request_body: '{"ConfigRuleName":"required-tags","Limit":2}' + responses: + - status_code: 200 + body: |- + {{ minify_json ` + { + "EvaluationResults": [ + { + "ComplianceType": "NON_COMPLIANT", + "ConfigRuleInvokedTime": 1844799479.852, + "EvaluationResultIdentifier": { + "EvaluationResultQualifier": { + "ConfigRuleName": "required-tags", + "EvaluationMode": "PROACTIVE", + "ResourceId": "i-0a4468fbfafeeg41h", + "ResourceType": "AWS::EC2::Instance" + }, + "OrderingTimestamp": 1843541951.883 + }, + "ResultRecordedTime": 1844799480.061 + } + ] + } + `}} diff --git a/packages/aws/data_stream/config/_dev/deploy/docker/files/private.key b/packages/aws/data_stream/config/_dev/deploy/docker/files/private.key new file mode 100644 index 00000000000..19b87300e6e --- /dev/null +++ b/packages/aws/data_stream/config/_dev/deploy/docker/files/private.key @@ -0,0 +1,28 @@ +-----BEGIN PRIVATE KEY----- +MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDDN+Uqnmo2EDLT +spkNerRxnTDocQ9Ya4+H7Wor5nEOpvI8Fhe00m/hdc4qYnCdUf5euLAv08DpP9H3 +rMivkZHuZPeUprhAZ2U4sCoVCMOzwcY2l48MskiY+ngqS3Cy9Feu9+Cw1UbLup6N +Pa58rQ6CzOlUNIfg1pdd3CkY65cXR7moujHUMWJSPg32e0BmrD8D1UsO/C4u947V +fhojdr3F9O5KsuJG8QPnuupb7dgm84Su7RJmfsXWlj35X2OQ+DJOXMrqy/fFteS3 +XuUrjDNS8/OLTDENFi4E2aWFniirbKww0BVtwQzN5s5DWN+70tMp4GfhyaW1dMf3 +L1F8yw6TAgMBAAECggEABVqznOMia6AvHLJR01ZRu6oBjOaI8rZkhehjmJel8y5u +B2rdtJZu/iKSiIQRrabxkJyFLJKkwGEBO8dP68zU0VKQndGizRVo59ChHtmSMIx4 +iMfIYyNCrXt1L0fJbAcanpBq5765xd20+o++COpgMwM2xRn5vhd0qFzg/a98geVn +9TpZs2a2usDwqINw0S1W0v87zaia6ZQk/oUoYljF9Vhbd3GJAoZU9Xc/PiZH/qud +1/7tPn2v8X+Ox8KkPIMgskKQc+2hUX75lES/XeJz3/CQ5YroCiNwgAL2VYMpof3W +APJzJ2ilhVSvZf51OX1Rd4qQRwdKSU7cnudQ/qZu8QKBgQDhiJG5rXb+acD/vCMp +iw8GnxZ9N9OGRIMayzXhZ2bS1YSFV06ugqeHuKKrZlpNAztA/2pJKJ3qgVCQZmJ4 +WASEsX0nsaGISQgvbc6fyWUvVnpo/wMayi/rRZ/KPQCv5l4qO6vitffG+985nKEf +z0sK09rSBmb4d8V5D+uFk8RrIwKBgQDdlva0JT6uCtvZQRhPKw/wfqhUg5JAEMH9 +mC6ZgiVhM8JB0EjdPPB5k/hQa6Odza6mJy0oUZ0Aw1zaUenDiV4JRqwfV+qq27PA +E5xtvZyeGZuskicbqroFA85+be39V0FaLp1P3OpAxIqlSfObL+OUFn+F0lUH4Jea +/TTbXhr90QKBgAkpKPInz5uJ5CL/G1aGpXeZYqp3aAoeIk0mT+v17UFHFvjrkPCZ +sgBbSZA4uhZCuVdsiH6sPa3WztTus7U7rgNNyk2gc3U7si9rAGeRIKEJnDNDmHaw +G74st87ZJ3v9mXmRruuohIX6mRiX+ht2qg+oh0zcobYZ91VxhhmI5QONAoGBAMc8 +q4mCS39ViCMpYmAcifJlD5kdy+wKpUINCSlBWbayQSHH0xwJZPcL0qMMhUqn2zbN +1s5/wzkib2RlblhANOsGPlDYTcleTZqQh4Askpuczto1dzBrK2LC73HCCdBWGg6q +Bwv9yCqADWFcwspwHqHSMMr0OTwh9m6G6HWtgXthAoGARN+NgbT8aTMjpPGEMebi +mCsSCQJ+nGnxyLXPaBlcxe0N43MEcfOPU1g0BZkXhzu+gAkRh92zkf67jZwVYBUi +4p3lMblvbDi+/nYdKuF1XK5OhN/Y+WwqMYmdYdO0l6NEF+H5ljL60e4+bKoKhyv2 +XEbVW0ymp4YZQs8jznxgBlo= +-----END PRIVATE KEY----- diff --git a/packages/aws/data_stream/config/_dev/test/pipeline/test-common-config.yml b/packages/aws/data_stream/config/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..37e8fa225fd --- /dev/null +++ b/packages/aws/data_stream/config/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,3 @@ +fields: + tags: + - preserve_duplicate_custom_fields diff --git a/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log b/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log new file mode 100644 index 00000000000..dc768c08f85 --- /dev/null +++ b/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log @@ -0,0 +1 @@ +{"ComplianceType":"COMPLIANT","ConfigRuleInvokedTime":1742799479.852,"EvaluationResultIdentifier":{"EvaluationResultQualifier":{"ConfigRuleName":"required-tags","EvaluationMode":"DETECTIVE","ResourceId":"i-0a4468fbfafee6a8f","ResourceType":"AWS::EC2::Instance"},"OrderingTimestamp":1742541951.883,"ResourceEvaluationId":"string"},"ResultRecordedTime":1742799480.061,"Annotation":"string","ResultToken":"string","ConfigRuleInfo":{"CreatedBy":"string","Scope":{"ComplianceResourceId":"string","ComplianceResourceTypes":["string"],"TagKey":"string","TagValue":"string"},"Source":{"CustomPolicyDetails":{"EnableDebugLogDelivery":false,"PolicyRuntime":"string","PolicyText":"string"},"Owner":"AWS","SourceDetails":[{"EventSource":"string","MaximumExecutionFrequency":"string","MessageType":"string"}],"SourceIdentifier":"ACCESS_KEYS_ROTATED"},"ConfigRuleArn":"arn:aws:config:us-east-1:329599655752:config-rule/config-rule-rwpvuz","ConfigRuleId":"config-rule-rwpvuz","ConfigRuleName":"access-keys-rotated","ConfigRuleState":"ACTIVE","Description":"Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.","EvaluationModes":[{"Mode":"DETECTIVE"}],"InputParameters":"{\"maxAccessKeyAge\":\"90\"}","MaximumExecutionFrequency":"TwentyFour_Hours"}} diff --git a/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log-expected.json b/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log-expected.json new file mode 100644 index 00000000000..464fa74a936 --- /dev/null +++ b/packages/aws/data_stream/config/_dev/test/pipeline/test-event.log-expected.json @@ -0,0 +1,115 @@ +{ + "expected": [ + { + "aws": { + "config": { + "annotation": "string", + "compliance_type": "COMPLIANT", + "config_rule_invoked_time": "2025-03-24T06:57:59.852Z", + "evaluation_result_identifier": { + "evaluation_result_qualifier": { + "config_rule_name": "required-tags", + "evaluation_mode": "DETECTIVE", + "resource_id": "i-0a4468fbfafee6a8f", + "resource_type": "AWS::EC2::Instance" + }, + "ordering_timestamp": "2025-03-21T07:25:51.883Z", + "resource_evaluation_id": "string" + }, + "result_recorded_time": "2025-03-24T06:58:00.061Z", + "result_token": "string", + "rule_info": { + "config_rule_arn": "arn:aws:config:us-east-1:329599655752:config-rule/config-rule-rwpvuz", + "config_rule_id": "config-rule-rwpvuz", + "config_rule_name": "access-keys-rotated", + "config_rule_state": "ACTIVE", + "created_by": "string", + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "evaluation_modes": [ + { + "mode": "DETECTIVE" + } + ], + "input_parameters": { + "maxAccessKeyAge": "90" + }, + "maximum_execution_frequency": "TwentyFour_Hours", + "scope": { + "compliance_resource_id": "string", + "compliance_resource_types": [ + "string" + ], + "tag_key": "string", + "tag_value": "string" + }, + "source": { + "custom_policy_details": { + "enable_debug_log_delivery": false, + "policy_runtime": "string", + "policy_text": "string" + }, + "owner": "AWS", + "source_details": [ + { + "event_source": "string", + "maximum_execution_frequency": "string", + "message_type": "string" + } + ], + "source_identifier": "ACCESS_KEYS_ROTATED" + } + } + } + }, + "cloud": { + "account": { + "id": "329599655752", + "name": "329599655752" + }, + "provider": "aws" + }, + "ecs": { + "version": "8.17.0" + }, + "event": { + "category": [ + "configuration" + ], + "created": "2025-03-21T07:25:51.883Z", + "id": "string", + "kind": "event", + "original": "{\"ComplianceType\":\"COMPLIANT\",\"ConfigRuleInvokedTime\":1742799479.852,\"EvaluationResultIdentifier\":{\"EvaluationResultQualifier\":{\"ConfigRuleName\":\"required-tags\",\"EvaluationMode\":\"DETECTIVE\",\"ResourceId\":\"i-0a4468fbfafee6a8f\",\"ResourceType\":\"AWS::EC2::Instance\"},\"OrderingTimestamp\":1742541951.883,\"ResourceEvaluationId\":\"string\"},\"ResultRecordedTime\":1742799480.061,\"Annotation\":\"string\",\"ResultToken\":\"string\",\"ConfigRuleInfo\":{\"CreatedBy\":\"string\",\"Scope\":{\"ComplianceResourceId\":\"string\",\"ComplianceResourceTypes\":[\"string\"],\"TagKey\":\"string\",\"TagValue\":\"string\"},\"Source\":{\"CustomPolicyDetails\":{\"EnableDebugLogDelivery\":false,\"PolicyRuntime\":\"string\",\"PolicyText\":\"string\"},\"Owner\":\"AWS\",\"SourceDetails\":[{\"EventSource\":\"string\",\"MaximumExecutionFrequency\":\"string\",\"MessageType\":\"string\"}],\"SourceIdentifier\":\"ACCESS_KEYS_ROTATED\"},\"ConfigRuleArn\":\"arn:aws:config:us-east-1:329599655752:config-rule/config-rule-rwpvuz\",\"ConfigRuleId\":\"config-rule-rwpvuz\",\"ConfigRuleName\":\"access-keys-rotated\",\"ConfigRuleState\":\"ACTIVE\",\"Description\":\"Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.\",\"EvaluationModes\":[{\"Mode\":\"DETECTIVE\"}],\"InputParameters\":\"{\\\"maxAccessKeyAge\\\":\\\"90\\\"}\",\"MaximumExecutionFrequency\":\"TwentyFour_Hours\"}}", + "outcome": "success", + "type": [ + "info" + ] + }, + "message": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "observer": { + "vendor": "AWS Config" + }, + "related": { + "user": [ + "string" + ] + }, + "resource": { + "id": "i-0a4468fbfafee6a8f", + "type": "AWS::EC2::Instance" + }, + "result": { + "evaluation": "passed" + }, + "rule": { + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "id": "config-rule-rwpvuz", + "name": "access-keys-rotated", + "reference": "arn:aws:config:us-east-1:329599655752:config-rule/config-rule-rwpvuz", + "tags": "string" + }, + "tags": [ + "preserve_duplicate_custom_fields" + ] + } + ] +} diff --git a/packages/aws/data_stream/config/_dev/test/system/test-default-config.yml b/packages/aws/data_stream/config/_dev/test/system/test-default-config.yml new file mode 100644 index 00000000000..4fb52f60950 --- /dev/null +++ b/packages/aws/data_stream/config/_dev/test/system/test-default-config.yml @@ -0,0 +1,40 @@ +input: cel +service: config +vars: + secret_access_key: xxxx + access_key_id: xxxx +data_stream: + vars: + aws_region: xxxx + batch_size: 2 + preserve_original_event: true + preserve_duplicate_custom_fields: true + enable_request_tracer: true + ssl: | + certificate_authorities: + - | + -----BEGIN CERTIFICATE----- + MIID4TCCAsmgAwIBAgIUBdbnNWnUUMxH4YR2GEfqbZN60m8wDQYJKoZIhvcNAQEL + BQAwgYUxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQH + DA1TYW4gRnJhbmNpc2NvMRMwEQYDVQQKDApNeSBDb21wYW55MRAwDgYDVQQLDAdN + eSBVbml0MSIwIAYDVQQDDBljb25maWcueHh4eC5hbWF6b25hd3MuY29tMB4XDTI1 + MDUwNjA2Mjc0M1oXDTI2MDUwNjA2Mjc0M1owgYUxCzAJBgNVBAYTAlVTMRMwEQYD + VQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNpc2NvMRMwEQYDVQQK + DApNeSBDb21wYW55MRAwDgYDVQQLDAdNeSBVbml0MSIwIAYDVQQDDBljb25maWcu + eHh4eC5hbWF6b25hd3MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC + AQEAwzflKp5qNhAy07KZDXq0cZ0w6HEPWGuPh+1qK+ZxDqbyPBYXtNJv4XXOKmJw + nVH+XriwL9PA6T/R96zIr5GR7mT3lKa4QGdlOLAqFQjDs8HGNpePDLJImPp4Kktw + svRXrvfgsNVGy7qejT2ufK0OgszpVDSH4NaXXdwpGOuXF0e5qLox1DFiUj4N9ntA + Zqw/A9VLDvwuLveO1X4aI3a9xfTuSrLiRvED57rqW+3YJvOEru0SZn7F1pY9+V9j + kPgyTlzK6sv3xbXkt17lK4wzUvPzi0wxDRYuBNmlhZ4oq2ysMNAVbcEMzebOQ1jf + u9LTKeBn4cmltXTH9y9RfMsOkwIDAQABo0cwRTAkBgNVHREEHTAbghljb25maWcu + eHh4eC5hbWF6b25hd3MuY29tMB0GA1UdDgQWBBSc5srLUGJ3wzkdpurLxqRL8dQQ + RDANBgkqhkiG9w0BAQsFAAOCAQEAAz/TkbmDvstJg6Fc0AUWdR4YDN9N4pQXBCJ/ + C4aB+JVHoJfWD3tmXZ1y7or9/q/UXxfutUzSpXzFOq5gG3mlduaDDfgz54tr7Fzf + FjMJMjNwuIxBILi2e6uJAwxuJRn7SmMtNv46PswR8N3XvM4kyTt/11nEB1YE2yr6 + 46XFW+1db4ds8lnwmdRYM0j6gCe3jswZ6M3mhF5SNCrp+LCb70LUnsLSnh7LdPp+ + xR+OxIwWBtgT2iL5ArdWJr219Ey40G0bSVPZmtlED4Hi2oue5KIt3MnVzpxIsu8p + UrA2ofnvUjhhk6CKjFBTE7BnkH9u6NAZseQLA42vtHvgm8tu5g== + -----END CERTIFICATE----- +assert: + hit_count: 4 diff --git a/packages/aws/data_stream/config/agent/stream/cel.yml.hbs b/packages/aws/data_stream/config/agent/stream/cel.yml.hbs new file mode 100644 index 00000000000..23430dc93f1 --- /dev/null +++ b/packages/aws/data_stream/config/agent/stream/cel.yml.hbs @@ -0,0 +1,400 @@ +config_version: 2 +interval: {{interval}} +resource.tracer: + enabled: {{enable_request_tracer}} + filename: "../../logs/cel/http-request-trace-*.ndjson" + maxbackups: 5 +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: https://config.{{aws_region}}.{{tld}}/ +state: + access_key: {{access_key_id}} + secret_key: {{secret_access_key}} + session_token: {{session_token}} + aws_region: {{aws_region}} + batch_size: {{batch_size}} + tld: {{tld}} +redact: + fields: + - access_key + - secret_key + - session_token +program: | + ( + has(state.?worklist.ConfigRules) && size(state.worklist.ConfigRules) > 0 ? + { + "worklist": state.worklist, + "batch_size": state.batch_size, + "has_next": state.has_next, + "next": state.next, + "next_page": state.next_page, + + // Perform a succession of keyed hash operations (HMAC) on the request date, Region, and service, + // with the AWS secret access key as the key for the initial hashing operation. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#derive-signing-key + + "signing_key": "aws4_request".hmac("sha256", + "config".hmac("sha256", + state.aws_region.hmac("sha256", + now.format("20060102").hmac("sha256", + bytes("AWS4" + state.secret_key) + ) + ) + ) + ), + // Create a string_to_sign that includes the algorithm, request timestamp, credential scope, and hashed canonical request. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#create-string-to-sign + + "string_to_sign": [ + 'AWS4-HMAC-SHA256', + now.format("20060102T150405Z"), + now.format("20060102") + '/' + state.aws_region + '/config/aws4_request', + [ + 'POST', + '/', + '', + "content-type:application/x-amz-json-1.1", + "host:config." + state.aws_region + "." + state.tld, + "x-amz-date:" + now.format("20060102T150405Z"), + "x-amz-target:StarlingDoveService.GetComplianceDetailsByConfigRule", + '', + "content-type;host;x-amz-date;x-amz-target", + { + ?"NextToken": state.?next_page.result_token, + "ConfigRuleName": state.worklist.ConfigRules[int(state.next)].ConfigRuleName, + "Limit": int(state.batch_size) + }.encode_json().sha256().hex() + ].join("\n").sha256().hex() + ].join("\n"), + ?"session_token": state.?session_token, + "access_key": state.access_key, + "secret_key": state.secret_key, + "aws_region": state.aws_region, + "tld": state.tld, + } + : + ( + state.?want_more.orValue(false) ? + state.with({ + + // Perform a succession of keyed hash operations (HMAC) on the request date, Region, and service, + // with the AWS secret access key as the key for the initial hashing operation. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#derive-signing-key + + "signing_key": "aws4_request".hmac("sha256", + "config".hmac("sha256", + state.aws_region.hmac("sha256", + now.format("20060102").hmac("sha256", + bytes("AWS4" + state.secret_key) + ) + ) + ) + ), + // Create a string_to_sign that includes the algorithm, request timestamp, credential scope, and hashed canonical request + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#create-string-to-sign + "string_to_sign": [ + 'AWS4-HMAC-SHA256', + now.format("20060102T150405Z"), + now.format("20060102") + '/' + state.aws_region + '/config/aws4_request', + [ + 'POST', + '/', + '', + "content-type:application/x-amz-json-1.1", + "host:config." + state.aws_region + "." + state.tld, + "x-amz-date:" + now.format("20060102T150405Z"), + "x-amz-target:StarlingDoveService.DescribeConfigRules", + '', + "content-type;host;x-amz-date;x-amz-target", + { + ?"NextToken": state.?next_page.rule_token + }.encode_json().sha256().hex() + ].join("\n").sha256().hex() + ].join("\n"), + }) + : + state.with({ + + // Perform a succession of keyed hash operations (HMAC) on the request date, Region, and service, + // with the AWS secret access key as the key for the initial hashing operation. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#derive-signing-key + + "signing_key": "aws4_request".hmac("sha256", + "config".hmac("sha256", + state.aws_region.hmac("sha256", + now.format("20060102").hmac("sha256", + bytes("AWS4" + state.secret_key) + ) + ) + ) + ), + // Create a string_to_sign that includes the algorithm, request timestamp, credential scope, and hashed canonical request + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#create-string-to-sign + + "string_to_sign": [ + 'AWS4-HMAC-SHA256', + now.format("20060102T150405Z"), + now.format("20060102") + '/' + state.aws_region + '/config/aws4_request', + [ + 'POST', + '/', + '', + "content-type:application/x-amz-json-1.1", + "host:config." + state.aws_region + "." + state.tld, + "x-amz-date:" + now.format("20060102T150405Z"), + "x-amz-target:StarlingDoveService.DescribeConfigRules", + '', + "content-type;host;x-amz-date;x-amz-target", + { + ?"NextToken": state.?next_page.rule_token + }.encode_json().sha256().hex() + ].join("\n").sha256().hex() + ].join("\n") + }) + ).as(state, + post_request( + state.url.trim_right("/"), + "application/json", + { + ?"NextToken": has(state.next_page) && has(state.next_page.rule_token) ? optional.of(state.next_page.rule_token) : optional.none(), + }.encode_json() + ).with( + { + "Header": { + "Content-Type": ["application/x-amz-json-1.1"], + "X-Amz-Date": [now.format("20060102T150405Z")], + + // Perform a keyed hash operation on the string to sign using the derived signing key as the hash key to calculate the signature + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#calculate-signature + // and construct the Authorization header by combining the algorithm, credential scope, signed headers, and calculated signature. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#add-signature-to-request + + "Authorization": [ + "AWS4-HMAC-SHA256 Credential=" + + state.access_key + + "/" + now.format("20060102") + + "/" + state.aws_region + + "/config/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=" + + (state.string_to_sign).hmac("sha256", state.signing_key).hex() + ], + "X-Amz-Target": ["StarlingDoveService.DescribeConfigRules"], + ?"X-Amz-Security-Token": has(state.session_token) ? optional.of([state.session_token]) : optional.none(), + }, + } + ).do_request().as(resp, (resp.StatusCode == 200) ? + resp.Body.decode_json().as(body, + { + "worklist": body, + "next": 0, + "has_next": has(body.NextToken), + "batch_size": state.batch_size, + "next_page": { + ?"rule_token": body.?NextToken, + }, + + // Perform a succession of keyed hash operations (HMAC) on the request date, Region, and service, + // with the AWS secret access key as the key for the initial hashing operation. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#derive-signing-key + + "signing_key": "aws4_request".hmac("sha256", + "config".hmac("sha256", + state.aws_region.hmac("sha256", + now.format("20060102").hmac("sha256", + bytes("AWS4" + state.secret_key) + ) + ) + ) + ), + // Create a string_to_sign that includes the algorithm, request timestamp, credential scope, and hashed canonical request. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#create-string-to-sign + + "string_to_sign": [ + 'AWS4-HMAC-SHA256', + now.format("20060102T150405Z"), + now.format("20060102") + '/' + state.aws_region + '/config/aws4_request', + [ + 'POST', + '/', + '', + "content-type:application/x-amz-json-1.1", + "host:config." + state.aws_region + "." + state.tld, + "x-amz-date:" + now.format("20060102T150405Z"), + "x-amz-target:StarlingDoveService.GetComplianceDetailsByConfigRule", + '', + "content-type;host;x-amz-date;x-amz-target", + { + ?"NextToken": state.?next_page.result_token, + "ConfigRuleName": body.ConfigRules[0].ConfigRuleName, + "Limit": int(state.batch_size) + }.encode_json().sha256().hex() + ].join("\n").sha256().hex() + ].join("\n"), + ?"session_token": state.?session_token, + "access_key": state.access_key, + "secret_key": state.secret_key, + "aws_region": state.aws_region, + "tld": state.tld, + } + ) + : + { + "events": { + "error": { + "code": string(resp.StatusCode), + "id": string(resp.Status), + "message": "POST " + state.url.trim_right("/") + "DescribeConfigRules " + + ( + (size(resp.Body) != 0) ? + string(resp.Body) + : + string(resp.Status) + " (" + string(resp.StatusCode) + ")" + ), + }, + }, + "want_more": false, + "batch_size": state.batch_size, + ?"session_token": state.?session_token, + "access_key": state.access_key, + "secret_key": state.secret_key, + "aws_region": state.aws_region, + "tld": state.tld, + } + ) + ) + ).as(config_rules, + !has(config_rules.worklist) ? // Exit early due to POST failure. + config_rules + : has(config_rules.worklist.ConfigRules) && size(config_rules.worklist.ConfigRules) > 0 ? + post_request( + state.url.trim_right("/"), + "application/json", + { + ?"NextToken": has(config_rules.next_page) && has(config_rules.next_page.result_token) ? optional.of(config_rules.next_page.result_token) : optional.none(), + "ConfigRuleName": config_rules.worklist.ConfigRules[int(config_rules.next)].ConfigRuleName, + "Limit": int(config_rules.batch_size), + }.encode_json() + ).with( + { + "Header": { + "Content-Type": ["application/x-amz-json-1.1"], + "X-Amz-Date": [now.format("20060102T150405Z")], + + // Perform a keyed hash operation on the string to sign using the derived signing key as the hash key to calculate the signature + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#calculate-signature + // and construct the Authorization header by combining the algorithm, credential scope, signed headers, and calculated signature. + // Refer https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html#add-signature-to-request + + "Authorization": [ + "AWS4-HMAC-SHA256 Credential=" + + config_rules.access_key + + "/" + now.format("20060102") + + "/" + config_rules.aws_region + + "/config/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=" + + (config_rules.string_to_sign).hmac("sha256", config_rules.signing_key).hex() + ], + "X-Amz-Target": ["StarlingDoveService.GetComplianceDetailsByConfigRule"], + ?"X-Amz-Security-Token": has(config_rules.session_token) ? optional.of([config_rules.session_token]) : optional.none(), + }, + } + ).do_request().as(resp, (resp.StatusCode == 200) ? + bytes(resp.Body).decode_json().as(body, + { + "events": (has(body.EvaluationResults) && size(body.EvaluationResults) > 0) ? + body.EvaluationResults.orValue([]).map(evt, + { + "message": evt.with( + { + "ConfigRuleInfo": config_rules.worklist.ConfigRules[int(config_rules.next)], + } + ).encode_json(), + } + ) + : + [{}], + "next_page": { + ?"result_token": body.?NextToken, + ?"rule_token": config_rules.?next_page.rule_token, + }, + "want_more": config_rules.has_next || has(body.NextToken) || (int(config_rules.next) + 1 < size(config_rules.worklist.ConfigRules)), + "next": has(body.NextToken) ? + config_rules.next + : + (int(config_rules.next) + 1 < size(config_rules.worklist.ConfigRules)) ? + (int(config_rules.next) + 1) + : + 0, + "worklist": has(body.NextToken) ? + config_rules.worklist + : (int(config_rules.next) + 1 < size(config_rules.worklist.ConfigRules)) ? + config_rules.worklist + : + {}, + "has_next": config_rules.has_next, + "batch_size": config_rules.batch_size, + ?"session_token": config_rules.?session_token, + "access_key": config_rules.access_key, + "secret_key": config_rules.secret_key, + "aws_region": config_rules.aws_region, + "tld": config_rules.tld, + } + ) + : + { + "events": { + "error": { + "code": string(resp.StatusCode), + "id": string(resp.Status), + "message": "POST " + state.url.trim_right("/") + "GetComplianceDetailsByConfigRule " + + ( + (size(resp.Body) != 0) ? + string(resp.Body) + : + string(resp.Status) + " (" + string(resp.StatusCode) + ")" + ), + }, + }, + "want_more": false, + "batch_size": config_rules.batch_size, + ?"session_token": config_rules.?session_token, + "access_key": config_rules.access_key, + "secret_key": config_rules.secret_key, + "aws_region": config_rules.aws_region, + "tld": config_rules.tld, + } + ) + : + { + "events": [], + "want_more": false, + "batch_size": config_rules.batch_size, + ?"session_token": config_rules.?session_token, + "access_key": config_rules.access_key, + "secret_key": config_rules.secret_key, + "aws_region": config_rules.aws_region, + "tld": config_rules.tld, + } + ) +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/aws/data_stream/config/elasticsearch/ingest_pipeline/default.yml b/packages/aws/data_stream/config/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..6ccee0234bf --- /dev/null +++ b/packages/aws/data_stream/config/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,431 @@ +--- +description: Pipeline for processing aws config logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.17.0 + - rename: + field: message + tag: rename_message_to_event_original + target_field: event.original + ignore_missing: true + description: Renames the original `message` field to `event.original` to store a copy of the original message. The `event.original` field is not touched if the document already has one; it may happen when Logstash sends the document. + if: ctx.event?.original == null + - remove: + field: message + tag: remove_message + ignore_missing: true + description: The `message` field is no longer required if the document has an `event.original` field. + if: ctx.event?.original != null + - json: + field: event.original + tag: json_event_original + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - fingerprint: + fields: + - json.EvaluationResultIdentifier.EvaluationResultQualifier.ResourceId + - json.ConfigRuleInvokedTime + - json.ConfigRuleInfo.ConfigRuleId + tag: fingerprint_aws_config + target_field: _id + ignore_missing: true + - set: + field: cloud.provider + tag: set_cloud_provider + value: aws + - set: + field: observer.vendor + tag: set_observer_vendor + value: AWS Config + - append: + field: event.category + tag: append_to_event_category + value: configuration + - set: + field: event.kind + tag: set_event_kind + value: event + - append: + field: event.type + tag: append_to_event_type + value: info + - rename: + field: json.Annotation + tag: rename_Annotation + target_field: aws.config.annotation + ignore_missing: true + - dissect: + field: json.ConfigRuleInfo.ConfigRuleArn + tag: dissect_ConfigRuleInfo_ConfigRuleArn + pattern: "%{}:%{}:%{}:%{}:%{cloud.account.id}:%{}" + ignore_failure: true + - set: + field: cloud.account.name + tag: set_cloud_account_name_from_cloud_account_id + copy_from: cloud.account.id + ignore_empty_value: true + - rename: + field: json.ConfigRuleInfo.ConfigRuleArn + tag: rename_ConfigRuleInfo_ConfigRuleArn + target_field: aws.config.rule_info.config_rule_arn + ignore_missing: true + - set: + field: rule.reference + tag: set_rule_reference_from_config_config_rule_info_config_rule_arn + copy_from: aws.config.rule_info.config_rule_arn + ignore_empty_value: true + - rename: + field: json.ConfigRuleInfo.ConfigRuleId + tag: rename_ConfigRuleInfo_ConfigRuleId + target_field: aws.config.rule_info.config_rule_id + ignore_missing: true + - set: + field: rule.id + tag: set_rule_id_from_config_config_rule_info_config_rule_id + copy_from: aws.config.rule_info.config_rule_id + ignore_empty_value: true + - rename: + field: json.ConfigRuleInfo.ConfigRuleName + tag: rename_ConfigRuleInfo_ConfigRuleName + target_field: aws.config.rule_info.config_rule_name + ignore_missing: true + - set: + field: rule.name + tag: set_rule_name_from_config_config_rule_info_config_rule_name + copy_from: aws.config.rule_info.config_rule_name + ignore_empty_value: true + - rename: + field: json.ConfigRuleInfo.ConfigRuleState + tag: rename_ConfigRuleInfo_ConfigRuleState + target_field: aws.config.rule_info.config_rule_state + ignore_missing: true + - append: + field: related.user + value: '{{{json.ConfigRuleInfo.CreatedBy}}}' + allow_duplicates: false + if: ctx.json?.ConfigRuleInfo?.CreatedBy !=null + - rename: + field: json.ConfigRuleInfo.CreatedBy + tag: rename_ConfigRuleInfo_CreatedBy + target_field: aws.config.rule_info.created_by + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Description + tag: rename_ConfigRuleInfo_Description + target_field: aws.config.rule_info.description + ignore_missing: true + - set: + field: rule.description + tag: set_rule_description_from_aws_config_config_rule_info_description + copy_from: aws.config.rule_info.description + ignore_empty_value: true + - set: + field: message + tag: set_message_from_aws_config_config_rule_info_description + copy_from: aws.config.rule_info.description + ignore_empty_value: true + - foreach: + field: json.ConfigRuleInfo.EvaluationModes + if: ctx.json?.ConfigRuleInfo?.EvaluationModes instanceof List + processor: + rename: + field: _ingest._value.Mode + tag: rename_ConfigRuleInfo_EvaluationModes_Mode + target_field: _ingest._value.mode + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.EvaluationModes + tag: rename_ConfigRuleInfo_EvaluationModes + target_field: aws.config.rule_info.evaluation_modes + ignore_missing: true + - json: + field: json.ConfigRuleInfo.InputParameters + if: ctx.json?.ConfigRuleInfo?.InputParameters != null && ctx.json.ConfigRuleInfo.InputParameters != '' + tag: json_ConfigRuleInfo_InputParameters + target_field: aws.config.rule_info.input_parameters + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.ConfigRuleInfo.MaximumExecutionFrequency + tag: rename_ConfigRuleInfo_MaximumExecutionFrequency + target_field: aws.config.rule_info.maximum_execution_frequency + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Scope.ComplianceResourceId + tag: rename_ConfigRuleInfo_Scope_ComplianceResourceId + target_field: aws.config.rule_info.scope.compliance_resource_id + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Scope.ComplianceResourceTypes + tag: rename_ConfigRuleInfo_Scope_ComplianceResourceTypes + target_field: aws.config.rule_info.scope.compliance_resource_types + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Scope.TagKey + tag: rename_ConfigRuleInfo_Scope_TagKey + target_field: aws.config.rule_info.scope.tag_key + ignore_missing: true + - convert: + field: json.ConfigRuleInfo.Source.CustomPolicyDetails.EnableDebugLogDelivery + tag: convert_ConfigRuleInfo_Source_CustomPolicyDetails_EnableDebugLogDelivery_to_boolean + target_field: aws.config.rule_info.source.custom_policy_details.enable_debug_log_delivery + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.ConfigRuleInfo.Source.CustomPolicyDetails.PolicyRuntime + tag: rename_ConfigRuleInfo_Source_CustomPolicyDetails_PolicyRuntime + target_field: aws.config.rule_info.source.custom_policy_details.policy_runtime + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Source.CustomPolicyDetails.PolicyText + tag: rename_ConfigRuleInfo_Source_CustomPolicyDetails_PolicyText + target_field: aws.config.rule_info.source.custom_policy_details.policy_text + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Source.Owner + tag: rename_ConfigRuleInfo_Source_Owner + target_field: aws.config.rule_info.source.owner + ignore_missing: true + - foreach: + field: json.ConfigRuleInfo.Source.SourceDetails + if: ctx.json?.ConfigRuleInfo?.Source?.SourceDetails instanceof List + processor: + rename: + field: _ingest._value.EventSource + tag: rename_ConfigRuleInfo_Source_SourceDetails_EventSource + target_field: _ingest._value.event_source + ignore_missing: true + - foreach: + field: json.ConfigRuleInfo.Source.SourceDetails + if: ctx.json?.ConfigRuleInfo?.Source?.SourceDetails instanceof List + processor: + rename: + field: _ingest._value.MaximumExecutionFrequency + tag: rename_ConfigRuleInfo_Source_SourceDetails_MaximumExecutionFrequency + target_field: _ingest._value.maximum_execution_frequency + ignore_missing: true + - foreach: + field: json.ConfigRuleInfo.Source.SourceDetails + if: ctx.json?.ConfigRuleInfo?.Source?.SourceDetails instanceof List + processor: + rename: + field: _ingest._value.MessageType + tag: rename_ConfigRuleInfo_Source_SourceDetails_MessageType + target_field: _ingest._value.message_type + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Source.SourceDetails + tag: rename_ConfigRuleInfo_Source_SourceDetails + target_field: aws.config.rule_info.source.source_details + ignore_missing: true + - rename: + field: json.ConfigRuleInfo.Source.SourceIdentifier + tag: rename_ConfigRuleInfo_Source_SourceIdentifier + target_field: aws.config.rule_info.source.source_identifier + ignore_missing: true + - date: + field: json.ConfigRuleInvokedTime + tag: date_ConfigRuleInvokedTime + target_field: aws.config.config_rule_invoked_time + formats: + - UNIX_MS + - UNIX + if: ctx.json?.ConfigRuleInvokedTime != null && ctx.json.ConfigRuleInvokedTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.EvaluationResultIdentifier.EvaluationResultQualifier.ConfigRuleName + tag: rename_EvaluationResultIdentifier_EvaluationResultQualifier_ConfigRuleName + target_field: aws.config.evaluation_result_identifier.evaluation_result_qualifier.config_rule_name + ignore_missing: true + - rename: + field: json.EvaluationResultIdentifier.EvaluationResultQualifier.EvaluationMode + tag: rename_EvaluationResultIdentifier_EvaluationResultQualifier_EvaluationMode + target_field: aws.config.evaluation_result_identifier.evaluation_result_qualifier.evaluation_mode + ignore_missing: true + - date: + field: json.EvaluationResultIdentifier.OrderingTimestamp + tag: date_EvaluationResultIdentifier_OrderingTimestamp + target_field: aws.config.evaluation_result_identifier.ordering_timestamp + formats: + - UNIX_MS + - UNIX + if: ctx.json?.EvaluationResultIdentifier?.OrderingTimestamp != null && ctx.json.EvaluationResultIdentifier.OrderingTimestamp != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.created + tag: set_event_created_from_config_evaluation_result_identifier_ordering_timestamp + copy_from: aws.config.evaluation_result_identifier.ordering_timestamp + ignore_empty_value: true + - rename: + field: json.EvaluationResultIdentifier.ResourceEvaluationId + tag: rename_EvaluationResultIdentifier_ResourceEvaluationId + target_field: aws.config.evaluation_result_identifier.resource_evaluation_id + ignore_missing: true + - set: + field: event.id + tag: set_event_id_from_config_evaluation_result_identifier_resource_evaluation_id + copy_from: aws.config.evaluation_result_identifier.resource_evaluation_id + ignore_empty_value: true + - date: + field: json.ResultRecordedTime + tag: date_ResultRecordedTime + target_field: aws.config.result_recorded_time + formats: + - UNIX_MS + - UNIX + if: ctx.json?.ResultRecordedTime != null && ctx.json.ResultRecordedTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.ResultToken + tag: rename_ResultToken + target_field: aws.config.result_token + ignore_missing: true + - rename: + field: json.EvaluationResultIdentifier.EvaluationResultQualifier.ResourceId + tag: rename_EvaluationResultIdentifier_EvaluationResultQualifier_ResourceId + target_field: aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_id + ignore_missing: true + - set: + field: resource.id + tag: set_resource_id_from_config_evaluation_result_identifier_evaluation_result_qualifier_resource_id + copy_from: aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_id + ignore_empty_value: true + - rename: + field: json.EvaluationResultIdentifier.EvaluationResultQualifier.ResourceType + tag: rename_EvaluationResultIdentifier_EvaluationResultQualifier_ResourceType + target_field: aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_type + ignore_missing: true + - set: + field: resource.type + tag: set_resource_type_from_config_evaluation_result_identifier_evaluation_result_qualifier_resource_type + copy_from: aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_type + ignore_empty_value: true + - rename: + field: json.ComplianceType + tag: rename_ComplianceType + target_field: aws.config.compliance_type + ignore_missing: true + - script: + tag: set_event_outcome_and_result_evaluation_from_aws_config_compliance_type + lang: painless + description: set event.outcome and result.evaluation from compliance_type + if : ctx.aws?.config?.compliance_type instanceof String + source: >- + if (ctx.aws.config.compliance_type == 'NON_COMPLIANT') { + ctx.event.outcome = 'failure'; + } else if (ctx.aws.config.compliance_type == 'COMPLIANT') { + ctx.event.outcome = 'success'; + } else { + ctx.event.outcome = 'unknown'; + } + - set: + field: result.evaluation + tag: set_result_evaluation_passed + value: passed + if: ctx.aws?.config?.compliance_type == 'COMPLIANT' + ignore_empty_value: true + - set: + field: result.evaluation + tag: set_result_evaluation_failed + value: failed + if: ctx.aws?.config?.compliance_type == 'NON_COMPLIANT' + ignore_empty_value: true + - set: + field: result.evaluation + tag: set_result_evaluation_unknown + value: unknown + if: ctx.result?.evaluation == null + ignore_empty_value: true + - rename: + field: json.ConfigRuleInfo.Scope.TagValue + tag: rename_ConfigRuleInfo_Scope_TagValue + target_field: aws.config.rule_info.scope.tag_value + ignore_missing: true + - set: + field: rule.tags + tag: set_rule_tags_from_config_config_rule_info_scope_tag_value + copy_from: aws.config.rule_info.scope.tag_value + ignore_empty_value: true + - remove: + field: + - aws.config.evaluation_result_identifier.ordering_timestamp + - aws.config.evaluation_result_identifier.resource_evaluation_id + - aws.config.rule_info.config_rule_arn + - aws.config.rule_info.config_rule_id + - aws.config.rule_info.config_rule_name + - aws.config.rule_info.description + tag: remove_custom_duplicate_fields + ignore_missing: true + if: ctx.tags == null || !ctx.tags.contains('preserve_duplicate_custom_fields') + - remove: + field: json + tag: remove_json + ignore_missing: true + - script: + description: This script processor iterates over the whole document to remove fields with null values. + tag: script_to_drop_null_values + lang: painless + source: | + void handleMap(Map map) { + map.values().removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + void handleList(List list) { + list.removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + handleMap(ctx); + - set: + field: event.kind + tag: set_pipeline_error_into_event_kind + value: pipeline_error + if: ctx.error?.message != null + - append: + field: tags + value: preserve_original_event + allow_duplicates: false + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + tag: set_pipeline_error_to_event_kind + value: pipeline_error + - append: + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/aws/data_stream/config/fields/base-fields.yml b/packages/aws/data_stream/config/fields/base-fields.yml new file mode 100644 index 00000000000..eecc4a25afb --- /dev/null +++ b/packages/aws/data_stream/config/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: aws +- name: event.dataset + type: constant_keyword + value: aws.config + description: Event dataset. +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/aws/data_stream/config/fields/beats.yml b/packages/aws/data_stream/config/fields/beats.yml new file mode 100644 index 00000000000..d5fd38748ba --- /dev/null +++ b/packages/aws/data_stream/config/fields/beats.yml @@ -0,0 +1,6 @@ +- name: input.type + type: keyword + description: Type of Filebeat input. +- name: log.offset + type: long + description: Log offset. diff --git a/packages/aws/data_stream/config/fields/ecs.yml b/packages/aws/data_stream/config/fields/ecs.yml new file mode 100644 index 00000000000..2fe489ab9ed --- /dev/null +++ b/packages/aws/data_stream/config/fields/ecs.yml @@ -0,0 +1,6 @@ +- name: cloud.provider + type: constant_keyword + external: ecs +- name: observer.vendor + type: constant_keyword + external: ecs diff --git a/packages/aws/data_stream/config/fields/fields.yml b/packages/aws/data_stream/config/fields/fields.yml new file mode 100644 index 00000000000..52bc45e532a --- /dev/null +++ b/packages/aws/data_stream/config/fields/fields.yml @@ -0,0 +1,126 @@ +- name: aws + type: group + fields: + - name: config + type: group + fields: + - name: annotation + type: keyword + description: Supplementary information about how the evaluation determined the compliance. + - name: compliance_type + type: keyword + description: Indicates whether the AWS resource complies with the AWS Config rule that evaluated it. + - name: rule_info + type: group + fields: + - name: config_rule_arn + type: keyword + description: The Amazon Resource Name (ARN) of the AWS Config rule. + - name: config_rule_id + type: keyword + description: The ID of the AWS Config rule. + - name: config_rule_name + type: keyword + description: The name that you assign to the AWS Config rule. The name is required if you are adding a new rule. + - name: config_rule_state + type: keyword + description: Indicates whether the AWS Config rule is active or is currently being deleted by AWS Config. + - name: created_by + type: keyword + description: Service principal name of the service that created the rule. + - name: description + type: keyword + description: The description that you provide for the AWS Config rule. + - name: evaluation_modes + type: group + fields: + - name: mode + type: keyword + description: The mode of an evaluation. + - name: input_parameters + type: flattened + description: A string, in JSON format, that is passed to the AWS Config rule Lambda function. + - name: maximum_execution_frequency + type: keyword + description: The maximum frequency with which AWS Config runs evaluations for a rule. + - name: scope + type: group + fields: + - name: compliance_resource_id + type: keyword + description: The ID of the only AWS resource that you want to trigger an evaluation for the rule. + - name: compliance_resource_types + type: keyword + description: The resource types of only those AWS resources that you want to trigger an evaluation for the rule. + - name: tag_key + type: keyword + description: The tag key that is applied to only those AWS resources that you want to trigger an evaluation for the rule. + - name: tag_value + type: keyword + description: The tag value applied to only those AWS resources that you want to trigger an evaluation for the rule. + - name: source + type: group + fields: + - name: custom_policy_details + type: group + fields: + - name: enable_debug_log_delivery + type: boolean + description: The boolean expression for enabling debug logging for your AWS Config Custom Policy rule. + - name: policy_runtime + type: keyword + description: The runtime system for your AWS Config Custom Policy rule. + - name: policy_text + type: keyword + description: The policy definition containing the logic for your AWS Config Custom Policy rule. + - name: owner + type: keyword + description: Indicates whether AWS or the customer owns and manages the AWS Config rule. + - name: source_details + type: group + fields: + - name: event_source + type: keyword + description: The source of the event, such as an AWS service, that triggers AWS Config to evaluate your AWS resources. + - name: maximum_execution_frequency + type: keyword + description: The frequency at which you want AWS Config to run evaluations for a custom rule with a periodic trigger. + - name: message_type + type: keyword + description: The type of notification that triggers AWS Config to run an evaluation for a rule. + - name: source_identifier + type: keyword + description: For AWS Config Managed rules, a predefined identifier from a list. + - name: config_rule_invoked_time + type: date + description: The time when the AWS Config rule evaluated the AWS resource. + - name: evaluation_result_identifier + type: group + fields: + - name: evaluation_result_qualifier + type: group + fields: + - name: config_rule_name + type: keyword + description: The name of the AWS Config rule that was used in the evaluation. + - name: evaluation_mode + type: keyword + description: The mode of an evaluation. The valid values are Detective or Proactive. + - name: resource_id + type: keyword + description: The ID of the evaluated AWS resource. + - name: resource_type + type: keyword + description: The type of AWS resource that was evaluated. + - name: ordering_timestamp + type: date + description: The time of the event that triggered the evaluation of your AWS resources. + - name: resource_evaluation_id + type: keyword + description: A Unique ID for an evaluation result. + - name: result_recorded_time + type: date + description: The time when AWS Config recorded the evaluation result. + - name: result_token + type: keyword + description: An encrypted token that associates an evaluation with an AWS Config rule. diff --git a/packages/aws/data_stream/config/fields/resource.yml b/packages/aws/data_stream/config/fields/resource.yml new file mode 100644 index 00000000000..6975e84ce3b --- /dev/null +++ b/packages/aws/data_stream/config/fields/resource.yml @@ -0,0 +1,7 @@ +- name: resource + type: group + fields: + - name: id + type: keyword + - name: type + type: keyword diff --git a/packages/aws/data_stream/config/fields/result.yml b/packages/aws/data_stream/config/fields/result.yml new file mode 100644 index 00000000000..75f840ce005 --- /dev/null +++ b/packages/aws/data_stream/config/fields/result.yml @@ -0,0 +1,5 @@ +- name: result + type: group + fields: + - name: evaluation + type: keyword diff --git a/packages/aws/data_stream/config/fields/rule.yml b/packages/aws/data_stream/config/fields/rule.yml new file mode 100644 index 00000000000..b1ccff8a384 --- /dev/null +++ b/packages/aws/data_stream/config/fields/rule.yml @@ -0,0 +1,5 @@ +- name: rule + type: group + fields: + - name: tags + type: keyword diff --git a/packages/aws/data_stream/config/manifest.yml b/packages/aws/data_stream/config/manifest.yml new file mode 100644 index 00000000000..838fd107250 --- /dev/null +++ b/packages/aws/data_stream/config/manifest.yml @@ -0,0 +1,119 @@ +title: Collect AWS Config Findings logs via API +type: logs +streams: + - input: cel + title: Collect AWS Config Findings from AWS + description: Collect AWS Config Findings from AWS via API. + template_path: cel.yml.hbs + enabled: false + vars: + - name: interval + type: text + title: Interval + description: Interval to fetch AWS Config Findings from AWS. + multi: false + required: true + show_user: true + default: 24h + - name: aws_region + type: text + title: AWS Region + description: AWS Region. + required: true + - name: tld + type: text + title: Top Level Domain + multi: false + required: true + default: amazonaws.com + - name: batch_size + type: integer + title: Batch Size + description: The maximum number of evaluation results returned on each page. You cannot specify a number greater than 100. + default: 100 + multi: false + required: true + show_user: false + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + default: false + description: >- + The request tracer logs requests and responses to the agent's local file-system for debugging configurations. + Enabling this request tracing compromises security and should only be used for debugging. Disabling the request + tracer will delete any stored traces. + See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable) + for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - aws-config + - name: ssl + type: yaml + title: SSL Configuration + description: SSL configuration options. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-ssl.html#ssl-common-config) for details. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: true + title: Preserve duplicate custom fields + description: Preserve aws.config fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/aws/data_stream/config/sample_event.json b/packages/aws/data_stream/config/sample_event.json new file mode 100644 index 00000000000..99f5a87492c --- /dev/null +++ b/packages/aws/data_stream/config/sample_event.json @@ -0,0 +1,107 @@ +{ + "@timestamp": "2025-05-22T05:50:38.505Z", + "agent": { + "ephemeral_id": "0624c520-ec3a-4e52-90e4-188181f07c6c", + "id": "9764cf02-72f4-48ac-af06-79c94b0e056b", + "name": "elastic-agent-35398", + "type": "filebeat", + "version": "8.18.0" + }, + "aws": { + "config": { + "compliance_type": "COMPLIANT", + "config_rule_invoked_time": "2015-10-14T05:11:19.852Z", + "evaluation_result_identifier": { + "evaluation_result_qualifier": { + "config_rule_name": "access-keys-rotated", + "evaluation_mode": "DETECTIVE", + "resource_id": "i-0a4468fbfafeeg20h", + "resource_type": "AWS::EC2::Instance" + }, + "ordering_timestamp": "2015-09-29T15:52:31.883Z" + }, + "result_recorded_time": "2015-10-14T05:11:20.061Z", + "rule_info": { + "config_rule_arn": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1", + "config_rule_id": "config-rule-id1", + "config_rule_name": "access-keys-rotated", + "config_rule_state": "ACTIVE", + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "evaluation_modes": [ + { + "mode": "DETECTIVE" + } + ], + "input_parameters": { + "maxAccessKeyAge": "90" + }, + "maximum_execution_frequency": "TwentyFour_Hours", + "source": { + "owner": "AWS", + "source_identifier": "ACCESS_KEYS_ROTATED" + } + } + } + }, + "cloud": { + "account": { + "id": "11223344556", + "name": "11223344556" + }, + "provider": "aws" + }, + "data_stream": { + "dataset": "aws.config", + "namespace": "64685", + "type": "logs" + }, + "ecs": { + "version": "8.17.0" + }, + "elastic_agent": { + "id": "9764cf02-72f4-48ac-af06-79c94b0e056b", + "snapshot": false, + "version": "8.18.0" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "configuration" + ], + "created": "2015-09-29T15:52:31.883Z", + "dataset": "aws.config", + "ingested": "2025-05-22T05:50:41Z", + "kind": "event", + "original": "{\"ComplianceType\":\"COMPLIANT\",\"ConfigRuleInfo\":{\"ConfigRuleArn\":\"arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1\",\"ConfigRuleId\":\"config-rule-id1\",\"ConfigRuleName\":\"access-keys-rotated\",\"ConfigRuleState\":\"ACTIVE\",\"Description\":\"Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.\",\"EvaluationModes\":[{\"Mode\":\"DETECTIVE\"}],\"InputParameters\":\"{\\\"maxAccessKeyAge\\\":\\\"90\\\"}\",\"MaximumExecutionFrequency\":\"TwentyFour_Hours\",\"Source\":{\"Owner\":\"AWS\",\"SourceIdentifier\":\"ACCESS_KEYS_ROTATED\"}},\"ConfigRuleInvokedTime\":1444799479.852,\"EvaluationResultIdentifier\":{\"EvaluationResultQualifier\":{\"ConfigRuleName\":\"access-keys-rotated\",\"EvaluationMode\":\"DETECTIVE\",\"ResourceId\":\"i-0a4468fbfafeeg20h\",\"ResourceType\":\"AWS::EC2::Instance\"},\"OrderingTimestamp\":1443541951.883},\"ResultRecordedTime\":1444799480.061}", + "outcome": "success", + "type": [ + "info" + ] + }, + "input": { + "type": "cel" + }, + "message": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "observer": { + "vendor": "AWS Config" + }, + "resource": { + "id": "i-0a4468fbfafeeg20h", + "type": "AWS::EC2::Instance" + }, + "result": { + "evaluation": "passed" + }, + "rule": { + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "id": "config-rule-id1", + "name": "access-keys-rotated", + "reference": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "aws-config" + ] +} diff --git a/packages/aws/docs/config.md b/packages/aws/docs/config.md new file mode 100644 index 00000000000..5959529e77f --- /dev/null +++ b/packages/aws/docs/config.md @@ -0,0 +1,241 @@ +# Config + +[AWS Config](https://docs.aws.amazon.com/config/) provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time. + +Use this integration to collect and parse data from your AWS Config APIs. Visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference data when troubleshooting an issue. + +**IMPORTANT: Extra AWS charges on API requests will be generated by this integration. Check [API Requests](https://www.elastic.co/docs/current/integrations/aws#api-requests) for more details.** + +## Data streams + +The AWS Config integration collects one type of data: logs. + +**Logs** help you keep a record of the findings in AWS Config, allowing you to track and audit compliance status of your resources. + +The AWS Config integration works by first retrieving all config rules using the [DescribeConfigRules](https://docs.aws.amazon.com/config/latest/APIReference/API_DescribeConfigRules.html) API. Then, for each specific config rule, the integration fetches its evaluation results using the [GetComplianceDetailsByConfigRule](https://docs.aws.amazon.com/config/latest/APIReference/API_GetComplianceDetailsByConfigRule.html) API. These evaluation results enrich their respective config rules, ultimately producing a finding log. + +See more details in the [Logs reference](#logs-reference). + +## Requirements + +### Agentless Enabled Integration +Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html). +Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features. + +### Agent Based Installation +- Elastic Agent must be installed +- You can install only one Elastic Agent per host. +- Elastic Agent is required to stream data from the REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines. + +Before using any AWS integration you will need: + +* **AWS Credentials** to connect with your AWS account. +* **AWS Permissions** to make sure the user you're using to connect has permission to share the relevant data. + +For more details about these requirements, please take a look at the [AWS integration documentation](https://docs.elastic.co/integrations/aws#requirements). + +## Setup + +Use this integration if you only need to collect data from the AWS Config service. + +### To collect data from AWS Config APIs, users must have an Access Key and a Secret Key. To create API token follow below steps: + +1. Login to https://console.aws.amazon.com/. +2. Go to https://console.aws.amazon.com/iam/ to access the IAM console. +3. On the navigation menu, choose Users. +4. Choose your IAM user name. +5. Select Create access key from the Security Credentials tab. +6. To see the new access key, choose Show. + +### Enabling the integration in Elastic: + +1. In Kibana navigate to Management > Integrations. +2. In "Search for integrations" top bar, search for `AWS Config`. +3. Select the "AWS Config" integration from the search results. +4. Select "Add AWS Config" to add the integration. +5. Add all the required integration configuration parameters, including the aws_region to enable data collection. +6. Select "Save and continue" to save the integration. + +**Note** +1. For the current integration package, it is compulsory to add Secret Access Key and Access Key ID. +2. The AWS Config integration performs a full ingestion of all findings during each interval. + +## Logs reference + +### Config + +This is the `config` dataset. + +#### Example + +An example event for `config` looks as following: + +An example event for `config` looks as following: + +```json +{ + "@timestamp": "2025-05-22T05:50:38.505Z", + "agent": { + "ephemeral_id": "0624c520-ec3a-4e52-90e4-188181f07c6c", + "id": "9764cf02-72f4-48ac-af06-79c94b0e056b", + "name": "elastic-agent-35398", + "type": "filebeat", + "version": "8.18.0" + }, + "aws": { + "config": { + "compliance_type": "COMPLIANT", + "config_rule_invoked_time": "2015-10-14T05:11:19.852Z", + "evaluation_result_identifier": { + "evaluation_result_qualifier": { + "config_rule_name": "access-keys-rotated", + "evaluation_mode": "DETECTIVE", + "resource_id": "i-0a4468fbfafeeg20h", + "resource_type": "AWS::EC2::Instance" + }, + "ordering_timestamp": "2015-09-29T15:52:31.883Z" + }, + "result_recorded_time": "2015-10-14T05:11:20.061Z", + "rule_info": { + "config_rule_arn": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1", + "config_rule_id": "config-rule-id1", + "config_rule_name": "access-keys-rotated", + "config_rule_state": "ACTIVE", + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "evaluation_modes": [ + { + "mode": "DETECTIVE" + } + ], + "input_parameters": { + "maxAccessKeyAge": "90" + }, + "maximum_execution_frequency": "TwentyFour_Hours", + "source": { + "owner": "AWS", + "source_identifier": "ACCESS_KEYS_ROTATED" + } + } + } + }, + "cloud": { + "account": { + "id": "11223344556", + "name": "11223344556" + }, + "provider": "aws" + }, + "data_stream": { + "dataset": "aws.config", + "namespace": "64685", + "type": "logs" + }, + "ecs": { + "version": "8.17.0" + }, + "elastic_agent": { + "id": "9764cf02-72f4-48ac-af06-79c94b0e056b", + "snapshot": false, + "version": "8.18.0" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "configuration" + ], + "created": "2015-09-29T15:52:31.883Z", + "dataset": "aws.config", + "ingested": "2025-05-22T05:50:41Z", + "kind": "event", + "original": "{\"ComplianceType\":\"COMPLIANT\",\"ConfigRuleInfo\":{\"ConfigRuleArn\":\"arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1\",\"ConfigRuleId\":\"config-rule-id1\",\"ConfigRuleName\":\"access-keys-rotated\",\"ConfigRuleState\":\"ACTIVE\",\"Description\":\"Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.\",\"EvaluationModes\":[{\"Mode\":\"DETECTIVE\"}],\"InputParameters\":\"{\\\"maxAccessKeyAge\\\":\\\"90\\\"}\",\"MaximumExecutionFrequency\":\"TwentyFour_Hours\",\"Source\":{\"Owner\":\"AWS\",\"SourceIdentifier\":\"ACCESS_KEYS_ROTATED\"}},\"ConfigRuleInvokedTime\":1444799479.852,\"EvaluationResultIdentifier\":{\"EvaluationResultQualifier\":{\"ConfigRuleName\":\"access-keys-rotated\",\"EvaluationMode\":\"DETECTIVE\",\"ResourceId\":\"i-0a4468fbfafeeg20h\",\"ResourceType\":\"AWS::EC2::Instance\"},\"OrderingTimestamp\":1443541951.883},\"ResultRecordedTime\":1444799480.061}", + "outcome": "success", + "type": [ + "info" + ] + }, + "input": { + "type": "cel" + }, + "message": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "observer": { + "vendor": "AWS Config" + }, + "resource": { + "id": "i-0a4468fbfafeeg20h", + "type": "AWS::EC2::Instance" + }, + "result": { + "evaluation": "passed" + }, + "rule": { + "description": "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is non-compliant if the access keys have not been rotated for more than maxAccessKeyAge number of days.", + "id": "config-rule-id1", + "name": "access-keys-rotated", + "reference": "arn:aws:config:us-east-1:11223344556:config-rule/config-rule-id1" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "aws-config" + ] +} +``` + +**ECS Field Reference** + +Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields. + +#### Exported fields + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| aws.config.annotation | Supplementary information about how the evaluation determined the compliance. | keyword | +| aws.config.compliance_type | Indicates whether the AWS resource complies with the AWS Config rule that evaluated it. | keyword | +| aws.config.config_rule_invoked_time | The time when the AWS Config rule evaluated the AWS resource. | date | +| aws.config.evaluation_result_identifier.evaluation_result_qualifier.config_rule_name | The name of the AWS Config rule that was used in the evaluation. | keyword | +| aws.config.evaluation_result_identifier.evaluation_result_qualifier.evaluation_mode | The mode of an evaluation. The valid values are Detective or Proactive. | keyword | +| aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_id | The ID of the evaluated AWS resource. | keyword | +| aws.config.evaluation_result_identifier.evaluation_result_qualifier.resource_type | The type of AWS resource that was evaluated. | keyword | +| aws.config.evaluation_result_identifier.ordering_timestamp | The time of the event that triggered the evaluation of your AWS resources. | date | +| aws.config.evaluation_result_identifier.resource_evaluation_id | A Unique ID for an evaluation result. | keyword | +| aws.config.result_recorded_time | The time when AWS Config recorded the evaluation result. | date | +| aws.config.result_token | An encrypted token that associates an evaluation with an AWS Config rule. | keyword | +| aws.config.rule_info.config_rule_arn | The Amazon Resource Name (ARN) of the AWS Config rule. | keyword | +| aws.config.rule_info.config_rule_id | The ID of the AWS Config rule. | keyword | +| aws.config.rule_info.config_rule_name | The name that you assign to the AWS Config rule. The name is required if you are adding a new rule. | keyword | +| aws.config.rule_info.config_rule_state | Indicates whether the AWS Config rule is active or is currently being deleted by AWS Config. | keyword | +| aws.config.rule_info.created_by | Service principal name of the service that created the rule. | keyword | +| aws.config.rule_info.description | The description that you provide for the AWS Config rule. | keyword | +| aws.config.rule_info.evaluation_modes.mode | The mode of an evaluation. | keyword | +| aws.config.rule_info.input_parameters | A string, in JSON format, that is passed to the AWS Config rule Lambda function. | flattened | +| aws.config.rule_info.maximum_execution_frequency | The maximum frequency with which AWS Config runs evaluations for a rule. | keyword | +| aws.config.rule_info.scope.compliance_resource_id | The ID of the only AWS resource that you want to trigger an evaluation for the rule. | keyword | +| aws.config.rule_info.scope.compliance_resource_types | The resource types of only those AWS resources that you want to trigger an evaluation for the rule. | keyword | +| aws.config.rule_info.scope.tag_key | The tag key that is applied to only those AWS resources that you want to trigger an evaluation for the rule. | keyword | +| aws.config.rule_info.scope.tag_value | The tag value applied to only those AWS resources that you want to trigger an evaluation for the rule. | keyword | +| aws.config.rule_info.source.custom_policy_details.enable_debug_log_delivery | The boolean expression for enabling debug logging for your AWS Config Custom Policy rule. | boolean | +| aws.config.rule_info.source.custom_policy_details.policy_runtime | The runtime system for your AWS Config Custom Policy rule. | keyword | +| aws.config.rule_info.source.custom_policy_details.policy_text | The policy definition containing the logic for your AWS Config Custom Policy rule. | keyword | +| aws.config.rule_info.source.owner | Indicates whether AWS or the customer owns and manages the AWS Config rule. | keyword | +| aws.config.rule_info.source.source_details.event_source | The source of the event, such as an AWS service, that triggers AWS Config to evaluate your AWS resources. | keyword | +| aws.config.rule_info.source.source_details.maximum_execution_frequency | The frequency at which you want AWS Config to run evaluations for a custom rule with a periodic trigger. | keyword | +| aws.config.rule_info.source.source_details.message_type | The type of notification that triggers AWS Config to run an evaluation for a rule. | keyword | +| aws.config.rule_info.source.source_identifier | For AWS Config Managed rules, a predefined identifier from a list. | keyword | +| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | constant_keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of Filebeat input. | keyword | +| log.offset | Log offset. | long | +| observer.vendor | Vendor name of the observer. | constant_keyword | +| resource.id | | keyword | +| resource.type | | keyword | +| result.evaluation | | keyword | +| rule.tags | | keyword | + diff --git a/packages/aws/img/config-findings-overview.png b/packages/aws/img/config-findings-overview.png new file mode 100644 index 00000000000..1344548897c Binary files /dev/null and b/packages/aws/img/config-findings-overview.png differ diff --git a/packages/aws/img/logo-aws-config.svg b/packages/aws/img/logo-aws-config.svg new file mode 100644 index 00000000000..8fc602b2dc0 --- /dev/null +++ b/packages/aws/img/logo-aws-config.svg @@ -0,0 +1 @@ +AWS Config \ No newline at end of file diff --git a/packages/aws/kibana/dashboard/aws-5bbce111-827a-423f-b7b0-64ef13c28396.json b/packages/aws/kibana/dashboard/aws-5bbce111-827a-423f-b7b0-64ef13c28396.json new file mode 100644 index 00000000000..dd3491ac644 --- /dev/null +++ b/packages/aws/kibana/dashboard/aws-5bbce111-827a-423f-b7b0-64ef13c28396.json @@ -0,0 +1,1330 @@ +{ + "attributes": { + "controlGroupInput": { + "chainingSystem": "HIERARCHICAL", + "controlStyle": "oneLine", + "ignoreParentSettingsJSON": { + "ignoreFilters": false, + "ignoreQuery": false, + "ignoreTimerange": false, + "ignoreValidations": false + }, + "panelsJSON": {}, + "showApplySelections": false + }, + "description": "Overview of Config Findings.", + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "aws.config" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "aws.config" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-f815b9b1-2d25-4e21-9afa-3333704bbfbe", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "f815b9b1-2d25-4e21-9afa-3333704bbfbe": { + "columnOrder": [ + "9f032f40-4817-4416-a095-c3f65329f1d1" + ], + "columns": { + "9f032f40-4817-4416-a095-c3f65329f1d1": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Total Active Rules", + "operationType": "unique_count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "rule.uuid" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "f815b9b1-2d25-4e21-9afa-3333704bbfbe", + "layerType": "data", + "metricAccessor": "9f032f40-4817-4416-a095-c3f65329f1d1" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "hidePanelTitles": true, + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 11, + "i": "7edcdad0-e26b-4e1e-90c6-5fdc409784a9", + "w": 9, + "x": 39, + "y": 0 + }, + "panelIndex": "7edcdad0-e26b-4e1e-90c6-5fdc409784a9", + "title": "Total Active Rules [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-ada73178-6424-447d-aaf9-1b19c62a709e", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "ada73178-6424-447d-aaf9-1b19c62a709e": { + "columnOrder": [ + "e908f159-3620-40bd-a2e7-3361bada5136" + ], + "columns": { + "e908f159-3620-40bd-a2e7-3361bada5136": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Total Resources", + "operationType": "unique_count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "resource.id" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#6092C0", + "layerId": "ada73178-6424-447d-aaf9-1b19c62a709e", + "layerType": "data", + "metricAccessor": "e908f159-3620-40bd-a2e7-3361bada5136" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "hidePanelTitles": true, + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 11, + "i": "233484fe-7de1-45bc-a223-1892195615ce", + "w": 9, + "x": 30, + "y": 0 + }, + "panelIndex": "233484fe-7de1-45bc-a223-1892195615ce", + "title": "Total Resource Evalutions [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-14a5a692-9e7b-4031-a119-2609b0399f8c", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "14a5a692-9e7b-4031-a119-2609b0399f8c": { + "columnOrder": [ + "0e3264e6-57f1-43c6-b1dd-86d60065aa25" + ], + "columns": { + "0e3264e6-57f1-43c6-b1dd-86d60065aa25": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Total Non-Compliant Resources", + "operationType": "count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "result.evaluation", + "index": "logs-*", + "key": "result.evaluation", + "negate": false, + "params": { + "query": "NON_COMPLIANT" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "result.evaluation": "NON_COMPLIANT" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#CC5642", + "layerId": "14a5a692-9e7b-4031-a119-2609b0399f8c", + "layerType": "data", + "metricAccessor": "0e3264e6-57f1-43c6-b1dd-86d60065aa25" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "result.evaluation", + "index": "logs-*", + "key": "result.evaluation", + "negate": false, + "params": { + "query": "NON_COMPLIANT" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "result.evaluation": "NON_COMPLIANT" + } + } + } + ], + "hidePanelTitles": true, + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 11, + "i": "8f34e93d-7551-4fef-9b73-d95e60d60597", + "w": 9, + "x": 21, + "y": 0 + }, + "panelIndex": "8f34e93d-7551-4fef-9b73-d95e60d60597", + "title": "Total Non-Compliant Resources [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-14a5a692-9e7b-4031-a119-2609b0399f8c", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "14a5a692-9e7b-4031-a119-2609b0399f8c": { + "columnOrder": [ + "0e3264e6-57f1-43c6-b1dd-86d60065aa25" + ], + "columns": { + "0e3264e6-57f1-43c6-b1dd-86d60065aa25": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Total Compliant Resources", + "operationType": "count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "result.evaluation", + "index": "48c08233-c2d5-4383-86cc-e5ef30d79bb7", + "key": "result.evaluation", + "negate": false, + "params": { + "query": "COMPLIANT" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "result.evaluation": "COMPLIANT" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#54B399", + "layerId": "14a5a692-9e7b-4031-a119-2609b0399f8c", + "layerType": "data", + "metricAccessor": "0e3264e6-57f1-43c6-b1dd-86d60065aa25" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "result.evaluation", + "index": "48c08233-c2d5-4383-86cc-e5ef30d79bb7", + "key": "result.evaluation", + "negate": false, + "params": { + "query": "COMPLIANT" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "result.evaluation": "COMPLIANT" + } + } + } + ], + "hidePanelTitles": true, + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 11, + "i": "a0c00725-95af-46e5-ba60-8bbd5716f113", + "w": 9, + "x": 12, + "y": 0 + }, + "panelIndex": "a0c00725-95af-46e5-ba60-8bbd5716f113", + "title": "Total Compliant Resources [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-63d062c1-9ede-4d9e-b452-27827d067475", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "63d062c1-9ede-4d9e-b452-27827d067475": { + "columnOrder": [ + "b1551374-33dc-42d9-a3a1-b394458004ac", + "6bf2e99e-a24d-4fdb-8097-b30e45c23f6b" + ], + "columns": { + "6bf2e99e-a24d-4fdb-8097-b30e45c23f6b": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "b1551374-33dc-42d9-a3a1-b394458004ac": { + "customLabel": true, + "dataType": "date", + "isBucketed": true, + "label": "Config Rule Invoked Time", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "ignoreTimeRange": true, + "includeEmptyRows": true, + "interval": "d" + }, + "scale": "interval", + "sourceField": "aws.config.config_rule_invoked_time" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layers": [ + { + "accessors": [ + "6bf2e99e-a24d-4fdb-8097-b30e45c23f6b" + ], + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "eui_amsterdam_color_blind", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "63d062c1-9ede-4d9e-b452-27827d067475", + "layerType": "data", + "position": "top", + "seriesType": "line", + "showGridlines": false, + "xAccessor": "b1551374-33dc-42d9-a3a1-b394458004ac" + } + ], + "legend": { + "isVisible": true, + "position": "right", + "shouldTruncate": false + }, + "preferredSeriesType": "line", + "title": "Empty XY chart", + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 15, + "i": "7a9e042e-b539-47ec-90b2-c9bd4fed1df1", + "w": 36, + "x": 12, + "y": 11 + }, + "panelIndex": "7a9e042e-b539-47ec-90b2-c9bd4fed1df1", + "title": "Resources Evaluation over Time [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "savedVis": { + "data": { + "aggs": [], + "searchSource": { + "filter": [], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "description": "", + "params": { + "fontSize": 12, + "markdown": "**Overview**\n\nThis dashboard provides an overview of AWS Config rule evaluations, enabling security and compliance teams to monitor resource configurations across their AWS environment. It highlights the most active and non-compliant rules, helping teams quickly identify potential misconfigurations and prioritize remediation efforts.\n\nVisualizations display key metrics such as total compliant and non-compliant resources, active rules, and total resources. Additional charts show resource compliance trends over time and breakdowns by rule name and compliance type. A detailed table of essential resource information supports deeper analysis and efficient response to configuration issues.\n\n[**Integrations Page**](/app/integrations/detail/aws/overview?integration=config)\n", + "openLinksInNewTab": false + }, + "title": "", + "type": "markdown", + "uiState": {} + } + }, + "gridData": { + "h": 26, + "i": "155f07f2-6d6a-4aa1-88e3-2cd2a405b44b", + "w": 12, + "x": 0, + "y": 0 + }, + "panelIndex": "155f07f2-6d6a-4aa1-88e3-2cd2a405b44b", + "title": "Table of Contents", + "type": "visualization" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-45c5d4ba-2930-487a-8797-63c4a2f09116", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "45c5d4ba-2930-487a-8797-63c4a2f09116": { + "columnOrder": [ + "f78c6cd4-0138-4431-ba97-4fd5a95be952", + "fb04adce-06d7-424a-a5e7-328ee645288a" + ], + "columns": { + "f78c6cd4-0138-4431-ba97-4fd5a95be952": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Compliance Type", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "fb04adce-06d7-424a-a5e7-328ee645288a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 5 + }, + "scale": "ordinal", + "sourceField": "result.evaluation" + }, + "fb04adce-06d7-424a-a5e7-328ee645288a": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "colorMapping": { + "assignments": [], + "colorMode": { + "type": "categorical" + }, + "paletteId": "eui_amsterdam_color_blind", + "specialAssignments": [ + { + "color": { + "type": "loop" + }, + "rule": { + "type": "other" + }, + "touched": false + } + ] + }, + "layerId": "45c5d4ba-2930-487a-8797-63c4a2f09116", + "layerType": "data", + "legendDisplay": "show", + "metrics": [ + "fb04adce-06d7-424a-a5e7-328ee645288a" + ], + "nestedLegend": false, + "numberDisplay": "percent", + "primaryGroups": [ + "f78c6cd4-0138-4431-ba97-4fd5a95be952" + ], + "truncateLegend": false + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 16, + "i": "5f474e90-bf85-47ab-84ba-efc9c05caaf5", + "w": 15, + "x": 0, + "y": 26 + }, + "panelIndex": "5f474e90-bf85-47ab-84ba-efc9c05caaf5", + "title": "Resources by Compliance Type [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "description": "", + "enhancements": { + "dynamicActions": { + "events": [] + } + } + }, + "gridData": { + "h": 22, + "i": "beba8dd3-cc18-4c46-9317-4a89eafbc0fe", + "w": 48, + "x": 0, + "y": 42 + }, + "panelIndex": "beba8dd3-cc18-4c46-9317-4a89eafbc0fe", + "panelRefName": "panel_beba8dd3-cc18-4c46-9317-4a89eafbc0fe", + "title": "Resources Essential Details [Logs AWS]", + "type": "search" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-46acc2d5-ade1-4515-861e-819aa3613479", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "46acc2d5-ade1-4515-861e-819aa3613479": { + "columnOrder": [ + "49f0cf61-c75f-41d1-ae4f-e73199aca01c", + "69e54fc2-7d70-4741-b729-a15db6896cdb", + "be2cb017-0f36-4d04-bb43-d4efce1fcb9d" + ], + "columns": { + "49f0cf61-c75f-41d1-ae4f-e73199aca01c": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Resources ID", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "be2cb017-0f36-4d04-bb43-d4efce1fcb9d", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "resource.id" + }, + "69e54fc2-7d70-4741-b729-a15db6896cdb": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Resources Type", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "be2cb017-0f36-4d04-bb43-d4efce1fcb9d", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "resource.type" + }, + "be2cb017-0f36-4d04-bb43-d4efce1fcb9d": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Number of Rules", + "operationType": "unique_count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "rule.uuid" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "49f0cf61-c75f-41d1-ae4f-e73199aca01c", + "isMetric": false, + "isTransposed": false, + "width": 248.33333333333337 + }, + { + "columnId": "be2cb017-0f36-4d04-bb43-d4efce1fcb9d", + "isMetric": true, + "isTransposed": false + }, + { + "columnId": "69e54fc2-7d70-4741-b729-a15db6896cdb", + "isMetric": false, + "isTransposed": false + } + ], + "layerId": "46acc2d5-ade1-4515-861e-819aa3613479", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 16, + "i": "8b381dab-a710-4533-83ed-2ac992502ba9", + "w": 17, + "x": 15, + "y": 26 + }, + "panelIndex": "8b381dab-a710-4533-83ed-2ac992502ba9", + "title": "Top 10 Resources by Config Rules [Logs AWS]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-46acc2d5-ade1-4515-861e-819aa3613479", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "46acc2d5-ade1-4515-861e-819aa3613479": { + "columnOrder": [ + "49f0cf61-c75f-41d1-ae4f-e73199aca01c", + "be2cb017-0f36-4d04-bb43-d4efce1fcb9d" + ], + "columns": { + "49f0cf61-c75f-41d1-ae4f-e73199aca01c": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Rule Name", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": true, + "orderBy": { + "columnId": "be2cb017-0f36-4d04-bb43-d4efce1fcb9d", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "rule.name" + }, + "be2cb017-0f36-4d04-bb43-d4efce1fcb9d": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Number of Resources", + "operationType": "count", + "params": { + "emptyAsNull": false, + "format": { + "id": "number", + "params": { + "decimals": 0 + } + } + }, + "scale": "ratio", + "sourceField": "resource.id" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "currentIndexPatternId": "logs-*", + "layers": {} + }, + "textBased": { + "indexPatternRefs": [ + { + "id": "logs-*", + "timeField": "@timestamp", + "title": "logs-*" + } + ], + "layers": {} + } + }, + "filters": [], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "columnId": "49f0cf61-c75f-41d1-ae4f-e73199aca01c", + "isMetric": false, + "isTransposed": false + }, + { + "columnId": "be2cb017-0f36-4d04-bb43-d4efce1fcb9d", + "isMetric": true, + "isTransposed": false + } + ], + "layerId": "46acc2d5-ade1-4515-861e-819aa3613479", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": { + "dynamicActions": { + "events": [] + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "" + }, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false + }, + "gridData": { + "h": 16, + "i": "c7817d63-7ad9-470f-ace4-9b81b2e90eba", + "w": 16, + "x": 32, + "y": 26 + }, + "panelIndex": "c7817d63-7ad9-470f-ace4-9b81b2e90eba", + "title": "Top 10 Config Rules by Resources [Logs AWS]", + "type": "lens" + } + ], + "refreshInterval": { + "pause": true, + "value": 60000 + }, + "timeFrom": "now-24h", + "timeRestore": true, + "timeTo": "now", + "title": "[Logs AWS] Config", + "version": 3 + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2025-05-07T07:49:18.557Z", + "id": "aws-5bbce111-827a-423f-b7b0-64ef13c28396", + "managed": false, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + }, + { + "id": "aws-17869325-12bc-4aa7-b3c0-be47a448472f", + "name": "beba8dd3-cc18-4c46-9317-4a89eafbc0fe:panel_beba8dd3-cc18-4c46-9317-4a89eafbc0fe", + "type": "search" + }, + { + "id": "logs-*", + "name": "7edcdad0-e26b-4e1e-90c6-5fdc409784a9:indexpattern-datasource-layer-f815b9b1-2d25-4e21-9afa-3333704bbfbe", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "233484fe-7de1-45bc-a223-1892195615ce:indexpattern-datasource-layer-ada73178-6424-447d-aaf9-1b19c62a709e", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "8f34e93d-7551-4fef-9b73-d95e60d60597:indexpattern-datasource-layer-14a5a692-9e7b-4031-a119-2609b0399f8c", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "a0c00725-95af-46e5-ba60-8bbd5716f113:indexpattern-datasource-layer-14a5a692-9e7b-4031-a119-2609b0399f8c", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7a9e042e-b539-47ec-90b2-c9bd4fed1df1:indexpattern-datasource-layer-63d062c1-9ede-4d9e-b452-27827d067475", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "5f474e90-bf85-47ab-84ba-efc9c05caaf5:indexpattern-datasource-layer-45c5d4ba-2930-487a-8797-63c4a2f09116", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "8b381dab-a710-4533-83ed-2ac992502ba9:indexpattern-datasource-layer-46acc2d5-ade1-4515-861e-819aa3613479", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c7817d63-7ad9-470f-ace4-9b81b2e90eba:indexpattern-datasource-layer-46acc2d5-ade1-4515-861e-819aa3613479", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + } + ], + "type": "dashboard", + "typeMigrationVersion": "10.2.0", + "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0" +} \ No newline at end of file diff --git a/packages/aws/kibana/search/aws-17869325-12bc-4aa7-b3c0-be47a448472f.json b/packages/aws/kibana/search/aws-17869325-12bc-4aa7-b3c0-be47a448472f.json new file mode 100644 index 00000000000..271eea2a02c --- /dev/null +++ b/packages/aws/kibana/search/aws-17869325-12bc-4aa7-b3c0-be47a448472f.json @@ -0,0 +1,47 @@ +{ + "attributes": { + "columns": [ + "resource.id", + "resource.type", + "rule.name", + "rule.description", + "result.evaluation", + "cloud.account.name" + ], + "description": "", + "grid": {}, + "hideChart": false, + "isTextBasedQuery": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "" + } + } + }, + "sort": [ + [ + "@timestamp", + "desc" + ] + ], + "timeRestore": false, + "title": "Resources Essential Details [Logs AWS]" + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2025-05-07T07:00:48.466Z", + "id": "aws-17869325-12bc-4aa7-b3c0-be47a448472f", + "managed": true, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + } + ], + "type": "search", + "typeMigrationVersion": "10.5.0" +} \ No newline at end of file diff --git a/packages/aws/manifest.yml b/packages/aws/manifest.yml index ee85bd6e3b0..d884aa4bd04 100644 --- a/packages/aws/manifest.yml +++ b/packages/aws/manifest.yml @@ -1,7 +1,7 @@ format_version: 3.3.2 name: aws title: AWS -version: 3.3.3 +version: 3.4.0 description: Collect logs and metrics from Amazon Web Services (AWS) with Elastic Agent. type: integration categories: @@ -172,6 +172,36 @@ policy_templates: title: AWS CloudWatch logo size: 32x32 type: image/svg+xml + - name: config + title: AWS Config + description: Collect AWS Config Logs with Elastic Agent. + deployment_modes: + default: + enabled: true + agentless: + enabled: true + organization: security + division: engineering + team: security-service-integrations + data_streams: + - config + categories: + - security + - cloudsecurity_cdr + inputs: + - type: cel + title: Collect AWS Config logs via API + description: Collecting AWS Config logs via API. + icons: + - src: /img/logo-aws-config.svg + title: AWS Config logo + size: 33x39 + type: image/svg+xml + screenshots: + - src: /img/config-findings-overview.png + title: Config Findings Overview Dashboard + size: 600x600 + type: image/png - name: dynamodb title: Amazon DynamoDB description: Collect Amazon DynamoDB metrics with Elastic Agent