Add new Wiz Cloud Configuration Finding Full Posture data stream and update misconfig transform to use it

packages/wiz/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "3.0.0"
+  changes:
+    - description: Add new Cloud Configiruation Finding Full Posture data stream and use it as a source for latest_cdr_misconfigurations transform. If you had Cloud Configiruation Finding enabled and relied on Findings > Misconfigurations view, you need to enable the Cloud Configiruation Finding Full Posture data stream.
+      type: breaking-change
+      link: https://github.com/elastic/integrations/pull/12961
 - version: "2.9.0"
   changes:
     - description: Enable Agentless deployment.

packages/wiz/data_stream/audit/manifest.yml

@@ -45,11 +45,7 @@ streams:
         required: false
         show_user: false
         description: >-
-          The request tracer logs requests and responses to the agent's local file-system for debugging configurations.
-          Enabling this request tracing compromises security and should only be used for debugging. Disabling the request
-          tracer will delete any stored traces.
-          See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable)
-          for details.
+          The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. Disabling the request tracer will delete any stored traces. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable) for details.
       - name: tags
         type: text
         title: Tags

packages/wiz/data_stream/cloud_configuration_finding/manifest.yml

@@ -46,11 +46,7 @@ streams:
         required: false
         show_user: false
         description: >-
-          The request tracer logs requests and responses to the agent's local file-system for debugging configurations.
-          Enabling this request tracing compromises security and should only be used for debugging. Disabling the request
-          tracer will delete any stored traces.
-          See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable)
-          for details.
+          The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. Disabling the request tracer will delete any stored traces. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-cel.html#_resource_tracer_enable) for details.
       - name: tags
         type: text
         title: Tags

packages/wiz/data_stream/cloud_configuration_finding_full_posture/_dev/test/pipeline/test-cloud-configuration-finding-full-posture.log

@@ -0,0 +1,7 @@
+{"id":"bdeba988-f41b-55e6-9b99-96b8d3dc67d4","targetExternalId":"k8s/pod/da99fd668e64c2def251b1d48b7b69ad3129638787a0f9144a993fe30fd4554f/default/cluster-autoscaler-azure-cluster-autoscaler-8bc677d64-z2qfx","targetObjectProviderUniqueId":"cd971d74-92db-495c-8244-82da9a988fd0","firstSeenAt":"2023-06-12T11:38:07.900129Z","analyzedAt":"2023-06-12T11:38:07.900129Z","severity":"LOW","result":"FAIL","status":"OPEN","remediation":"Follow the step below to ensure that each [Pod](https://kubernetes.io/docs/concepts/workloads/pods) should runs containers with allowed additional capabilities: \r\n* The following capabilities are not allowed : {{removeUnnecessaryCapabilities}} .  \r\n* `securityContext.capabilities.drop` key is set to `ALL`. \r\n","resource":{"id":"0e814bb7-29e8-5c15-be9c-8da42c67ee99","providerId":"provider-id-0e814bb7-29e8-5c15-be9c-8da42c67ee99","name":"cluster-autoscaler-azure-cluster-autoscaler-8bc677d64-z2qfx","nativeType":"Pod","type":"POD","region":null,"subscription":{"id":"a3a3cc43-1dfd-50f1-882e-692840d4a891","name":"Wiz - DEV Outpost","externalId":"cfd132be-3bc7-4f86-8efd-ed53ae498fec","cloudProvider":"Azure"},"projects":null,"tags":[{"key":"pod-template-hash","value":"8bc677d64"},{"key":"app.kubernetes.io/name","value":"azure-cluster-autoscaler"},{"key":"app.kubernetes.io/instance","value":"cluster-autoscaler"}]},"rule":{"id":"73553de7-f2ad-4ffb-b425-c69815033530","shortId":"Pod-32","graphId":"99ffeef7-75df-5c88-9265-5ab50ffbc2b9","name":"Pod should run containers with authorized additional capabilities (PSS Restricted)","description":"This rule is part of the Kubernetes [Pod Security Standards (PSS) restricted policies](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted).   \nThis rule checks whether the pod is running containers with authorized additional capabilities.     \nThis rule fails if the `securityContext.capabilities.add` contains any capability beyond `NET_BIND_SERVICE` and if `securityContext.capabilities.drop` is not set to `ALL`.  \nBy default, if the `securityContext.capabilities.add` key is not set, the pod will not run with additional capabilities, and the rule will pass.   \nLinux capabilities allow granting certain privileges to a container without granting any unnecessary ones intended for the root user.","remediationInstructions":"Follow the step below to ensure that each [Pod](https://kubernetes.io/docs/concepts/workloads/pods) should runs containers with allowed additional capabilities: \r\n* The following capabilities are not allowed : {{removeUnnecessaryCapabilities}} .  \r\n* `securityContext.capabilities.drop` key is set to `ALL`. \r\n","functionAsControl":false},"securitySubCategories":[{"id":"wsct-id-5206","title":"Container Security","category":{"id":"wct-id-423","name":"9 Container Security","framework":{"id":"wf-id-1","name":"Wiz"}}},{"id":"wsct-id-8176","title":"5.1 Containers should not run with additional capabilities","category":{"id":"wct-id-1295","name":"5 Capabilities","framework":{"id":"wf-id-57","name":"Kubernetes Pod Security Standards (Restricted)"}}},{"id":"wsct-id-8344","title":"Cluster misconfiguration","category":{"id":"wct-id-1169","name":"2 Container & Kubernetes Security","framework":{"id":"wf-id-53","name":"Wiz Detailed"}}}]}
+{"analyzedAt":"2024-08-07T12:55:52.012378Z","id":"1243196d-a365-589a-a8aa-13817c9877b2","remediation":null,"resource":{"id":"f0f4163d-cbd7-517c-ba9e-f96bb90ab5ea","name":"Root user","nativeType":"rootUser","providerId":"arn:aws:iam::998231069301:root","region":null,"cloudPlatform":"EKS","subscription":{"cloudProvider":"AWS","externalId":"998231069301","id":"94e76baa-85fd-5928-b829-1669a2ca9660","name":"wiz-integrations"},"tags":[],"type":"USER_ACCOUNT"},"result":"PASS","rule":{"description":"This rule checks if the AWS Root Account has access keys. \nThis rule fails if `AccountAccessKeysPresent` is not set to `0`. Note that it does not take into consideration the status of the keys if present. \nThe root account should avoid using access keys. Since the root account has full permissions across the entire account, creating access keys for it increases the chance that they will be compromised. Instead, it is recommended to create IAM users with predefined roles.\n>**Note** \nSee Cloud Configuration Rule `IAM-207` to see if the Root account's access keys are active.","id":"563ed717-4fb6-47fd-929e-9c794e201d0a","name":"Root account access keys should not exist","remediationInstructions":"Perform the following steps, while being signed in as the Root user, in order to delete the root user's access keys via AWS CLI: \n1. Use the following command to list the Root user's access keys. \nCopy the `AccessKeyId` from the output and paste it into the `access-key-id` value in the next step. \n```\naws iam list-access-keys\n```\n2. Use the following command to delete the access key(s). \n```\naws iam delete-access-key /\n --access-key-id <value>\n```\n>**Note** \nOnce an access key is removed, any application using it will not work until a new one is configured for it.","shortId":"IAM-006"},"severity":"MEDIUM"}
+{"analyzedAt":"2024-08-15T11:41:17.517926Z","id":"6fe49e83-2f3a-5b62-99de-beae16c7bfae","remediation":null,"resource":{"id":"8a53b2d9-f6c6-59e4-bce0-736a45e9aa3f","name":"annam-vm","nativeType":"Microsoft.Compute/virtualMachines","providerId":"80045425-a0a9-4457-82c2-2c5f47419d83","region":"eastus","subscription":{"cloudProvider":"Azure","externalId":"434f3cbb-30f2-4bc0-8bba-cb080280652b","id":"064ecbb5-19ee-540d-b9f5-99c3a4e2d0db","name":"partner integrations"},"tags":[],"type":"VIRTUAL_MACHINE"},"result":"PASS","rule":{"description":"This rule checks if the Virtual Machine (VM) is stopped (and not deallocated) for at least a week. \nThis rule fails if `code: PowerState/stopped` and the timestamp was more than 7 days ago. \nWhen you are logged in to the operating system of an Azure VM, you can issue a command to shut down the server (or via Azure CLI). This will kick you out of the OS and stop all processes but will maintain the allocated hardware (including the IP addresses currently assigned). If you find the VM in the Azure console, you will see the state listed as `Stopped`. The biggest thing you need to know about this state is that **you are still being charged by the hour for this instance**. \nFor cost optimization and management purposes, it is recommended to deallocate (charges no longer apply) VMs that have been stopped for more than a week.","id":"56c8890d-ad68-4659-9414-fb0ed7258c31","name":"Virtual Machine should not be stopped (allocated) for more than a week","remediationInstructions":"Perform the following command to deallocate the VM via Azure CLI:\n```\naz vm deallocate \\\n\t--ids {{vmId}}\n```","shortId":"VirtualMachines-021"},"severity":"LOW","evidence":{"cloudConfigurationLink":"https://learn.microsoft.com/en-us/azure/virtual-machines/states-billing","configurationPath":null,"currentValue":"The VM is stopped(allocated) since 2024-08-15","expectedValue":"The VM should be used or deallocated"}}
+{"analyzedAt":"2024-08-15T11:41:17.517926Z","id":"6fe49e83-2f3a-5b62-99de-beae16c7bfae","remediation":null,"resource":{"id":"8a53b2d9-f6c6-59e4-bce0-736a45e9aa3f","name":"annam-VM","nativeType":"Microsoft.Compute/virtualMachines","providerId":"80045425-a0a9-4457-82c2-2c5f47419d83","region":"eastus","subscription":{"cloudProvider":"Azure","externalId":"434f3cbb-30f2-4bc0-8bba-cb080280652b","id":"064ecbb5-19ee-540d-b9f5-99c3a4e2d0db","name":"partner integrations"},"tags":[],"type":"VIRTUAL_MACHINE"},"result":"IN_PROGRESS","rule":{"description":"This rule checks if the Virtual Machine (VM) is stopped (and not deallocated) for at least a week. \nThis rule fails if `code: PowerState/stopped` and the timestamp was more than 7 days ago. \nWhen you are logged in to the operating system of an Azure VM, you can issue a command to shut down the server (or via Azure CLI). This will kick you out of the OS and stop all processes but will maintain the allocated hardware (including the IP addresses currently assigned). If you find the VM in the Azure console, you will see the state listed as `Stopped`. The biggest thing you need to know about this state is that **you are still being charged by the hour for this instance**. \nFor cost optimization and management purposes, it is recommended to deallocate (charges no longer apply) VMs that have been stopped for more than a week.","id":"56c8890d-ad68-4659-9414-fb0ed7258c31","name":"Virtual Machine should not be stopped (allocated) for more than a week","remediationInstructions":"Perform the following command to deallocate the VM via Azure CLI:\n```\naz vm deallocate \\\n\t--ids {{vmId}}\n```","shortId":"VirtualMachines-021"},"severity":"LOW"}
+{"analyzedAt":"2024-08-15T11:41:17.517926Z","id":"6fe49e83-2f3a-5b62-99de-beae16c7bfae","remediation":null,"resource":{"id":"8a53b2d9-f6c6-59e4-bce0-736a45e9aa3f","name":"annam-VM","nativeType":"Microsoft.Compute/virtualMachines","providerId":"80045425-a0a9-4457-82c2-2c5f47419d83","region":"eastus","subscription":{"cloudProvider":"Azure","externalId":"434f3cbb-30f2-4bc0-8bba-cb080280652b","id":"064ecbb5-19ee-540d-b9f5-99c3a4e2d0db","name":"partner integrations"},"tags":[],"type":"VIRTUAL_MACHINE"},"result":"IN_PROGRESS","rule":{"description":"This rule checks if the Virtual Machine (VM) is stopped (and not deallocated) for at least a week. \nThis rule fails if `code: PowerState/stopped` and the timestamp was more than 7 days ago. \nWhen you are logged in to the operating system of an Azure VM, you can issue a command to shut down the server (or via Azure CLI). This will kick you out of the OS and stop all processes but will maintain the allocated hardware (including the IP addresses currently assigned). If you find the VM in the Azure console, you will see the state listed as `Stopped`. The biggest thing you need to know about this state is that **you are still being charged by the hour for this instance**. \nFor cost optimization and management purposes, it is recommended to deallocate (charges no longer apply) VMs that have been stopped for more than a week.","id":"56c8890d-ad68-4659-9414-fb0ed7258c31","name":"Virtual Machine should not be stopped (allocated) for more than a week","remediationInstructions":"Perform the following command to deallocate the VM via Azure CLI:\n```\naz vm deallocate \\\n\t--ids {{vmId}}\n```","shortId":"VirtualMachines-021"},"severity":"LOW"}
+{"analyzedAt":"2024-08-15T11:41:17.517926Z","id":"6fe49e83-2f3a-5b62-99de-beae16c7bfae-empty-provider-id","remediation":null,"resource":{"id":"8a53b2d9-f6c6-59e4-bce0-736a45e9aa3f","name":"annam-VM","nativeType":"Microsoft.Compute/virtualMachines","providerId":"","region":"eastus","subscription":{"cloudProvider":"Azure","externalId":"434f3cbb-30f2-4bc0-8bba-cb080280652b","id":"064ecbb5-19ee-540d-b9f5-99c3a4e2d0db","name":"partner integrations"},"tags":[],"type":"VIRTUAL_MACHINE"},"result":"IN_PROGRESS","rule":{"description":"This rule checks if the Virtual Machine (VM) is stopped (and not deallocated) for at least a week. \nThis rule fails if `code: PowerState/stopped` and the timestamp was more than 7 days ago. \nWhen you are logged in to the operating system of an Azure VM, you can issue a command to shut down the server (or via Azure CLI). This will kick you out of the OS and stop all processes but will maintain the allocated hardware (including the IP addresses currently assigned). If you find the VM in the Azure console, you will see the state listed as `Stopped`. The biggest thing you need to know about this state is that **you are still being charged by the hour for this instance**. \nFor cost optimization and management purposes, it is recommended to deallocate (charges no longer apply) VMs that have been stopped for more than a week.","id":"56c8890d-ad68-4659-9414-fb0ed7258c31","name":"Virtual Machine should not be stopped (allocated) for more than a week","remediationInstructions":"Perform the following command to deallocate the VM via Azure CLI:\n```\naz vm deallocate \\\n\t--ids {{vmId}}\n```","shortId":"VirtualMachines-021"},"severity":"LOW"}
+{"analyzedAt":"2024-08-15T11:41:17.517926Z","id":"6fe49e83-2f3a-5b62-99de-beae16c7bfae-missing-provider-id","remediation":null,"resource":{"id":"8a53b2d9-f6c6-59e4-bce0-736a45e9aa3f","name":"annam-VM","nativeType":"Microsoft.Compute/virtualMachines","region":"eastus","subscription":{"cloudProvider":"Azure","externalId":"434f3cbb-30f2-4bc0-8bba-cb080280652b","id":"064ecbb5-19ee-540d-b9f5-99c3a4e2d0db","name":"partner integrations"},"tags":[],"type":"VIRTUAL_MACHINE"},"result":"IN_PROGRESS","rule":{"description":"This rule checks if the Virtual Machine (VM) is stopped (and not deallocated) for at least a week. \nThis rule fails if `code: PowerState/stopped` and the timestamp was more than 7 days ago. \nWhen you are logged in to the operating system of an Azure VM, you can issue a command to shut down the server (or via Azure CLI). This will kick you out of the OS and stop all processes but will maintain the allocated hardware (including the IP addresses currently assigned). If you find the VM in the Azure console, you will see the state listed as `Stopped`. The biggest thing you need to know about this state is that **you are still being charged by the hour for this instance**. \nFor cost optimization and management purposes, it is recommended to deallocate (charges no longer apply) VMs that have been stopped for more than a week.","id":"56c8890d-ad68-4659-9414-fb0ed7258c31","name":"Virtual Machine should not be stopped (allocated) for more than a week","remediationInstructions":"Perform the following command to deallocate the VM via Azure CLI:\n```\naz vm deallocate \\\n\t--ids {{vmId}}\n```","shortId":"VirtualMachines-021"},"severity":"LOW"}