Removed deprecated effects, added RP mode applicability logic, added compliance weights, added info about effect interchangeability

Author: shanhix1

articles/governance/policy/concepts/definition-structure.md

@@ -136,9 +136,8 @@ see [Tag support for Azure resources](../../../azure-resource-manager/management
 
 The following Resource Provider modes are fully supported:
 
-- `Microsoft.Kubernetes.Data` for managing your Kubernetes clusters on or off Azure. Definitions
-  using this Resource Provider mode use effects _audit_, _deny_, and _disabled_. Use
-  of the [EnforceOPAConstraint](./effects.md#enforceopaconstraint) effect is _deprecated_.
+- `Microsoft.Kubernetes.Data` for managing your Kubernetes clusters on or off Azure, and for Azure Policy components that target [Azure Arc-enabled Kubernetes clusters](../../../aks/intro-kubernetes.md) components such as pods, containers, and ingresses. Definitions
+  using this Resource Provider mode use effects _audit_, _deny_, and _disabled_.
 - `Microsoft.KeyVault.Data` for managing vaults and certificates in
   [Azure Key Vault](../../../key-vault/general/overview.md). For more information on these policy
   definitions, see

articles/governance/policy/concepts/effects.md

@@ -23,16 +23,15 @@ These effects are currently supported in a policy definition:
 - [Manual (preview)](#manual-preview)
 - [Modify](#modify)
 
-The following effects are _deprecated_:
+## Interchanging effects
 
-- [EnforceOPAConstraint](#enforceopaconstraint)
-- [EnforceRegoPolicy](#enforceregopolicy)
+Sometimes multiple effects can be valid for a given policy definition. Parameters are often used to specify allowed effect values so that a single definition can be more versatile. However, it is important to note that not all effects are interchangeable. Resource properties and logic in the policy rule can determine whether a cerain effect is considered valid to the policy definition. For example, policy definitions with effect **AuditIfNotExists** require additional details in the policy rule which are not required for policies with effect **Audit**. The effects also behave differently. **Audit** policies will assess a resource's compliance based on its own properties, while **AuditIfNotExists** policies will assess a resource's compliance based on a child or extension resource's properties.
 
-> [!IMPORTANT]
-> In place of the **EnforceOPAConstraint** or **EnforceRegoPolicy** effects, use _audit_ and
-> _deny_ with Resource Provider mode `Microsoft.Kubernetes.Data`. The built-in policy definitions
-> have been updated. When existing policy assignments of these built-in policy definitions are
-> modified, the _effect_ parameter must be changed to a value in the updated _allowedValues_ list.
+The following is general guidance around interchangeable effects:
+- **Audit**, **Deny**, and either **Modify** or **Append** are often interchangeable.
+- **AuditIfNotExists** and **DeployIfNotExists** are often interchangeable.
+- **Manual** is not interchangeable.
+- **Disabled** is interchangeable with any effect.
 
 ## Order of evaluation
 
@@ -621,133 +620,6 @@ When **enforcementMode** is **Disabled**_**, resources are still evaluated. Logg
 logs, and the policy effect don't occur. For more information, see
 [policy assignment - enforcement mode](./assignment-structure.md#enforcement-mode).
 
-## EnforceOPAConstraint
-
-This effect is used with a policy definition _mode_ of `Microsoft.Kubernetes.Data`. It's used to
-pass Gatekeeper v3 admission control rules defined with
-[OPA Constraint Framework](https://github.com/open-policy-agent/frameworks/tree/master/constraint#opa-constraint-framework)
-to [Open Policy Agent](https://www.openpolicyagent.org/) (OPA) to Kubernetes clusters on Azure.
-
-> [!IMPORTANT]
-> The limited preview policy definitions with **EnforceOPAConstraint** effect and the related
-> **Kubernetes Service** category are _deprecated_. Instead, use the effects _audit_ and _deny_ with
-> Resource Provider mode `Microsoft.Kubernetes.Data`.
-
-### EnforceOPAConstraint evaluation
-
-The Open Policy Agent admission controller evaluates any new request on the cluster in real time.
-Every 15 minutes, a full scan of the cluster is completed and the results reported to Azure Policy.
-
-### EnforceOPAConstraint properties
-
-The **details** property of the EnforceOPAConstraint effect has the subproperties that describe the
-Gatekeeper v3 admission control rule.
-
-- **constraintTemplate** (required)
-  - The Constraint template CustomResourceDefinition (CRD) that defines new Constraints. The
-    template defines the Rego logic, the Constraint schema, and the Constraint parameters that are
-    passed via **values** from Azure Policy.
-- **constraint** (required)
-  - The CRD implementation of the Constraint template. Uses parameters passed via **values** as
-    `{{ .Values.<valuename> }}`. In the following example, these values are `{{ .Values.cpuLimit }}`
-    and `{{ .Values.memoryLimit }}`.
-- **values** (optional)
-  - Defines any parameters and values to pass to the Constraint. Each value must exist in the
-    Constraint template CRD.
-
-### EnforceOPAConstraint example
-
-Example: Gatekeeper v3 admission control rule to set container CPU and memory resource limits in
-Kubernetes.
-
-```json
-"if": {
-    "allOf": [
-        {
-            "field": "type",
-            "in": [
-                "Microsoft.ContainerService/managedClusters",
-                "AKS Engine"
-            ]
-        },
-        {
-            "field": "location",
-            "equals": "westus2"
-        }
-    ]
-},
-"then": {
-    "effect": "enforceOPAConstraint",
-    "details": {
-        "constraintTemplate": "https://raw.githubusercontent.com/Azure/azure-policy/master/built-in-references/Kubernetes/container-resource-limits/template.yaml",
-        "constraint": "https://raw.githubusercontent.com/Azure/azure-policy/master/built-in-references/Kubernetes/container-resource-limits/constraint.yaml",
-        "values": {
-            "cpuLimit": "[parameters('cpuLimit')]",
-            "memoryLimit": "[parameters('memoryLimit')]"
-        }
-    }
-}
-```
-
-## EnforceRegoPolicy
-
-This effect is used with a policy definition _mode_ of `Microsoft.ContainerService.Data`. It's used
-to pass Gatekeeper v2 admission control rules defined with
-[Rego](https://www.openpolicyagent.org/docs/latest/policy-language/#what-is-rego) to
-[Open Policy Agent](https://www.openpolicyagent.org/) (OPA) on
-[Azure Kubernetes Service](../../../aks/intro-kubernetes.md).
-
-> [!IMPORTANT]
-> The limited preview policy definitions with **EnforceRegoPolicy** effect and the related
-> **Kubernetes Service** category are _deprecated_. Instead, use the effects _audit_ and _deny_ with
-> Resource Provider mode `Microsoft.Kubernetes.Data`.
-
-### EnforceRegoPolicy evaluation
-
-The Open Policy Agent admission controller evaluates any new request on the cluster in real time.
-Every 15 minutes, a full scan of the cluster is completed and the results reported to Azure Policy.
-
-### EnforceRegoPolicy properties
-
-The **details** property of the EnforceRegoPolicy effect has the subproperties that describe the
-Gatekeeper v2 admission control rule.
-
-- **policyId** (required)
-  - A unique name passed as a parameter to the Rego admission control rule.
-- **policy** (required)
-  - Specifies the URI of the Rego admission control rule.
-- **policyParameters** (optional)
-  - Defines any parameters and values to pass to the rego policy.
-
-### EnforceRegoPolicy example
-
-Example: Gatekeeper v2 admission control rule to allow only the specified container images in AKS.
-
-```json
-"if": {
-    "allOf": [
-        {
-            "field": "type",
-            "equals": "Microsoft.ContainerService/managedClusters"
-        },
-        {
-            "field": "location",
-            "equals": "westus2"
-        }
-    ]
-},
-"then": {
-    "effect": "EnforceRegoPolicy",
-    "details": {
-        "policyId": "ContainerAllowedImages",
-        "policy": "https://raw.githubusercontent.com/Azure/azure-policy/master/built-in-references/KubernetesService/container-allowed-images/limited-preview/gatekeeperpolicy.rego",
-        "policyParameters": {
-            "allowedContainerImagesRegex": "[parameters('allowedContainerImagesRegex')]"
-        }
-    }
-}
-```
-
 ## Manual (preview)
 
 The new `manual` (preview) effect enables you to self-attest the compliance of resources or scopes. Unlike other policy definitions that actively scan for evaluation, the Manual effect allows for manual changes to the compliance state. To change the compliance of a resource or scope targeted by a manual policy, you'll need to create an [attestation](attestation-structure.md). The [best practice](attestation-structure.md#best-practices) is to design manual policies that target the scope which defines the boundary of resources whose compliance need attesting.

articles/governance/policy/concepts/policy-applicability.md

@@ -21,7 +21,9 @@ Condition(s) in the `if` block of the policy rule are evaluated for applicabilit
 > [!NOTE]
 > Applicability is different from compliance, and the logic used to determine each is different. If a resource is **applicable** that means it is relevant to the policy. If a resource is **compliant** that means it adheres to the policy. Sometimes only certain conditions from the policy rule impact applicability, while all conditions of the policy rule impact compliance state.
 
-## Applicability logic for Append/Modify/Audit/Deny/RP Mode specific effects
+## Applicability logic for resource manager modes
+
+### Append, Audit, Manual, Modify and Deny policy effects
 
 Azure Policy evaluates only `type`, `name`, and `kind` conditions in the policy rule `if` expression and treats other conditions as true (or false when negated). If the final evaluation result is true, the policy is applicable. Otherwise, it's not applicable.
 
@@ -35,9 +37,27 @@ Following are special cases to the previously described applicability logic:
 |When the `if` conditions consist of only `type` and `kind` or `type` and `name` conditions     |Only type conditions are considered when deciding applicability |
 |When any conditions (including deployment parameters) include a `location` condition     |Will not be applicable to subscriptions |
 
-## Applicability logic for AuditIfNotExists and DeployIfNotExists policy effects
+### AuditIfNotExists and DeployIfNotExists policy effects
+
+The applicability of `AuditIfNotExists` and `DeployIfNotExists` policies is based off the entire `if` condition of the policy rule. When the `if` evaluates to false, the policy is not applicable.
+
+## Applicability logic for resource provider modes
+
+### Microsoft.Kubernetes.Data
+
+The applicability of `Microsoft.Kubernetes.Data` policies is based off the entire `if` condition of the policy rule. When the `if` evaluates to false, the policy is not applicable.
+
+### Microsoft.KeyVault.Data
+
+Policies with mode `Microsoft.KeyVault.Data` are applicable if the `type` condition of the policy rule evaluates to true. The `type` refers to component type, such as:
+- Microsoft.KeyVault.Data/vaults/certificates
+- Microsoft.KeyVault.Data/vaults/keys
+- Microsoft.KeyVault.Data/vaults/secrets
+
+### Microsoft.Network.Data
 
-The applicability of AuditIfNotExists and DeployIfNotExists policies is based off the entire `if` condition of the policy rule. When the `if` evaluates to false, the policy is not applicable.
+Policies with mode `Microsoft.Network.Data` are applicable if the `type` and `name` conditions of the policy rule evaluate to true. The `type` refers to  component type:
+- Microsoft.Network/virtualNetworks
 
 ## Next steps
 

articles/governance/policy/how-to/get-compliance-data.md

@@ -238,9 +238,18 @@ For details and steps, see
 
 ## How compliance works
 
-In an assignment, a resource is **Non-compliant** if it doesn't follow policy or initiative rules
-and isn't _exempt_. The following table shows how different policy effects work with the condition
-evaluation for the resulting compliance state:
+When initiative or policy definitions are assigned and evaluated, resulting compliance states are determined based on conditions in the policy rule and resources' adherence to those requirements.
+
+Azure Policy supports the following compliance states:
+- Non-compliant
+- Compliant
+- Conflict
+- Exempted
+- Unknown (preview)
+
+### Compliant and non-compliant states
+
+In an assignment, a resource is **non-compliant** if it is applicable to the policy assignment and does not adhere to conditions in the policy rule. The following table shows how different policy effects work with the condition evaluation for the resulting compliance state:
 
 | Resource State | Effect | Policy Evaluation | Compliance State |
 | --- | --- | --- | --- |
@@ -254,6 +263,8 @@ evaluation for the resulting compliance state:
 > existence condition to be FALSE to be non-compliant. When TRUE, the IF condition triggers
 > evaluation of the existence condition for the related resources.
 
+#### Example
+
 For example, assume that you have a resource group - ContsoRG, with some storage accounts
 (highlighted in red) that are exposed to public networks.
 
@@ -270,6 +281,14 @@ audits the three non-compliant storage accounts, consequently changing their sta
    Diagram showing images for five storage accounts in the Contoso R G resource group. Storage accounts one and three now have green checkmarks beneath them, while storage accounts two, four, and five now have red warning signs beneath them.
 :::image-end:::
 
+#### Understand non-compliance
+
+When a resource is determined to be **non-compliant**, there are many possible reasons. To determine
+the reason a resource is **non-compliant** or to find the change responsible, see
+[Determine non-compliance](./determine-non-compliance.md).
+
+### Other compliance states
+
 Besides **Compliant** and **Non-compliant**, policies and resources have four other states:
 
 - **Exempt**: The resource is in scope of an assignment, but has a
@@ -280,11 +299,7 @@ Besides **Compliant** and **Non-compliant**, policies and resources have four ot
 - **Not registered**: The Azure Policy Resource Provider hasn't been registered or the account
   logged in doesn't have permission to read compliance data.
 
-Azure Policy uses the **type**, **name**, or **kind** fields in the definition to determine whether
-a resource is a match. When the resource matches, it's considered applicable and has a status of
-either **Compliant**, **Non-compliant**, or **Exempt**. If either **name** or **kind** is the only
-property in the definition, then all included and non-exempt resources are considered applicable and
-are evaluated.
+Azure Policy relies on several factors to determine whether a resource is considered [applicable](../concepts/policy-applicability.md), then to determine its compliance state.
 
 The compliance percentage is determined by dividing **Compliant**, **Exempt**, and **Unknown** resources by _total
 resources_. _Total resources_ is defined as the sum of the **Compliant**, **Non-compliant**,
@@ -300,7 +315,29 @@ The overall resource compliance is 95% (19 out of 20).
 > pages in portal are different for enabled initiatives. For more information, see
 > [Regulatory Compliance](../concepts/regulatory-compliance.md)
 
-## Portal
+### Compliance roll up
+
+There are several ways to view aggregated compliance results:
+
+| Aggregate scope | Factors determining resulting compliance state |
+| --- | --- |
+| Initiative | All policies within |
+| Initiative group or control | All policies within  |
+| Policy | All applicable resources |
+| Resource | All applicable policies |
+
+So how is the aggregate compliance state determined if multiple resources or policies have different compliance states themselves? This is done by ranking each compliance state so that one "wins" over another in this situation. The rank order is:
+1. Non-compliant
+1. Compliant
+1. Conflict
+1. Exempted
+1. Unknown (preview)
+
+This means that if there are both non-compliant and compliant states, the rolled up aggregate would be non-compliant, and so on. Let's look at an example.
+
+Assume an initiative contains 10 policies, and a resource is exempt from one policy but compliant to the remaining nine. Because a compliant state has a higher rank than an exempted state, the resource would register as compliant in the rolled-up summary of the initiative. So, a resource will only show as exempt for the entire initiative if it is exempt from, or has unknown compliance to, every other single applicable policy in that initiative. On the other extreme, if the resource is non-compliant to at least one applicable policy in the initiative, it will have an overall compliance state of non-compliant, regardless of the remaining applicable policies.
+
+## Compliance reporting through Azure Portal
 
 The Azure portal showcases a graphical experience of visualizing and understanding the state of
 compliance in your environment. On the **Policy** page, the **Overview** option provides details for
@@ -323,9 +360,6 @@ resources for the current assignment. The tab defaults to **Non-compliant**, but
 Events (append, audit, deny, deploy, modify) triggered by the request to create a resource are shown
 under the **Events** tab.
 
-> [!NOTE]
-> For an AKS Engine policy, the resource shown is the resource group.
-
 :::image type="content" source="../media/getting-compliance-data/compliance-events.png" alt-text="Screenshot of the Events tab on Compliance Details page." border="false":::
 
 <a name="component-compliance"></a>
@@ -344,13 +378,10 @@ log provides additional context and information about those events.
 
 :::image type="content" source="../media/getting-compliance-data/compliance-activitylog.png" alt-text="Screenshot of the Activity Log for Azure Policy activities and evaluations." border="false":::
 
-### Understand non-compliance
-
-When a resource is determined to be **non-compliant**, there are many possible reasons. To determine
-the reason a resource is **non-compliant** or to find the change responsible, see
-[Determine non-compliance](./determine-non-compliance.md).
+> [!NOTE]
+> Compliance results can be exported from the Portal through [Azure Resource Graph queries](../samples/resource-graph-samples.md).
 
-## Command line
+## Compliance reporting through the command line
 
 The same information available in the portal can be retrieved with the REST API (including with
 [ARMClient](https://github.com/projectkudu/ARMClient)), Azure PowerShell, and Azure CLI. For full
@@ -832,7 +863,7 @@ PS> (Get-AzADUser -ObjectId {principalOid}).DisplayName
 Trent Baker
 ```
 
-## Azure Monitor logs
+## Compliance reporting through Azure Monitor logs
 
 If you have a [Log Analytics workspace](../../../azure-monitor/logs/log-query-overview.md) with
 `AzureActivity` from the