|
| 1 | +--- |
| 2 | +title: Perform secure multiparty data collaboration on Azure |
| 3 | +description: Learn how Azure Confidential Clean Rooms enables multiparty collaborations while keeping your data safe from other collaborators. |
| 4 | +author: mathapli |
| 5 | +ms.service: azure |
| 6 | +ms.subservice: confidential-computing |
| 7 | +ms.topic: conceptual |
| 8 | +ms.date: 10/28/2024 |
| 9 | +ms.author: mathapli |
| 10 | +--- |
| 11 | + |
| 12 | +# Azure Confidential Clean Rooms |
| 13 | + |
| 14 | +> [!NOTE] |
| 15 | +> Azure Confidential Clean Rooms is currently in Gated Preview. Please fill the form at https://aka.ms/ACCRPreview and we will reach out to you with next steps. |
| 16 | +
|
| 17 | +Azure Confidential Clean Rooms, aka ACCR, offers a secure and compliant environment that helps organizations overcome the challenges of using privacy-sensitive data for AI model development. This solution ensures that the model's intellectual property remains intact while also enabling advanced data analytics. |
| 18 | +Organizations can safely collaborate and analyze sensitive data without violating compliance standards or risking data breaches by using advanced privacy-enhancing technologies like secure governance & audit, secure collaboration (TEE), verifiable trust, differential privacy, and controlled access. |
| 19 | + |
| 20 | +## Who should use Azure Confidential Clean Rooms? |
| 21 | +Azure Confidential Clean Rooms could be a great choice for you if you have these scenarios: |
| 22 | + |
| 23 | +- Data analytics and inferencing: Organizations looking to build insights on second-party data while ensuring data privacy can leverage ACCR. ACCR is useful when data providers are concerned about data exfiltration. ACCR ensures that data is only used for agreed purposes and safeguards against unauthorized access or egress (as it is a sandboxed environment). |
| 24 | +- Data privacy ISVs: Independent Software Vendors (ISVs) who provide secure multiparty data collaboration services can use ACCR as an extensible platform. It allows them to add enforceable tamperproof contracts with governance and audit capabilities, and uses confidential hardware underneath to ensure data is encrypted during processing so that their customers' data remains secure. |
| 25 | +- ML fine tuning: For organizations that require data from various sources to train or fine-tune machine learning models but face data sharing regulations, ACCR provides a solution. It allows any party to audit and confirm that data is being used only for the agreed purpose, such as ML modeling |
| 26 | +- ML inferencing: For organizations that require data from various sources to train or fine-tune machine learning models but face data sharing regulations, ACCR provides a solution. It allows any party to audit and confirm that data is being used only for the agreed purpose, such as ML modeling |
| 27 | + |
| 28 | +### Industries which can successfully utilize ACCR |
| 29 | +1. Healthcare- In the healthcare industry, Azure Confidential Clean Rooms enable secure collaboration on sensitive patient data. For example, healthcare providers can use clean rooms to train and fine-tune AI/ML models for predictive diagnostics, personalized medicine, and clinical decision support. By leveraging confidential computing, healthcare organizations can protect patient privacy while collaborating with other institutions to improve healthcare outcomes. |
| 30 | +Subsequently, ACCR can also be used for ML inferencing where partner hospitals can utilize power of these models for early detection. |
| 31 | +2. Advertising- In the advertising industry, Azure Confidential Clean Rooms facilitates secure data sharing between advertisers and publishers. ACCR enables targeted advertising and campaign effectiveness measurement without exposing sensitive user data. |
| 32 | +3. Banking, Financial Services and Insurance (BFSI) - The BFSI sector can use Azure Confidential Clean Rooms to securely collaborate on financial data, ensuring compliance with regulatory requirements. This enables financial institutions to perform joint data analysis and develop risk models, fraud detection models, lending scenarios among others without exposing sensitive customer information. |
| 33 | +4. Retail- In the retail industry, Azure Confidential Clean Rooms enables secure collaboration on customer data to enhance personalized marketing and inventory management. Retailers can use clean rooms to analyze customer behavior and preferences to create personalized marketing campaigns without compromising data privacy. |
| 34 | + |
| 35 | +## Benefits |
| 36 | + |
| 37 | +:::image type="content" source="./media/confidential-clean-rooms/accr-benefits.png" alt-text="Graphic of Azure Confidential Clean Rooms benefits, showing zero trust, no data duplicationm container workloads, and managed governance."::: |
| 38 | + |
| 39 | +Azure Confidential Clean Rooms (ACCR) provides a secure and compliant environment for multi-party data collaboration. Built on confidential hardware, ACCR ensures that sensitive data remains protected throughout the collaboration process. Here are some key benefits of using Azure Confidential Clean Rooms: |
| 40 | + |
| 41 | +- Secure collaboration and governance: |
| 42 | +ACCR allows collaborators to create tamper-proof contracts. ACCR also enforces all the constraints which are part of the contract. Governance ensures validity of constraints before allowing data to be released into clean rooms and drives transparency amongst collaborators by generating tamper-proof audit trails. ACCR uses the open-sourced [confidential consortium framework](../managed-ccf/confidential-consortium-framework-overview.md) to enable these capabilities. |
| 43 | +- Compliance: |
| 44 | +Confidential computing can address some of the regulatory and privacy concerns by providing a secure environment for data collaboration. This capability is beneficial for industries such as financial services, healthcare, and telecom, which deal with highly sensitive data and personally identifiable information (PII). |
| 45 | +- Enhanced data security: |
| 46 | +ACCR is built using confidential computing to provide a hardware-based, trusted execution environment (TEE). This environment is sandboxed and allows only authorized workloads to execute and prevents unauthorized access to data or code during processing, ensuring that sensitive information remains secure. |
| 47 | +- Verifiable trust at each step with the help of cryptographic remote attestation forms the cornerstone of Azure Confidential Clean Rooms. |
| 48 | + |
| 49 | +:::image type="content" source="./media/confidential-clean-rooms/accr-illustration.png" alt-text="Graphic of Azure Confidential Clean Rooms benefits, showing all steps of clean room creation."::: |
| 50 | + |
| 51 | +- Cost-effective: |
| 52 | +By providing a secure and compliant environment for data collaboration, ACCR reduces the need for costly and complex data protection measures. This makes it a cost-effective solution for organizations looking to leverage sensitive data for analysis and insights |
| 53 | + |
| 54 | + |
| 55 | +## Onboarding to Azure Confidential Clean Rooms |
| 56 | +ACCR is currently in Gated Preview. To express your interest in joining the gated preview, please follow these steps: |
| 57 | +- Fill and submit the form at https://aka.ms/ACCR-Preview-Onboarding |
| 58 | +- Once you submit the form, we will review it and reach out to you with next steps. |
| 59 | +- Here are steps to execute some sample scenarios to deploy a clean room. Please use them for guidance. |
| 60 | + |
| 61 | +> [!Important] |
| 62 | +> The scenarios mentioned here are sample scenarios and have the goal to give you glimpse into how you can set up clean room for your own specific scenario. They are not exhaustive and are for reference purpose only. |
| 63 | +
|
| 64 | +### Prereqs |
| 65 | +#### Dev Container/GitHub Codespaces |
| 66 | +##### Dev Container/GitHub Codespaces: |
| 67 | +You can open [this repo](https://github.com/Azure/azure-cleanroom-private/blob/main/README.md#quickstart) in a dev container or a GitHub Codespace. These environments are pre-configured so the only other setup required is to add the CleanRoom Azure CLI extension using: |
| 68 | +```azurecli |
| 69 | +az extension add --source https://cleanroomazcli.blob.core.windows.net/azcli/cleanroom-0.0.6-py2.py3-none-any.whl -y --allow-preview true |
| 70 | +``` |
| 71 | +##### Local Setup: |
| 72 | +We recommend running the following steps in PowerShell on WSL using Ubuntu 22.04. |
| 73 | +- Instructions for setting up WSL can be found [here](https://learn.microsoft.com/windows/wsl/install). |
| 74 | +- To install PowerShell on WSL, follow the instructions [here](https://learn.microsoft.com/powershell/scripting/install/install-ubuntu). |
| 75 | + |
| 76 | +To set this infrastructure up, we will need the following tools to be installed prior to running the setup scripts. |
| 77 | +1. An Azure subscription with adequate permissions to create resources and manage permissions on these resources. |
| 78 | +1. Azure CLI version >= 2.57. Installation instructions [here](https://learn.microsoft.com/cli/azure/install-azure-cli-linux). |
| 79 | +1. You need Docker for Linux installed locally. Installation instructions [here](https://docs.docker.com/engine/install/#supported-platforms). |
| 80 | +1. Confidential containers Azure CLI extension, version >= 0.3.5. You can install this extension using az extension add --name confcom -y. You can check the version of the extension using az extension show --name confcom. Learn about it [here](https://learn.microsoft.com/cli/azure/confcom). |
| 81 | +1. azcopy versions >= 10.25.0. Installation instructions [here](https://learn.microsoft.com/azure/storage/common/storage-use-azcopy-v10). |
| 82 | +1. openssl - Download instructions for Linux [here](https://openssl-library.org/source/). |
| 83 | +1. jq - Download / install instructions for Linux [here](https://jqlang.github.io/jq/download/). |
| 84 | +1. Add the CleanRoom Azure CLI extension using the command here: |
| 85 | + ```azurecli |
| 86 | + az extension add --source https://cleanroomazcli.blob.core.windows.net/azcli/cleanroom-0.0.6-py2.py3-none-any.whl -y --allow-preview true |
| 87 | + ``` |
| 88 | + |
| 89 | + |
| 90 | + |
| 91 | +#### [Data Analytics with DB engine- CLI example](#tab/analyticscli) |
| 92 | +In this sample scenario, there is a data provider and data consumer. The data consumer wants to run a matching query on the data provider's data and find matches from their own data. The clean room will run inside the data consumer's tenant. |
| 93 | + |
| 94 | +##### Creating Identity |
| 95 | + |
| 96 | +1. The data consumer creates an identity public and private key pair for themselves. Each collaboration memeber is identified by a public-key certificate used for client authentication and command signing. |
| 97 | + ```azurecli |
| 98 | + # Ensure you have openssl installed before running the command below. |
| 99 | + az cleanroom governance member keygenerator-sh | bash -s -- --name "consumer" --out ./demo-resources |
| 100 | + ``` |
| 101 | + Above command shows output similar to below. |
| 102 | + ```azurecli |
| 103 | + -- Generating identity private key and certificate for participant "consumer"... |
| 104 | + Identity curve: secp384r1 |
| 105 | + Identity private key generated at: ./demo-resources/publisher_privk.pem |
| 106 | + Identity certificate generated at: ./demo-resources/publisher_cert.pem (to be registered in CCF) |
| 107 | + ``` |
| 108 | + |
| 109 | +1. The data publisher also created identity for themselves using similar steps |
| 110 | + ```azurecli |
| 111 | + # Ensure you have openssl installed before running the command below. |
| 112 | + az cleanroom governance member keygenerator-sh | bash -s -- --name "publisher" --out ./demo-resources |
| 113 | + ``` |
| 114 | + Above command shows output similar to below. |
| 115 | + ```azurecli |
| 116 | + -- Generating identity private key and certificate for participant "publisher"... |
| 117 | + Identity curve: secp384r1 |
| 118 | + Identity private key generated at: ./demo-resources/publisher_privk.pem |
| 119 | + Identity certificate generated at: ./demo-resources/publisher_cert.pem (to be registered in CCF) |
| 120 | + ``` |
| 121 | + |
| 122 | + |
| 123 | +1. The data publisher creates a collaboration called 'Analytics Collab' |
| 124 | + ```azurecli |
| 125 | + |
| 126 | + ``` |
| 127 | + |
| 128 | +1. |
| 129 | + |
| 130 | +#### [Model Inferencing- CLI example](#tab/inferencingcli) |
| 131 | + |
| 132 | + |
| 133 | + |
| 134 | +- For further questions on onboarding reach out to <[email protected]>. |
| 135 | +- After reviewing details, we will reach out to you with detailed steps for onboarding. |
| 136 | + |
| 137 | +## Frequently asked questions |
| 138 | + |
| 139 | +- Question: Where is the location Microsoft published side cars? |
| 140 | + Answer: The Microsoft published side cars are available at: https://mcr.microsoft.com/cleanroom |
| 141 | + |
| 142 | +- Question: Can more than two collaborators participate in a collaboration? |
| 143 | + Answer: Yes, more than two collaborators can become part of collaboration. This allows multiple data providers to share data in the clean room. |
| 144 | + |
| 145 | +If you have questions about Azure Confidential Clean Rooms, reach out to <[email protected]>. |
| 146 | + |
| 147 | +## Next steps |
| 148 | + |
| 149 | +- [Deploy Confidential container group with Azure Container Instances](/azure/container-instances/container-instances-tutorial-deploy-confidential-containers-cce-arm) |
| 150 | +- [Microsoft Azure Attestation](/azure/attestation/overview) |
0 commit comments