Skip to content

Commit 7aa1c54

Browse files
authored
Merge pull request #294625 from jonburchel/2025-02-13-adf-freshness-update
Q3 ADF Freshness updates
2 parents 981a785 + 1a35a98 commit 7aa1c54

File tree

85 files changed

+252
-253
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

85 files changed

+252
-253
lines changed

articles/data-factory/airflow-configurations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: nabhishek
55
ms.author: abnarain
66
ms.reviewer: jburchel
77
ms.topic: how-to
8-
ms.date: 12/11/2023
8+
ms.date: 02/13/2025
99
---
1010

1111
# Supported Apache Airflow configurations

articles/data-factory/airflow-get-ip-airflow-cluster.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: nabhishek
55
ms.author: abnarain
66
ms.reviewer: jburchel
77
ms.topic: how-to
8-
ms.date: 09/19/2023
8+
ms.date: 02/13/2025
99
---
1010

1111
# Retrieve the IP address of a Workflow Orchestration Manager cluster

articles/data-factory/airflow-import-dags-blob-storage.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: This article shows the steps required to import DAGs by using Azure
55
ms.topic: conceptual
66
author: nabhishek
77
ms.author: abnarain
8-
ms.date: 10/20/2023
8+
ms.date: 02/13/2025
99
---
1010

1111
# Import DAGs by using Azure Blob Storage

articles/data-factory/airflow-install-private-package.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: nabhishek
55
ms.author: abnarain
66
ms.reviewer: jburchel
77
ms.topic: how-to
8-
ms.date: 09/23/2023
8+
ms.date: 02/13/2025
99
---
1010

1111
# Install a Private package

articles/data-factory/airflow-pricing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: This article describes the pricing for Workflow Orchestration Manag
44
author: nabhishek
55
ms.subservice: pricing
66
ms.topic: conceptual
7-
ms.date: 10/20/2023
7+
ms.date: 02/13/2025
88
ms.author: abnarain
99
---
1010

articles/data-factory/airflow-sync-github-repository.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: nabhishek
55
ms.author: abnarain
66
ms.reviewer: jburchel
77
ms.topic: how-to
8-
ms.date: 09/19/2023
8+
ms.date: 02/13/2025
99
---
1010

1111
# Sync a GitHub repository in Workflow Orchestration Manager
@@ -45,7 +45,7 @@ To sync your GitHub repository by using the Workflow Orchestration Manager UI:
4545
- **GitLab**
4646
- **Bitbucket**
4747

48-
:::image type="content" source="media/airflow-git-sync-repository/git-service-type.png" alt-text="Screenshot that shows the Git service type selection dropdown in the environment setup dialog that appears during creation of an Workflow Orchestration Manager integration runtime.":::
48+
:::image type="content" source="media/airflow-git-sync-repository/git-service-type.png" alt-text="Screenshot that shows the Git service type selection dropdown in the environment setup dialog that appears during creation of a Workflow Orchestration Manager integration runtime.":::
4949

5050
1. Select a credential type:
5151

@@ -59,7 +59,7 @@ To sync your GitHub repository by using the Workflow Orchestration Manager UI:
5959
- GitLab personal access token
6060
- Bitbucket personal access token
6161

62-
:::image type="content" source="media/airflow-git-sync-repository/git-pat-credentials.png" alt-text="Screenshot that shows the Git PAT credential options in the Airflow environment setup dialog that appears during creation of an AWorkflow Orchestration Manager integration runtime.":::
62+
:::image type="content" source="media/airflow-git-sync-repository/git-pat-credentials.png" alt-text="Screenshot that shows the Git PAT credential options in the Airflow environment setup dialog that appears during creation of a Workflow Orchestration Manager integration runtime.":::
6363
- **SPN** ([service principal name](https://devblogs.microsoft.com/devops/introducing-service-principal-and-managed-identity-support-on-azure-devops/)): Only ADO supports this credential type.
6464
After you select this option, fill out the remaining fields based on the selected **Git service type**:
6565
- **Git repo url** (required): The clone URL to the Git repository to sync.
@@ -68,7 +68,7 @@ To sync your GitHub repository by using the Workflow Orchestration Manager UI:
6868
- **Service principal secret** (required): A manually generated secret in the service principal whose value is used to authenticate and access the ADO repo.
6969
- **Service principal tenant id** (required): The service principal tenant ID.
7070

71-
:::image type="content" source="media/airflow-git-sync-repository/git-spn-credentials.png" alt-text="Screenshot that shows the Git SPN credential options in the Airflow environment setup dialog that appears during creation of an Workflow Orchestration Manager integration runtime.":::
71+
:::image type="content" source="media/airflow-git-sync-repository/git-spn-credentials.png" alt-text="Screenshot that shows the Git SPN credential options in the Airflow environment setup dialog that appears during creation of a Workflow Orchestration Manager integration runtime.":::
7272

7373
1. Fill in the rest of the fields with the required information.
7474
1. Select **Create**.
@@ -295,7 +295,7 @@ This process assumes that your private package was autosynced via Git sync. You
295295

296296
For example, if your private package is in `/dags/test/private.whl` in a GitHub repo, you should add the requirement `/opt/airflow/git/\<repoName\>.git/dags/test/private.whl` in the Workflow Orchestration Manager environment.
297297

298-
:::image type="content" source="media/airflow-git-sync-repository/airflow-private-package.png" alt-text="Screenshot that shows the Airflow requirements section in the Airflow environment setup dialog that appears during creation of an Workflow Orchestration Manager integration runtime.":::
298+
:::image type="content" source="media/airflow-git-sync-repository/airflow-private-package.png" alt-text="Screenshot that shows the Airflow requirements section in the Airflow environment setup dialog that appears during creation of a Workflow Orchestration Manager integration runtime.":::
299299

300300
## Related content
301301

articles/data-factory/author-global-parameters.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,15 @@ ms.subservice: authoring
55
ms.topic: conceptual
66
author: nabhishek
77
ms.author: abnarain
8-
ms.date: 01/05/2024
8+
ms.date: 02/13/2025
99
ai-usage: ai-assisted
1010
---
1111

1212
# Global parameters in Azure Data Factory
1313

1414
[!INCLUDE[appliesto-adf-xxx-md](includes/appliesto-adf-xxx-md.md)]
1515

16-
Global parameters are constants across a data factory that can be consumed by a pipeline in any expression. They're useful when you have multiple pipelines with identical parameter names and values. When promoting a data factory using the continuous integration and deployment process (CI/CD), you can override these parameters in each environment.
16+
Global parameters are constants across a data factory that pipelines can consume in any expression. They're useful when you have multiple pipelines with identical parameter names and values. When promoting a data factory using the continuous integration and deployment process (CI/CD), you can override these parameters in each environment.
1717

1818
## Creating global parameters
1919

@@ -38,17 +38,18 @@ Global parameters can be used in any [pipeline expression](control-flow-expressi
3838

3939
## <a name="cicd"></a> Global parameters in CI/CD
4040

41-
We recommend including global parameters in the ARM template during the CI/CD. The new mechanism of including global parameters in the ARM template (from 'Manage hub' -> 'ARM template' -> ‘Include global parameters in ARM template') as illustrated below, will not conflict/ override the factory-level settings as it used to do earlier, hence not requiring additional PowerShell for global parameters deployment during CI/CD.
41+
We recommend including global parameters in the ARM template during the CI/CD. The new mechanism of including global parameters in the ARM template (from 'Manage hub' -> 'ARM template' -> ‘Include global parameters in ARM template') as illustrated in the following image, won't conflict/ override the factory-level settings as it used to do earlier, hence not requiring extra PowerShell for global parameters deployment during CI/CD.
4242

4343
:::image type="content" source="media/author-global-parameters/include-arm-template.png" alt-text="Screenshot of 'Include in ARM template'.":::
4444

4545
> [!NOTE]
46-
> We have moved the UI experience for including global parameters from the 'Global parameters' section to the 'ARM template' section in the manage hub.
47-
If you are already using the older mechanism (from 'Manage hub' -> 'Global parameters' -> 'Include in ARM template'), you can continue. We will continue to support it.
46+
> We moved the UI experience for including global parameters from the 'Global parameters' section to the 'ARM template' section in the Manage hub.
47+
48+
If you're already using the older mechanism (from 'Manage hub' -> 'Global parameters' -> 'Include in ARM template'), you can continue. We'll continue to support it.
4849

4950
The **Parameters** folder in the downloaded ARM template contains JSON files that define the parameters used in the ARM template. Each file corresponds to a specific global parameter.
5051

51-
If you are using the older flow of integrating global parameters in your continuous integration and deployment solution, it will continue to work:
52+
If you're using the older flow of integrating global parameters in your continuous integration and deployment solution, it continues to work:
5253

5354
* Include global parameters in the ARM template (from 'Manage hub' -> 'Global parameters' -> 'Include in ARM template')
5455
:::image type="content" source="media/author-global-parameters/include-arm-template-deprecated.png" alt-text="Screenshot of deprecated 'Include in ARM template'.":::
@@ -58,10 +59,10 @@ If you are using the older flow of integrating global parameters in your continu
5859
We strongly recommend using the new mechanism of including global parameters in the ARM template (from 'Manage hub' -> 'ARM template' -> 'Include global parameters in an ARM template') since it makes the CICD with global parameters much more straightforward and easier to manage.
5960

6061
> [!NOTE]
61-
> The **Include global parameters in an ARM template** configuration is only available in "Git mode". Currently it is disabled in "live mode" or "Data Factory" mode.
62+
> The **Include global parameters in an ARM template** configuration is only available in "Git mode". Currently it's disabled in "live mode" or "Data Factory" mode.
6263
6364
> [!WARNING]
64-
>You cannot use ‘-‘ in the parameter name. You will receive an errorcode "{"code":"BadRequest","message":"ErrorCode=InvalidTemplate,ErrorMessage=The expression >'pipeline().globalParameters.myparam-dbtest-url' is not valid: .....}". But, you can use the ‘_’ in the parameter name.
65+
>You can't use ‘-‘ in the parameter name. You receive an errorcode "{"code":"BadRequest","message":"ErrorCode=InvalidTemplate,ErrorMessage=The expression >'pipeline().globalParameters.myparam-dbtest-url' isn't valid: .....}". But, you can use the ‘_’ in the parameter name.
6566
6667
## Related content
6768

articles/data-factory/azure-integration-runtime-ip-addresses.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ ms.author: lle
55
author: lrtoyou1223
66
ms.subservice: integration-runtime
77
ms.topic: conceptual
8-
ms.date: 01/05/2024
8+
ms.date: 02/13/2025
99
---
1010

1111
# Azure Integration Runtime IP addresses
@@ -15,9 +15,7 @@ ms.date: 01/05/2024
1515
The IP addresses that Azure Integration Runtime uses depends on the region where your Azure integration runtime is located. *All* Azure integration runtimes that are in the same region use the same IP address ranges.
1616

1717
> [!IMPORTANT]
18-
> Azure Integration Runtime which enable Managed Virtual Network and all data flows don't support the use of fixed IP ranges.
19-
>
20-
> You can use these IP ranges for Data Movement, Pipeline and External activities executions. These IP ranges can be used for filtering in data stores/ Network Security Group (NSG)/ Firewalls for inbound access from Azure Integration runtime.
18+
> The Azure integration runtime lets you used a managed virtual network. Some data flows require the use of fixed IP ranges. You can use these IP ranges for data movement, pipeline and external activity executions, as well as for filtering in data stores, network security groups (NSGs), and firewalls for inbound access from the Azure integration runtime.
2119
2220
## Azure Integration Runtime IP addresses: Specific regions
2321

0 commit comments

Comments
 (0)