Merge pull request #237161 from ssabat/master

Removed old  mechanism section at the bottom

Author: JamesJBarnett

articles/data-factory/author-global-parameters.md

@@ -6,7 +6,7 @@ ms.subservice: authoring
 ms.topic: conceptual
 author: nabhishek
 ms.author: abnarain
-ms.date: 09/26/2022
+ms.date: 05/05/2023
 ms.custom: devx-track-azurepowershell
 ---
 
@@ -65,62 +65,6 @@ We strongly recommend using the new mechanism of including global parameters in
 
 
 
-### Deploying using PowerShell (older mechanism)
-
-> [!NOTE]
-> This is not required if you're including global parameters using the 'Manage hub' -> 'ARM template' -> 'Include global parameters in an ARM template' since you can deploy the ARM with the ARM templates without breaking the Factory-level configurations. For backward compatability we will continue to support it. 
-
-The following steps outline how to deploy global parameters via PowerShell. This is useful when your target factory has a factory-level setting such as customer-managed key.
-
-When you publish a factory or export an ARM template with global parameters, a folder called *globalParameters* is created with a file called *your-factory-name_GlobalParameters.json*. This file is a JSON object that contains each global parameter type and value in the published factory.
-
-:::image type="content" source="media/author-global-parameters/global-parameters-adf-publish.png" alt-text="Publishing global parameters":::
-
-If you're deploying to a new environment such as TEST or PROD, it's recommended to create a copy of this global parameters file and overwrite the appropriate environment-specific values. When you republish the original global parameters file will get overwritten, but the copy for the other environment will be untouched.
-
-For example, if you have a factory named 'ADF-DEV' and a global parameter of type string named 'environment' with a value 'dev', when you publish a file named *ADF-DEV_GlobalParameters.json* will get generated. If deploying to a test factory named 'ADF_TEST', create a copy of the JSON file (for example named ADF-TEST_GlobalParameters.json) and replace the parameter values with the environment-specific values. The parameter 'environment' may have a value 'test' now. 
-
-:::image type="content" source="media/author-global-parameters/powershell-task.png" alt-text="Deploying global parameters":::
-
-Use the below PowerShell script to promote global parameters to additional environments. Add an Azure PowerShell DevOps task before your ARM Template deployment. In the DevOps task, you must specify the location of the new parameters file, the target resource group, and the target data factory.
-
-> [!NOTE]
-> To deploy global parameters using PowerShell, you must use at least version 4.4.0 of the Az module.
-
-```powershell
-param
-(
-    [parameter(Mandatory = $true)] [String] $globalParametersFilePath,
-    [parameter(Mandatory = $true)] [String] $resourceGroupName,
-    [parameter(Mandatory = $true)] [String] $dataFactoryName
-)
-
-Import-Module Az.DataFactory
-
-$newGlobalParameters = New-Object 'system.collections.generic.dictionary[string,Microsoft.Azure.Management.DataFactory.Models.GlobalParameterSpecification]'
-
-Write-Host "Getting global parameters JSON from: " $globalParametersFilePath
-$globalParametersJson = Get-Content $globalParametersFilePath
-
-Write-Host "Parsing JSON..."
-$globalParametersObject = [Newtonsoft.Json.Linq.JObject]::Parse($globalParametersJson)
-
-# $gp in $factoryFileObject.properties.globalParameters.GetEnumerator()) 
-# may  be used in case you use non-standard location for global parameters. It is not recommended. 
-foreach ($gp in $globalParametersObject.GetEnumerator()) {
-    Write-Host "Adding global parameter:" $gp.Key
-    $globalParameterValue = $gp.Value.ToObject([Microsoft.Azure.Management.DataFactory.Models.GlobalParameterSpecification])
-    $newGlobalParameters.Add($gp.Key, $globalParameterValue)
-} 
-
-$dataFactory = Get-AzDataFactoryV2 -ResourceGroupName $resourceGroupName -Name $dataFactoryName
-$dataFactory.GlobalParameters = $newGlobalParameters
-
-Write-Host "Updating" $newGlobalParameters.Count "global parameters."
-
-Set-AzDataFactoryV2 -InputObject $dataFactory -Force -PublicNetworkAccess $dataFactory.PublicNetworkAccess
-```
-
 ## Next steps
 
 * Learn about Azure Data Factory's [continuous integration and deployment process](continuous-integration-delivery-improvements.md)

articles/data-factory/concepts-integration-runtime-performance.md

@@ -8,7 +8,7 @@ ms.author: makromer
 ms.service: data-factory
 ms.subservice: data-flows
 ms.custom: synapse
-ms.date: 03/10/2023
+ms.date: 04/21/2023
 ---
 
 # Optimizing performance of the Azure Integration Runtime
@@ -55,17 +55,16 @@ Dataflow divides the data into partitions and transforms it using different proc
 
 While increasing the shuffle partitions, make sure data is spread across well. A rough number is to have approximately 1.5 GB of data per partition. If data is skewed, increasing the "Shuffle partitions" won't be helpful. For example, if you have 500 GB of data, having a value between 400 to 500 should work. Default limit for shuffle partitions is 200 that works well for approximately 300 GB of data.
 
-Here are the steps on how it's set in a custom integration runtime. You can't set it for autoresolve integration runtime.
 
 1. From ADF portal under **Manage**, select a custom integration run time and you go to edit mode.
 2. Under dataflow run time tab, go to **Compute Custom Properties** section.
-3. Select **Shuffle Partitions** under Property name, input value of your choice, like 250, 500 etc.
+3. Select **Shuffle partitions** under Property name, input value of your choice, like 250, 500 etc.
 
 You can do same by editing JSON file of runtime by adding an array with property name and value after an existing property like *cleanup* property.
 
 ## Time to live
 
-By default, every data flow activity spins up a new Spark cluster based upon the Azure IR configuration. Cold cluster start-up time takes a few minutes and data processing can't start until it is complete. If your pipelines contain multiple **sequential** data flows, you can enable a time to live (TTL) value. Specifying a time to live value keeps a cluster alive for a certain period of time after its execution completes. If a new job starts using the IR during the TTL time, it will reuse the existing cluster and start up time will greatly reduced. After the second job completes, the cluster will again stay alive for the TTL time.
+By default, every data flow activity spins up a new Spark cluster based upon the Azure IR configuration. Cold cluster start-up time takes a few minutes and data processing can't start until it is complete. If your pipelines contain multiple **sequential** data flows, you can enable a time to live (TTL) value. Specifying a time to live value keeps a cluster alive for a certain period of time after its execution completes. If a new job starts using the IR during the TTL time, it will reuse the existing cluster and start up time will be greatly reduced. After the second job completes, the cluster will again stay alive for the TTL time.
 
 However, if most of your data flows execute in parallel, it is not recommended that you enable TTL for the IR that you use for those activities. Only one job can run on a single cluster at a time. If there is an available cluster, but two data flows start, only one will use the live cluster. The second job will spin up its own isolated cluster.