Skip to content

Commit 1bb067b

Browse files
authored
Update concepts-data-flow-performance.md
1 parent 58c3e5f commit 1bb067b

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

articles/data-factory/concepts-data-flow-performance.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ ms.topic: conceptual
66
ms.author: makromer
77
ms.service: data-factory
88
ms.custom: seo-lt-2019
9-
ms.date: 12/19/2019
9+
ms.date: 01/24/2020
1010
---
1111

1212
# Mapping data flows performance and tuning guide
@@ -125,6 +125,12 @@ Setting throughput and batch properties on CosmosDB sinks only take effect durin
125125
* Throughput: Set a higher throughput setting here to allow documents to write faster to CosmosDB. Please keep in mind the higher RU costs based upon a high throughput setting.
126126
* Write Throughput Budget: Use a value which is smaller than total RUs per minute. If you have a data flow with a high number of Spark partitions, setting a a budget throughput will allow more balance across those partitions.
127127

128+
## Join performance
129+
130+
Managing the performance of joins in your data flow is a very common operation that you will perform throughout the lifecycle of your data transformations. In ADF, data flows do not require data to be sorted prior to joins as these operations are performed as hash joins in Spark. However, you can benefit from improved performance with the "Broadcast" Join optimization. This will avoid shuffles by pushing down the conents of either side of your join relationship into the Spark node. This works well for smaller tables that are used for reference lookups. Larger tables that my not fit into the node's memory are not good candidates for broadcast optimization.
131+
132+
Another Join optimization is to build your joins in such a way that it avoids Spark's tendency to implement cross joins. For example, when you include literal values in your join conditions, Spark may see that as a requirement to perform a full cartesian product first, then filter out the joined values. But if you ensure that you have column values on both sides of your join condition, you can avoid this Spark-induced cartesian product and improve the performance of your joins and data flows.
133+
128134
## Next steps
129135

130136
See other Data Flow articles related to performance:

0 commit comments

Comments
 (0)