Merge pull request #106361 from kromerm/dataflow-1

Added data flow details

Author: PRMerger9

articles/data-factory/concepts-data-flow-performance.md

@@ -54,6 +54,9 @@ By default, turning on debug will use the default Azure Integration runtime that
 
 ![Source Part](media/data-flow/sourcepart3.png "Source Part")
 
+> [!NOTE]
+> A good guide to help you choose number of partitions for your source is based on the number of cores that you have set for your Azure Integration Runtime and multiply that number by five. So, for example, if you are transforming a series of files in your ADLS folders and you are going to utilize a 32-core Azure IR, the number of partitions you would target is 32 x 5 = 160 partitions.
+
 ### Source batch size, input, and isolation level
 
 Under **Source Options** in the source transformation, the following settings can affect performance:
@@ -95,7 +98,7 @@ To avoid row-by-row inserts into your DW, check **Enable staging** in your Sink
 
 At each transformation, you can set the partitioning scheme you wish data factory to use in the Optimize tab. It is a good practice to first test file-based sinks keeping the default partitioning and optimizations.
 
-* For smaller files, you may find selecting *Single Partition* can sometimes work better and faster than asking Spark to partition your small files.
+* For smaller files, you may find choosing fewer partitions can sometimes work better and faster than asking Spark to partition your small files.
 * If you don't have enough information about your source data, choose *Round Robin* partitioning and set the number of partitions.
 * If your data has columns that can be good hash keys, choose *Hash partitioning*.
 

articles/data-factory/format-avro.md

@@ -8,7 +8,8 @@ ms.reviewer: craigg
 ms.service: data-factory
 ms.workload: data-services
 ms.topic: conceptual
-ms.date: 02/13/2020
+ms.date: 03/03/2020
+
 ms.author: jingwang
 
 ---
@@ -80,7 +81,11 @@ The following properties are supported in the copy activity ***\*sink\**** secti
 
 ## Data type support
 
-Avro [complex data types](https://avro.apache.org/docs/current/spec.html#schema_complex) are not supported (records, enums, arrays, maps, unions, and fixed).
+### Copy activity
+Avro [complex data types](https://avro.apache.org/docs/current/spec.html#schema_complex) are not supported (records, enums, arrays, maps, unions, and fixed) in Copy Activity.
+
+### Data flows
+When working with Avro files in data flows, you can read and write complex data types, but be sure to clear the physical schema from the dataset first. In data flows, you can set your logical projection and derive columns that are complex structures, then auto-map those fields to an Avro file.
 
 ## Next steps