Merge pull request #253578 from KrishnakumarRukmangathan/patch-17

Update connector-google-cloud-storage.md

Author: prmerger-automator[bot]

articles/data-factory/connector-google-cloud-storage.md

@@ -24,6 +24,7 @@ This Google Cloud Storage connector is supported for the following capabilities:
 | Supported capabilities|IR |
 |---------| --------|
 |[Copy activity](copy-activity-overview.md) (source/-)|① ②|
+|[Mapping data flow](concepts-data-flow-overview.md) (source/-)|① |
 |[Lookup activity](control-flow-lookup-activity.md)|① ②|
 |[GetMetadata activity](control-flow-get-metadata-activity.md)|① ②|
 |[Delete activity](delete-activity.md)|① ②|
@@ -246,6 +247,81 @@ Assume that you have the following source folder structure and want to copy the
 | ------------------------------------------------------------ | --------------------------------------------------------- | ------------------------------------------------------------ |
 | bucket<br/>&nbsp;&nbsp;&nbsp;&nbsp;FolderA<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**File1.csv**<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;File2.json<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subfolder1<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**File3.csv**<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;File4.json<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**File5.csv**<br/>&nbsp;&nbsp;&nbsp;&nbsp;Metadata<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FileListToCopy.txt | File1.csv<br>Subfolder1/File3.csv<br>Subfolder1/File5.csv | **In dataset:**<br>- Bucket: `bucket`<br>- Folder path: `FolderA`<br><br>**In copy activity source:**<br>- File list path: `bucket/Metadata/FileListToCopy.txt` <br><br>The file list path points to a text file in the same data store that includes a list of files you want to copy, one file per line, with the relative path to the path configured in the dataset. |
 
+## Mapping data flow properties
+
+When you're transforming data in mapping data flows, you can read files from Google Cloud Storage in the following formats:
+
+- [Avro](format-avro.md#mapping-data-flow-properties)
+- [Delta](format-delta.md#mapping-data-flow-properties)
+- [CDM](format-common-data-model.md#mapping-data-flow-properties)
+- [Delimited text](format-delimited-text.md#mapping-data-flow-properties)
+- [Excel](format-excel.md#mapping-data-flow-properties)
+- [JSON](format-json.md#mapping-data-flow-properties)
+- [ORC](format-orc.md#mapping-data-flow-properties)
+- [Parquet](format-parquet.md#mapping-data-flow-properties)
+- [XML](format-xml.md#mapping-data-flow-properties)
+
+Format specific settings are located in the documentation for that format. For more information, see [Source transformation in mapping data flow](data-flow-source.md).
+
+### Source transformation
+
+In source transformation, you can read from a container, folder, or individual file in Google Cloud Storage. Use the **Source options** tab to manage how the files are read. 
+
+:::image type="content" source="media/data-flow/source-options-1.png" alt-text="Screenshot of Source options.":::
+
+**Wildcard paths:** Using a wildcard pattern will instruct the service to loop through each matching folder and file in a single source transformation. This is an effective way to process multiple files within a single flow. Add multiple wildcard matching patterns with the plus sign that appears when you hover over your existing wildcard pattern.
+
+From your source container, choose a series of files that match a pattern. Only a container can be specified in the dataset. Your wildcard path must therefore also include your folder path from the root folder.
+
+Wildcard examples:
+
+- `*` Represents any set of characters.
+- `**` Represents recursive directory nesting.
+- `?` Replaces one character.
+- `[]` Matches one or more characters in the brackets.
+
+- `/data/sales/**/*.csv` Gets all .csv files under /data/sales.
+- `/data/sales/20??/**/` Gets all files in the 20th century.
+- `/data/sales/*/*/*.csv` Gets .csv files two levels under /data/sales.
+- `/data/sales/2004/*/12/[XY]1?.csv` Gets all .csv files in December 2004 starting with X or Y prefixed by a two-digit number.
+
+**Partition root path:** If you have partitioned folders in your file source with  a `key=value` format (for example, `year=2019`), then you can assign the top level of that partition folder tree to a column name in your data flow's data stream.
+
+First, set a wildcard to include all paths that are the partitioned folders plus the leaf files that you want to read.
+
+:::image type="content" source="media/data-flow/part-file-2.png" alt-text="Screenshot of partition source file settings.":::
+
+Use the **Partition root path** setting to define what the top level of the folder structure is. When you view the contents of your data via a data preview, you'll see that the service will add the resolved partitions found in each of your folder levels.
+
+:::image type="content" source="media/data-flow/partfile1.png" alt-text="Screenshot of partition root path.":::
+
+**List of files:** This is a file set. Create a text file that includes a list of relative path files to process. Point to this text file.
+
+**Column to store file name:** Store the name of the source file in a column in your data. Enter a new column name here to store the file name string.
+
+**After completion:** Choose to do nothing with the source file after the data flow runs, delete the source file, or move the source file. The paths for the move are relative.
+
+To move source files to another location post-processing, first select "Move" for file operation. Then, set the "from" directory. If you're not using any wildcards for your path, then the "from" setting will be the same folder as your source folder.
+
+If you have a source path with wildcard, your syntax will look like this:
+
+`/data/sales/20??/**/*.csv`
+
+You can specify "from" as:
+
+`/data/sales`
+
+And you can specify "to" as:
+
+`/backup/priorSales`
+
+In this case, all files that were sourced under `/data/sales` are moved to `/backup/priorSales`.
+
+> [!NOTE]
+> File operations run only when you start the data flow from a pipeline run (a pipeline debug or execution run) that uses the Execute Data Flow activity in a pipeline. File operations *do not* run in Data Flow debug mode.
+
+**Filter by last modified:** You can filter which files you process by specifying a date range of when they were last modified. All datetimes are in UTC.
+
 ## Lookup activity properties
 
 To learn details about the properties, check [Lookup activity](control-flow-lookup-activity.md).