Skip to content

Commit 296c084

Browse files
authored
Merge pull request #269836 from jonburchel/2024-03-21-add-details-on-template-gallery
Add detail on selecting template from gallery
2 parents ae5d5d8 + 40f44e3 commit 296c084

File tree

3 files changed

+27
-19
lines changed

3 files changed

+27
-19
lines changed
41.4 KB
Loading
Loading

articles/data-factory/solution-template-copy-new-files-last-modified-date.md

Lines changed: 27 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -27,72 +27,80 @@ The template defines six parameters:
2727
- *Directory_Source* is the subfolder path where you can read the files from the source store. You need to replace the default value with your own subfolder path.
2828
- *FolderPath_Destination* is the folder path where you want to copy files to the destination store. You need to replace the default value with your own folder path.
2929
- *Directory_Destination* is the subfolder path where you want to copy files to the destination store. You need to replace the default value with your own subfolder path.
30-
- *LastModified_From* is used to select the files whose LastModifiedDate attribute is after or equal to this datetime value. In order to select the new files only, which has not been copied last time, this datetime value can be the time when the pipeline was triggered last time. You can replace the default value '2019-02-01T00:00:00Z' to your expected LastModifiedDate in UTC timezone.
31-
- *LastModified_To* is used to select the files whose LastModifiedDate attribute is before this datetime value. In order to select the new files only, which has not been copied last time, this datetime value can be the present time. You can replace the default value '2019-02-01T00:00:00Z' to your expected LastModifiedDate in UTC timezone.
30+
- *LastModified_From* is used to select the files whose LastModifiedDate attribute is after or equal to this datetime value. In order to select the new files only, which hasn't been copied last time, this datetime value can be the time when the pipeline was triggered last time. You can replace the default value '2019-02-01T00:00:00Z' to your expected LastModifiedDate in UTC timezone.
31+
- *LastModified_To* is used to select the files whose LastModifiedDate attribute is before this datetime value. In order to select the new files only, which weren't copied in prior runs, this datetime value can be the present time. You can replace the default value '2019-02-01T00:00:00Z' to your expected LastModifiedDate in UTC timezone.
3232

3333
## How to use this solution template
3434

35-
1. Go to template **Copy new files only by LastModifiedDate**. Create a **New** connection to your destination store. The destination store is where you want to copy files to.
35+
1. Navigate to the **Template Gallery** from the **Author** tab in Azure Data Factory, then choose the **+** button, **Pipeline**, and finally **Template Gallery**.
36+
37+
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/open-template-gallery.png" alt-text="Screenshot showing how to open the Template gallery from the Azure Data Factory Studio's Author tab.":::
38+
39+
1. Search for the template **Copy new files only by LastModifiedDate**, select it, and then select **Continue**.
40+
41+
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/select-last-modified-date-template.png" alt-text="Screenshot showing how to find and select the Copy new files only by LastModifiedDate template.":::
42+
43+
1. Create a **New** connection to your destination store. The destination store is where you want to copy files to.
3644

3745
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-1.png" alt-text="Create a new connection to the source":::
3846

39-
2. Create a **New** connection to your source storage store. The source storage store is where you want to copy files from.
47+
1. Create a **New** connection to your source storage store. The source storage store is where you want to copy files from.
4048

4149
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-3.png" alt-text="Create a new connection to the destination":::
4250

43-
3. Select **Use this template**.
51+
1. Select **Use this template**.
4452

4553
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-4.png" alt-text="Use this template":::
4654

47-
4. You will see the pipeline available in the panel, as shown in the following example:
55+
1. You see the pipeline available in the panel, as shown in the following example:
4856

4957
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-5.png" alt-text="Show the pipeline":::
5058

51-
5. Select **Debug**, write the value for the **Parameters** and select **Finish**. In the picture below, we set the parameters as following.
59+
1. Select **Debug**, write the value for the **Parameters**, and select **Finish**. In the picture that follows, we set the parameters as following.
5260
- **FolderPath_Source** = sourcefolder
5361
- **Directory_Source** = subfolder
5462
- **FolderPath_Destination** = destinationfolder
5563
- **Directory_Destination** = subfolder
5664
- **LastModified_From** = 2019-02-01T00:00:00Z
5765
- **LastModified_To** = 2019-03-01T00:00:00Z
5866

59-
The example is indicating that the files, which have been last modified within the timespan (**2019-02-01T00:00:00Z** to **2019-03-01T00:00:00Z**) will be copied from the source path **sourcefolder/subfolder** to the destination path **destinationfolder/subfolder**. You can replace these with your own parameters.
67+
The example is indicating that the files, which were last modified within the timespan (**2019-02-01T00:00:00Z** to **2019-03-01T00:00:00Z**) will be copied from the source path **sourcefolder/subfolder** to the destination path **destinationfolder/subfolder**. You can replace these times or folders with your own parameters.
6068

6169
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-6.png" alt-text="Run the pipeline":::
6270

63-
6. Review the result. You will see only the files last modified within the configured timespan has been copied to the destination store.
71+
1. Review the result. You see only the files last modified within the configured timespan are copied to the destination store.
6472

6573
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-7.png" alt-text="Review the result":::
6674

67-
7. Now you can add a tumbling windows trigger to automate this pipeline, so that the pipeline can always copy new and changed files only by LastModifiedDate periodically. Select **Add trigger**, and select **New/Edit**.
75+
1. Now you can add a tumbling windows trigger to automate this pipeline, so that the pipeline can always copy new and changed files only by LastModifiedDate periodically. Select **Add trigger**, and select **New/Edit**.
6876

6977
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-8.png" alt-text="Screenshot that highlights the New/Edit menu option that appears when you select Add trigger.":::
7078

71-
8. In the **Add Triggers** window, select **+ New**.
79+
1. In the **Add Triggers** window, select **+ New**.
7280

73-
9. Select **Tumbling Window** for the trigger type, set **Every 15 minute(s)** as the recurrence (you can change to any interval time). Select **Yes** for Activated box, and then select **OK**.
81+
1. Select **Tumbling Window** for the trigger type, set **Every 15 minute(s)** as the recurrence (you can change to any interval time). Select **Yes** for Activated box, and then select **OK**.
7482

7583
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-10.png" alt-text="Create trigger":::
7684

77-
10. Set the value for the **Trigger Run Parameters** as following, and select **Finish**.
85+
1. Set the value for the **Trigger Run Parameters** as following, and select **Finish**.
7886
- **FolderPath_Source** = **sourcefolder**. You can replace with your folder in source data store.
7987
- **Directory_Source** = **subfolder**. You can replace with your subfolder in source data store.
8088
- **FolderPath_Destination** = **destinationfolder**. You can replace with your folder in destination data store.
8189
- **Directory_Destination** = **subfolder**. You can replace with your subfolder in destination data store.
82-
- **LastModified_From** = **\@trigger().outputs.windowStartTime**. It is a system variable from the trigger determining the time when the pipeline was triggered last time.
83-
- **LastModified_To** = **\@trigger().outputs.windowEndTime**. It is a system variable from the trigger determining the time when the pipeline is triggered this time.
90+
- **LastModified_From** = **\@trigger().outputs.windowStartTime**. It's a system variable from the trigger determining the time when the pipeline was triggered last time.
91+
- **LastModified_To** = **\@trigger().outputs.windowEndTime**. It's a system variable from the trigger determining the time when the pipeline is triggered this time.
8492

8593
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-11.png" alt-text="Input parameters":::
8694

87-
11. Select **Publish All**.
95+
1. Select **Publish All**.
8896

8997
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-12.png" alt-text="Publish All":::
9098

91-
12. Create new files in your source folder of data source store. You are now waiting for the pipeline to be triggered automatically and only the new files will be copied to the destination store.
99+
1. Create new files in your source folder of data source store. You're now waiting for the pipeline to be triggered automatically and only the new files are copied to the destination store.
92100

93-
13. Select **Monitor** tab in the left navigation panel, and wait for about 15 minutes if the recurrence of trigger has been set to every 15 minutes.
101+
1. Select **Monitor** tab in the left navigation panel, and wait for about 15 minutes if the recurrence of trigger was set to every 15 minutes.
94102

95-
14. Review the result. You will see your pipeline will be triggered automatically every 15 minutes, and only the new or changed files from source store will be copied to the destination store in each pipeline run.
103+
1. Review the result. You see your pipeline is triggered automatically every 15 minutes, and only the new or changed files from source store are copied to the destination store in each pipeline run.
96104

97105
:::image type="content" source="media/solution-template-copy-new-files-last-modified-date/copy-new-files-last-modified-date-15.png" alt-text="Screenshot that shows the results that return when the pipeline is triggered.":::
98106

0 commit comments

Comments
 (0)