You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/sap-change-data-capture-advanced-topics.md
+9-7Lines changed: 9 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,18 +42,18 @@ When **Source type** is **Inline**, the following properties can be parametrized
42
42
-**ODP_SELF** and **RANDOM** are ODP contexts used for technical validation and testing, and are typically not relevant.
43
43
-**ODP name**: provide the ODP name you want to extract data from.
44
44
-**Run mode**: valid parameter values are
45
-
-**fullAndIncrementalLoad** for **Full on the first run, then incremental**, which initiates a change data capture process and extracts a current full data snapshot
45
+
-**fullAndIncrementalLoad** for **Full on the first run, then incremental**, which initiates a change data capture process and extracts a current full data snapshot.
46
46
-**fullLoad** for **Full on every run**, which extracts a current full data snapshot without initiating a change data capture process.
47
47
-**incrementalLoad** for **Incremental changes only**, which initiates a change data capture process without extracting a current full snapshot.
48
48
-**Key columns**: key columns are provided as an array of (double-quoted) strings. For example, when working with SAP table **VBAP** (sales order items), the key definition would have to be \["VBELN", "POSNR"\] (or \["MANDT","VBELN","POSNR"\] in case the client field is taken into account as well).
49
49
50
50
### Parametrizing the filter conditions for source partitioning
51
51
52
52
In the **Optimize** tab, a source partitioning scheme (see [optimizing performance for full or initial loads](connector-sap-change-data-capture.md#optimizing-performance-of-full-or-initial-loads-with-source-partitioning)) can be defined via parameters. Typically, two steps are required:
53
-
1. Define a parameter for the source partitioning scheme on pipeline level.
53
+
1. Define the source partitioning scheme.
54
54
2. Ingest the partitioning parameter into the mapping data flow.
55
55
56
-
#### JSON format of a partitioning scheme
56
+
#### Define a source partitioning scheme
57
57
58
58
The format in step 1 follows the JSON standard, consisting of an array of partition definitions, each of which itself is an array of individual filter conditions. These conditions themselves are JSON objects with a structure aligned with so-called **selection options** in SAP. In fact, the format required by the SAP ODP framework is basically the same as dynamic DTP filters in SAP BW:
59
59
@@ -93,7 +93,7 @@ where the first partition contains fiscal years (GJAHR) 2011 through 2015, and t
93
93
>[!NOTE]
94
94
> Azure Data Factory doesn't perform any checks on these conditions. For example, it is in the user's responsibility to ensure that partition conditions don't overlap.
95
95
96
-
Partition conditions can be more complex, consisting of multiple elementary filter conditions themselves. There are no logical conjunctions that explicitly define how to combine multiple elementary conditions within one partition. The implicit definition in SAP is as follows (only for **including** selections, that is, "sign": "I" - for **excluding**):
96
+
Partition conditions can be more complex, consisting of multiple elementary filter conditions themselves. There are no logical conjunctions that explicitly define how to combine multiple elementary conditions within one partition. The implicit definition in SAP is as follows:
97
97
1.**including** conditions ("sign": "I") for the same field name are combined with **OR** (mentally, put brackets around the resulting condition)
98
98
2.**excluding** conditions ("sign": "E") for the same field name are combined with **OR** (again, mentally, put brackets around the resulting condition)
99
99
3. the resulting conditions of steps 1 and 2 are
@@ -105,11 +105,13 @@ As an example, the partition condition
corresponds to a SQL WHERE clause ... **WHERE ("BUKRS" = '1000') AND ("GJAHR" BETWEEN '2020' AND '2025') AND NOT ("GJAHR" = '2023')**
114
+
corresponds to a SQL WHERE clause ... **WHERE ("BUKRS" = '1000' OR "BUKRS" = '1010') AND ("GJAHR" BETWEEN '2010' AND '2025') AND NOT ("GJAHR" = '2021' or "GJARH" = '2023')**
113
115
114
116
>[!NOTE]
115
117
> Make sure to use the SAP internal format for the low and high values, include leading zeroes, and express calendar dates as an eight character string with the format \"YYYYMMDD\".
0 commit comments