@@ -43,69 +43,17 @@ The resulting CosmosDB container will embed the inner query into a single docume
43
43
44
44
1 . Select ** +New Pipeline** to create a new pipeline.
45
45
46
- 2 . Add a data flow activity, which will be used for processing fixed-width files:
46
+ 2 . Add a data flow activity
47
47
48
- ![ Fixed Width Pipeline] ( media/data-flow/fwpipe.png )
48
+ ![ CosmosDB Pipeline] ( media/data-flow/fwpipe.png )
49
49
50
50
3 . In the data flow activity, select ** New mapping data flow** .
51
51
52
- 4 . Add a Source, Derived Column, Select, and Sink transformation:
52
+ 4 . We will construct this data flow graph below
53
53
54
- ![ Fixed Width Data Flow] ( media/data-flow/fw2 .png )
54
+ ![ Data Flow Graph ] ( media/data-flow/cosmosb1 .png )
55
55
56
- 5 . Configure the Source transformation to use a new dataset, which will be of the Delimited Text type.
57
-
58
- 6 . Don't set any column delimiter or headers.
59
-
60
- Now we'll set field starting points and lengths for the contents of this file:
61
-
62
- ```
63
- 1234567813572468
64
- 1234567813572468
65
- 1234567813572468
66
- 1234567813572468
67
- 1234567813572468
68
- 1234567813572468
69
- 1234567813572468
70
- 1234567813572468
71
- 1234567813572468
72
- 1234567813572468
73
- 1234567813572468
74
- 1234567813572468
75
- 1234567813572468
76
- ```
77
-
78
- 7. On the **Projection** tab of your Source transformation, you should see a string column that's named *Column_1*.
79
-
80
- 8. In the Derived column, create a new column.
81
-
82
- 9. We'll give the columns simple names like *col1*.
83
-
84
- 10. In the expression builder, type the following:
85
-
86
- ```substring(Column_1,1,4)```
87
-
88
- 
89
-
90
- 11. Repeat step 10 for all the columns you need to parse.
91
-
92
- 12. Select the **Inspect** tab to see the new columns that will be generated:
93
-
94
- 
95
-
96
- 13. Use the Select transform to remove any of the columns that you don't need for transformation:
97
-
98
- 
99
-
100
- 14. Use Sink to output the data to a folder:
101
-
102
- 
103
-
104
- Here's what the output looks like:
105
-
106
- 
107
-
108
- The fixed-width data is now split, with four characters each and assigned to Col1, Col2, Col3, Col4, and so on. Based on the preceding example, the data is split into four columns.
56
+ 5 .
109
57
110
58
## Next steps
111
59
0 commit comments