Skip to content

Commit 3657c80

Browse files
committed
roll back type inheritance for the fixedWidth intrinsic
the arguments and results don't necessarily match when the whole file may be missing columns the transform is intended to repair
1 parent 12ce718 commit 3657c80

File tree

3 files changed

+2
-7
lines changed

3 files changed

+2
-7
lines changed

tessellate-main/src/main/antora/modules/reference/pages/transforms.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ Placeholders:::::
106106
`counterToZero`:::: Resets the counter portion when the millisecond changes, defaults to `false`.
107107

108108
`fixedWidth`:: Pads a row/tuple to a fixed width by inserting nulls at a given index.
109-
If any, type information is inherited from the current fields.
109+
Ensure type information is declared for result fields if required.
110110
Def:::
111111
- `^fixedWidth{width:...,insertAt:...} ->` - replace ALL fields with the fixed with result
112112
- `^fixedWidth{width:...,insertAt:...} -> new_field|type + new_field2|type + etc` - name the new fields

tessellate-main/src/main/java/io/clusterless/tessellate/pipeline/intrinsic/FixedWidthIntrinsic.java

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,6 @@ public Result create(Fields currentFields, Operation operation) {
4949
throw new IllegalArgumentException("result fields width must match fixed width");
5050
}
5151

52-
// currentFields is equivalent to Field.ALL so we can safely copy over the type information
53-
if (!toFields.hasTypes() && currentFields.hasTypes()) {
54-
toFields = toFields.applyTypes(currentFields.getTypes());
55-
}
56-
5752
FixedWidthFunction function = new FixedWidthFunction(toFields, width, insertAt);
5853

5954
return new Result(Fields.ALL, function, toFields);

tessellate-main/src/test/java/io/clusterless/tessellate/pipeline/PipelineTest.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ void badWidthWithFields(@PathForResource("/data/delimited-variable-width.csv") U
108108
@Test
109109
void badWidthWithFieldsAndTypes(@PathForResource("/data/delimited-variable-width.csv") URI input, @PathForOutput URI output) throws IOException {
110110
Transform transform = new Transform(
111-
"^fixedWidth{ width:5, insertAt:3 } -> _0+_1+_2+_3+_4"
111+
"^fixedWidth{ width:5, insertAt:3 } -> _0|string+_1|string+_2|string+_3|string+_4|string"
112112
);
113113

114114
fixedWidthBase(input, output, transform, Field.asField("a|string", "b|string", "c|string", "d|string", "e|string"));

0 commit comments

Comments
 (0)