@@ -219,6 +219,39 @@ scala> df.nestedMapColumn(inputColumnName = "my_array.a", outputColumnName = "c"
219219+---+---------------------------------------+
220220```
221221
222+ ## Other transformations
223+
224+ ### Unstruct
225+
226+ Syntax: ` df.nestedUnstruct("NestedStructColumnName") ` .
227+
228+ Flattens one level of nesting when a struct is nested in another struct. For example,
229+
230+ ``` scala
231+ scala> df.printSchema
232+ root
233+ |-- id : long (nullable = true )
234+ |-- my_array : array (nullable = true )
235+ | |-- element : struct (containsNull = true )
236+ | | |-- a : long (nullable = true )
237+ | | |-- b : string (nullable = true )
238+ | | |-- c : struct (containsNull = true )
239+ | | | |-- nestedField1 : string (nullable = true )
240+ | | | |-- nestedField2 : long (nullable = true )
241+
242+ scala> df.nestedUnstruct(" my_array.c" ).printSchema
243+ root
244+ |-- id : long (nullable = true )
245+ |-- my_array : array (nullable = true )
246+ | |-- element : struct (containsNull = true )
247+ | | |-- a : long (nullable = true )
248+ | | |-- b : string (nullable = true )
249+ | | |-- nestedField1 : string (nullable = true )
250+ | | |-- nestedField2 : long (nullable = true )
251+ ```
252+
253+ Note that the output schema doesn't have the ` c ` struct. All fields of ` c ` are now part of the parent struct.
254+
222255## Changelog
223256- #### 0.2.1 released 21 January 2020.
224257 - [ #10 ] ( https://github.com/AbsaOSS/spark-hats/issues/10 ) Fixed error column aggregation when the input array is ` null ` .
0 commit comments