Add a method to push struct validity into children

[blaginin]

Unlike Arrow's RecordBatches, StructArray can be "nullable" in two places: in the child arrays and also at the top level. This makes some struct operations tricky - for example, how do you zip two arrays with different top-level validity?

One solution could be to add a method that pushes top-level validity down to the children (assuming all fields are nullable). For example:

{a: 1, b: 2}, NULL would become {a: 1, b: 2}, {a: NULL, b: NULL}

[robert3005]

We have https://github.com/vortex-data/vortex/blob/develop/vortex-array/src/compute/mask.rs#L48, however maybe the faster way is to accept mask in some compute functions and let them decide how to apply it

[blaginin]

We have https://github.com/vortex-data/vortex/blob/develop/vortex-array/src/compute/mask.rs#L48

Sure! But what do you think about adding a separate method for the struct? This feels like a super common usecase - have written this three times in spiral last week

[blaginin]

Okay, so what needs to be done here:

• We need to implement a method to push validity
• We need to add docs
• Method should be tested

I actually wrote an in-place method, feel free to use this as a reference / base on top of that: https://gist.github.com/blaginin/9927afd94a8d5bb272135d0c5dc99652

[amorynan]

Is this issue on going ? and can I pick this up for diving deep into vortex ?

[gatesn]

Yes!

Since this issue was filed, we now have a "Mask" expression that essentially performs intersection with validity.

So you could take a look at how you might take the validity from the struct array itself, and wrap up each child field in a mask expression (see builtins.rs), before constructing a new StructArray with or without the previous validity. These two are different behaviors.

Also please note most of the team are off for the holidays until early next year, so responses may be slow or delayed. Thanks for getting involved!