`fit_*_random_forest` : allow vector-cube column selection

[clausmichele]

After a discussion with @ValentinaHutter @LukeWeidenwalker and @mattia6690 we concluded that:

• The fit_*_random_forest processes have vector-cubes as input data types
• The process should allow the user to select which property (or column, if we refer to a general DB) of the vector-cube to select to get the required data (@jdries mentioned this already in the last dev meeting)
• This is required if the target vector-cube is not the output of aggregate_spatial but it comes from a file (it is the case of UC8). Sorry if this didn't come up earlier but I wasn't fully aware on how the target vector-cube should look like.

@m-mohr how would you add the column selection to the process? My idea of vector-cube, consists in a table where we must have the geometry column, all the rest is optional. We would like to select a column among those optional ones.

For example, from this vector-cube we could select the class field from the geoJSON properties; https://raw.githubusercontent.com/clausmichele/openeo_aggregate_spatial_vector_cubes/master/urban_forest_points.geojson

[m-mohr]

I'm also not 100% sure yet how we handle properties in such cases. In theory, such a property could be a dimension or it still resides in the features, which would require additional processes.

Regardless of how the vector cube looks like, we likely need to add a new parameter for this use case to the fit_* processes.

[clausmichele]

This is for example my version of a vector-cube: a table with the geometry field plus some other columns. This would be the output of aggregate_spatial for instance

idx	class	geometry	result	result_meta
0	0	POINT (11.12157 46.06955)	{'B04_10m': 1522.0, 'B08_10m': 2127.5}	{'total_count': 2.0, 'valid_count': 2.0}
1	0	POINT (11.12171 46.06909)	{'B04_10m': 1121.5, 'B08_10m': 1547.0}	{'total_count': 2.0, 'valid_count': 2.0}
2	0	POINT (11.12324 46.06875)	{'B04_10m': 1045.5, 'B08_10m': 1406.0}	{'total_count': 2.0, 'valid_count': 2.0}
3	0	POINT (11.12326 46.06900)	{'B04_10m': 1093.5, 'B08_10m': 1497.5}	{'total_count': 2.0, 'valid_count': 2.0}
4	0	POINT (11.12370 46.06757)	{'B04_10m': 1455.0, 'B08_10m': 2059.5}	{'total_count': 2.0, 'valid_count': 2.0}
...	...	...	...	...
100	1	POINT (11.14474 46.13561)	{'B04_10m': 812.5, 'B08_10m': 3255.5}	{'total_count': 2.0, 'valid_count': 2.0}
101	1	POINT (11.15225 46.13465)	{'B04_10m': 614.0, 'B08_10m': 2137.5}	{'total_count': 2.0, 'valid_count': 2.0}
102	1	POINT (11.15215 46.13448)	{'B04_10m': 559.5, 'B08_10m': 2053.0}	{'total_count': 2.0, 'valid_count': 2.0}
103	1	POINT (11.15340 46.13687)	{'B04_10m': 549.5, 'B08_10m': 2001.0}	{'total_count': 2.0, 'valid_count': 2.0}
104	1	POINT (11.15475 46.13695)	{'B04_10m': 596.0, 'B08_10m': 1806.5}	{'total_count': 2.0, 'valid_count': 2.0}

[edzer]

The process should allow the user to select which property (or column, if we refer to a general DB) of the vector-cube to select to get the required data (@jdries mentioned this already in the last dev meeting)

In our definition of a data cube, cell values are scalars (see here, sixth paragraph), we do not allow for multiple variables (or attributes) at each combination of dimension values (cell = scalar). For vector data cubes, this should also hold. So by defining the vector data cube from the geojson file above you need to specify first which field holds the data cube values (cell values), which should be class, before you can call it a data cube. Then you end up with a one-dimensional vector data cube with geometries (points) as dimension values, and the class values as cell values. And hence, then you don't need to select an attribute.

If the geojson file above would have 4 attributes, B1, B2, B3, B4, they could contain the values of a second dimension, band, with dimension values B1,...,B4, and imported as a vector data cube with dimension npoints x 4. Each combination (POINT, band) would give a single, scalar value.

In the above geojson file the attributes are of different type, so can never be molded into a (useful) data cube dimension.

[m-mohr]

In the above geojson file the attributes are of different type, so can never be molded into a (useful) data cube dimension.

I assume "geojson file" was meant to refer to the table that Michele posted here? #341 (comment)

[clausmichele]

But then how should the output of aggregate_spatial look like if the input has dimensions x,y,bands or x,y,time ? I couldn't find any other way of satisfying the requirements of the process, since we need to create a new target dimension (not multiple) which can hold values from different bands or timesteps.
https://processes.openeo.org/#aggregate_spatial

[m-mohr]

As far as I understand it, the definition of aggregate_spatial simply doesn't work with the definition that @edzer proposes.

[clausmichele]

Well, we will keep this internal representation of the vector-cube for now since we need it for UC8, when we'll have a clearer definition we could modify it.

[m-mohr]

@clausmichele I guess it would be better to flatten result and result_meta? Does that make sense?

If I understood @edzer correctly, then the data cube may look like this:

Dimensions:

• geometry with labels: POINT (11.12157 46.06955), POINT (11.12171 46.06909), ...
• properties (or result to follow the default target dimension in aggregate_spatial) with labels: class, B04_10m, B08_10m, total_count, valid_count

geometry v \ properties >	class	B04_10m	B08_10m	total_count	valid_count
POINT (11.12157 46.06955)	0	1522.0	2127.5	2	2
POINT (11.12171 46.06909)	0	1121.5	1547.0	2	2
…	…	…	…	…	…

We'd need to fix the description of the returned vector cube in aggregate_spatial then.

[clausmichele]

Fine for me! I've just discussed with @ValentinaHutter and she was also implementing it like this.