Order of Dimensions when converting DataArray to DataFrame back to DataArray.

[benHeid]

I try to figure out the conversion process between xarray and pandas. In particular, I want to transform a xarray dataarray into an pandas dataframe and then back to a an array dataarray.

I used the following dataarray

da = xr.DataArray([[1, 3], [4, 7], [0.5, 2], [-3, -3 / 7]], dims=["index", "columns"], coords={"index": [0,1,2,3], "columns":["a", "b"]})

which I convert to pandas via to_pandas

da.to_pandas()

columns    a         b
index                 
0        1.0  3.000000
1        4.0  7.000000
2        0.5  2.000000
3       -3.0 -0.428571

now I convert this dataframe back to a xarray DataArray

da.to_pandas().to_xarray().to_array()

<xarray.DataArray (variable: 2, index: 4)>
array([[ 1.        ,  4.        ,  0.5       , -3.        ],
       [ 3.        ,  7.        ,  2.        , -0.42857143]])
Coordinates:
  * index     (index) int64 0 1 2 3
  * variable  (variable) object 'a' 'b'

My question is why are the dimensions now changed? Before conversion the dataarray has the shape (4,2). After the conversion the shape is (2,4).

I am not sure if this is a bug or if there is a reason for this behaviour, which I do not understand yet. Thus, I posted this as a discussion. I really would appreciate any comments on that.

[mathause]

I am not sure what goes wrong, but you should be able to round-trip the dataarray using to_dataframe:

da.to_dataframe(name="foo").to_xarray()["foo"]

[benHeid]

Thank you for your reply. The reason why I selected to_pandas was that this creates a two-dimensional data frame, which is prettier than unstacking the DataFrame after to_dataframe.

I suppose the reason for this behaviour is that each columns becomes a data_var and that inside of the to_array function. In line 5796 of the Dataset the data_vars are stacked on the axis=0, for my use case it has to be the axis=1. This is probably nothing which can be changed easily without breaking other things. So I have to go with a workaround I suppose. :)

[rhkleijn]

It is easily overlooked, but the documentation of to_pandas() mentions: The DataArray constructor performs the inverse transformation.

With that approach roundtripping indeed seems to work correctly for your example:

import xarray as xr
da = xr.DataArray([[1, 3], [4, 7], [0.5, 2], [-3, -3 / 7]], dims=["index", "columns"], coords={"index": [0,1,2,3], "columns":["a", "b"]})
df = da.to_pandas()
xr.DataArray(df).identical(da)
Out[1]: True