Make sure string column with vec size 1 work#427
Conversation
These get read as scalars, but make sure the data round trips
In numpy 2 the ambiguity for 1-d vec seems to be gone
beckermr
left a comment
There was a problem hiding this comment.
The code looks fine but I think I am missing some details on what the special case is exactly. I've asked for a few comments in places to help future us understand this edge case.
beckermr
left a comment
There was a problem hiding this comment.
I missed that this is a breaking change wrt the behavior of returning dims of (1,) versus a scalar. We should figure that bit out.
See also #426 (comment)
|
It is a breaking change, in that those columns will now be properly round tripped whereas they were not before. On the other hand, I consider this a bug in numpy that I couldn't get this to work in 1.X. Seen that way it is breaking in the sense of correcting behavior. |
|
Is there a way we can make it work for non-strings? |
|
There is no way to set the TDIM for length-1 vector columns, they must be length 2 or more. I can see hacks working. We could add a header keyword that tells us the situation for a given column so fitsio can reshape the data. |
|
My feeling on this specific item is that this specific behavior of FITSIO is so ingrained in people's codes + minds that if we break it, we're going to cause silent or at least very weird bugs. I get that it can be considered a bug, but long-standing bugs that people code around effectively become part of the API. |
Right. My feeling is that we
This seems to me to be the only way to do this without causing lots of bad side effects. |
|
We could include some of the other big ideas we've discussed in version 2 as well, including the new I/O modes+features. |
|
I don't actually know how to workaround the new numpy behavior, will have to figure it out. |
|
Can't we just change the output dtype for strings if we find TDIM has 2 elements? |
|
There is code reuse for read and write. It seems like different behavior might be needed for read vs write in this case, but need to look into it. |
closes #426
This fixes a bug where the shape of the column was not correct.
Adds tests that string and number vec1 column data get returned, even though read as scalar