Gzipped FITS BinTableHDU readonly now

[nabobalis]

Recently, we noticed a test failure within SunPy against astropy master.

We have a test that opens a gzipped fits file and makes some adjusts to the data to remove potential bad values. We get that

ValueError: assignment destination is read-only

This seems to only occur for any FITS data that is returned with a astropy.io.fits.hdu.table.BinTableHDU, astropy.io.fits.hdu.image.PrimaryHDU are unaffected by this.

Example would be:

from astropy.io import fits
afile = fits.open('go1520120601.fits.gz') #from https://github.com/sunpy/sunpy/blob/master/sunpy/data/test/go1520120601.fits.gz
print(afile)
#[<astropy.io.fits.hdu.image.PrimaryHDU object at 0x7fbc7c00fc88>, 
#<astropy.io.fits.hdu.table.BinTableHDU object at 0x7fbc7c018080>,
#<astropy.io.fits.hdu.table.BinTableHDU object at 0x7fbc7a5975f8>,
#<astropy.io.fits.hdu.table.BinTableHDU object at 0x7fbc7a5a1438>]
afile[1].data[0][0][0][0] = 2
# ValueError: assignment destination is read-only

Is this intended behaviour?

[saimn]

From the slack discussion, this was introduced by the np.fromstring -> np.frombuffer replacement (#6785). np.frombuffer produces a readonly array, which was not intended there. Not sure what's the best solution.

[saimn]

pandas did the same change (pandas-dev/pandas#17987), copying the array to set .flags.writeable = True. But first I think we need to find which line changed in #6785 is causing this, then there may be a better solution (I guess this is the change in _array_from_file).

[Cadair]

Would we want the copy? I don't really understand the ramifactions of copying the table data like that. Would just setting the writeable flag be sufficient? (I tested this and it fixes the bug on our end).

[nabobalis]

The panda comment says:
buf = buf.copy() # required to not mutate the original data

[cdeil]

We also have a very similar test fail in Gammapy. For us, the issue occurs with Table like this, i.e. we call table.sort after reading a gzipped FITS BinTable:

>>> from astropy.table import Table
>>> t = Table(dict(a=[1, 2, 3]))
>>> t.write('a.fits.gz')
>>> t2 = Table.read('a.fits.gz')
>>> t2.sort('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/deil/Library/Python/3.6/lib/python/site-packages/astropy/table/table.py", line 2450, in sort
    col[:] = col.take(indexes, axis=0)
  File "/Users/deil/Library/Python/3.6/lib/python/site-packages/astropy/table/column.py", line 932, in __setitem__
    self.data[index] = value
ValueError: assignment destination is read-only

cc @taldcroft @astrofrog

[saimn]

Ok, see #6923 for a fix, doing the copy.

@cdeil - Yes, it's the same issue.

[saimn]

@cdeil @nabobalis - It's now fixed, if you can check that it works for you, thanks.

[nabobalis]

I will run a job now. Thanks for the fix!