bug: possibility of writing invalid headers yielding unreadable files

[alisterburt]

Hey @colinpalmer - as per our discussion:

In juglab/cryoCARE_pip#43 we saw that files were being written, copying the original header but not the extended header. The length of the extended header is stored in the header and as such the start of the data is incorrectly specified in the files mrcfile writes.

Proposal: check and update the header entry containing the number of bytes of the extended header when writing a file to avoid this problem in the future

[cyanching]

We see the same issue in cryoCARE output files as well as mrc files we generated from these output files, @alisterburt do you maybe have more information on a workaround when reading such files with mrcfile? Thank you so much for your help!

[alisterburt]

Hey @cyanching ! You could probably open the file with header_only=True and permissive=True then set the number of bytes in the extended header to 0 and save it - haven't tested this but it should work!

[cyanching]

Hey Alister! Thank you very much for your prompt and helpful reply as always! :) Sorry for the late reply, I somehow did not get a notification earlier. After updating mrcfile to 1.5.0 (released in Jan, 2024), it stopped complaining about header info generated by cryoCARE which was kept for cryoCARE outputs and mrc files we generated from them. I will definitely try your suggestion if we encounter more header issues!

[alisterburt]

Thanks @cyanching
@colinpalmer did you push a fix for this? Should we close this issue?

[colinpalmer]

No, I haven't made any changes to fix this issue yet. I'm guessing that maybe the thing that fixed it for @cyanching was using a version of cryocare that included your fix, rather than the mrcfile version change?

[alisterburt]

That would make sense, forgot about that PR

[cyanching]

Thank you @colinpalmer for the link! Since @alisterburt contributed the header fix for cryoCARE in March, 2023. I indeed used a later version of cryoCARE but believed it was changes in mrcfile that solved this problem.