After zfs rollback, XML files have many trailing null (0x00) bytes

[anyone2k15]

I noticed that after performing a zfs rollback on my dataset, some XML files have many trailing null bytes (0x00) appended at the end.

For example, the correct XML file ends cleanly like this (hex dump):

00002010: 6974 653e 6874 7470 733a 2f2f 7365 656a  ite>https://seej
00002020: 6176 2e63 6c75 622f 444d 4f57 2d32 3133  av.club/DMOW-213
00002030: 3c2f 7765 6273 6974 653e 0a3c 2f6d 6f76  </website>.</mov
00002040: 6965 3e                                  ie>

But after rollback, the same file contains many trailing zero bytes:

00001ff0: 6974 653e 6874 7470 733a 2f2f 7365 656a  ite>https://seej
00002000: 6176 2e63 6c75 622f 444d 4f57 2d32 3133  av.club/DMOW-213
00002010: 3c2f 7765 6273 6974 653e 0a3c 2f6d 6f76  </website>.</mov
00002020: 6965 3e00 0000 0000 0000 0000 0000 0000  ie>.............
00002030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00002040: 0000 00                                  ...

I suspect this is due to ZFS rollback restoring file blocks but not correctly updating the file size metadata, leaving zero-padding at the end of the file.

Has anyone encountered this issue? What is the recommended way to fix or avoid trailing null bytes in files after a ZFS rollback?

[amotin]

ZFS rollbacks to exact state of things that was when the snapshot was created. The question to me here is how exactly that file is written so interestingly that it is somehow resized and written in different transactions, creating a window for snapshot to happen i between?

[anyone2k15]

Heya! ZFS does rollback to the exact state of filesystems/zvols when the snapshot was taken. That is CoW!

Let me introduce the story. I have tons of film metadata on my ZFS storage. I just wrote a small Python program to parse and re-write several elements in the XML film metadata. The quantity of those XMLs is around 50,000 and each of them is around 150KiB. I'm not confident in my program so I just copy those XMLs to a separate ZFS filesystem/dataset and then run sync and zfs snapshot. After that, I can safely run my program and check the result.

My program has been edited and needs to be run again and again to debug. Before rerunning it I shall run zfs rollback && sync at first. But the XML parser said some xml file is invalid! Using xxd to check the file internally, I found several NULL bytes were appended to the invalid xml.

After that, I have checked the original xml files and there is no NULL bytes. The most interesting thing is that after I delete those invalid NULL bytes and run zfs rollback again, those bytes were vanished! 🫠