Implement feed.audit()

[yoshuawuyts]

Feature Request

Summary

Implement feed.audit() to verify all data you think you have is actually there.

Motivation

This gives us parity with holepunchto/hypercore#180.

[tdeebswihart]

Good morning. I've implemented feed.audit() over here.

However thanks to Rust's borrow checker I'm having trouble writing a test for the "bad data" scenario: it wisely prevents me from altering storage while the Feed has ownership of it.

Any idea of how to insert bad data here?

[yoshuawuyts]

@chronoslynx oh that's awesome! -- a PR would be super welcome, even if there isn't any test data included yet! Apologies for not replying sooner; I was on vacation for the past few weeks.

I'm not quite sure how to edit the bad data yet, haha. Perhaps if we re-initialize the feed from disk we can get it into a bad state? Perhaps we can workshop this a bit in the PR?

Thanks so much for putting in the work; that's amazing! :D

[tdeebswihart]

Hey, no need to apologize. Vacation is important!

Inserting bad data will be... weird. Does it write the data immediately to disk (if you're using disk-backed storage)? If so (and we can get the filename), we can open a second handle to it and stomp the data.
I'll need to read into your storage implementation to see

[yoshuawuyts]

Inserting bad data will be... weird. Does it write the data immediately to disk (if you're using disk-backed storage)? If so (and we can get the filename), we can open a second handle to it and stomp the data.

Yeah, that's what I was thinking! -- It's very possible that might surface some other fun issues, but that'll be good to verify!

We implement fsync() on Drop in random-access-storage, so that should probably work!

[tdeebswihart]

fsync is also called after every write, and the disk is consulted on every read which leads to this being simpler than I expected.

It's brittle, however. If the disk implementation every changes the name of the data file, then this test will break