Improve robustness coverage of revision window for Watch and other read methods. #20349
Description
What would you like to be added?
Improving revision coverage with regards to reads
Recently, a correctness issue #20221 was discovered related to watch operations regarding future revisions. The issue was not reproducible in robustness test because we lack dedicated testing for future revision. This means that the issue is very rare, that robustness cannot reproduce but Antithesis can thanks to dedicated assertion that guide exploration.
We should improve coverage of revision for watch and other read methods, as it’s currently lacking.
Problem:
Revisions represent history of changes, so covering them means we are testing a moving window of etcd history. As etcd receives writes and compacts the window changes. At any point there are 3 areas:
- Future revisions, ask about changes that etcd cluster or etcd member has not yet observed.
- Available revisions, ask about history of changes that etcd is still storing and can respond to
- Compacted revisions, ask about history so old that etcd has already purged.
During the test we are doing concurrent reads and writes, so the window of future,available,compacted revisions is shifting. We should propose a method that covers all those thee cases well enough and has some way to guarantee that.
Why is this needed?
Be able to prevent issues like #20221 in the future