As a testing framework, we need to ensure our correctness. Currently we can be relatively confident whether previously passed tests will fail after upgrading sqllogictest by testing agains real usage in e.g., risingwave. But it would definitely be better to add a test suite in our repo. #123
A more dangerous problem which is also harder to detect is that we won't pass previously failed tests. Imagine that after some changes, we allow all test cases to pass. 😇 We need to test errors.
Here's an example of error that won't be run:
|
# statement is expected to fail with error: "Hey we", but got error: "Hey you got FakeDBError!" |
|
# statement error Hey we |
|
# give me an error |
As a testing framework, we need to ensure our correctness. Currently we can be relatively confident whether previously passed tests will fail after upgrading sqllogictest by testing agains real usage in e.g., risingwave. But it would definitely be better to add a test suite in our repo. #123
A more dangerous problem which is also harder to detect is that we won't pass previously failed tests. Imagine that after some changes, we allow all test cases to pass. 😇 We need to test errors.
Here's an example of error that won't be run:
sqllogictest-rs/examples/basic/basic.slt
Lines 32 to 34 in d753e4c