automated integration tests

[slingamn]

Investigate https://github.com/charmbracelet/vhs for this: https://www.youtube.com/watch?v=dtVezRX8OEM

[wader]

Played around a bit, very impressive, have heard about before but didn't really understand what it was about. I'm using some expect for some basic fq CLI/REPL tests but it is a mess and very shakey.

[tpodowd]

Yeah. I am using expect also for testing. This looks promising.

[wader]

I tried vhs a bit for testing fq but i'm having trouble making the "golden" output stable, maybe i'm doing something wrong.

@tpodowd Having any success using expect to test readline completion? having trouble with that too :)

Example tape that i tried that changes a bit with every run:

Output test.ascii

Type "fq -i . doc/file.mp3"
Enter
Sleep 0.2s
Type "repeat(empty)"
Enter
Sleep 0.5s
Enter
Ctrl+C
Sleep 0.5s
Type ".h"
Tab
Sleep 0.5s
Enter
Sleep 0.5s
Type ".frames[0] | repl"
Enter
Type ".audi"
Tab
Sleep 0.5s
Enter
Sleep 0.5s
Ctrl+D
Sleep 0.5s
Ctrl+D
Sleep 0.5s
Ctrl+D

[tpodowd]

Hi @wader - I'm going to be no help and say that for completion testing, I'm not actually testing the terminal output but instead I'm constructing tests around the AutoCompleter interface Do() function which I implement and ensuring my behaviour is right at that level as that is where my complexity is.

[wader]

@tpodowd I see, no worries 👍 Yeah i do something similar but turned out the test script version of the readline works a tiny bit different so didn't catch a regression. Can post here if i figure how to do it, maybe useful for someone