Currently, our synthesis engine is dog slow:
- We're mostly brute-forcing things with a very naive implementation of minikanren;
- The way we're sampling from the goals is extremely biased in a bad way, most of our candidates consist of literals from the first letters of the alphabet since we generate them sequentially;
- Our failure is a boolean criteria, which makes for a very poor feedback mechanism;
Spawn a team to ideate and research ways to improve this, such as: bottom-up candidate elaboration rather than top down, genetic algorithms (like coverage guided fuzzing does maybe?), or maybe some kind of RL that learns to generate better candidates, or maybe allowing us to access the context of the whole AST when generating candidates (undestanding that the current lit node is child of an alt, and maybe even accessing what it's siblings are)
There's a lot of ground to cover in the design space for this. We plan on taking a parsed corpus of the language (which was parsed by another already existing parser) and then learning a parser that generates identical output. Spawn a team to ideate on plausible solutions. The ones that think should use Opus, the ones that search the web (if any) should use Sonnet