Reason in a continuous latent space instead of a language space

[Consistos]

This leads to a noticeable performance improvement and flexibility over forcing the model to output language at each reasoning step, as showcased in this publication. Code for an implementation has recently been open-sourced.

I think this is an improvement of R1, and as such begs whether this project aims to merely be a 1:1 reproduction or improve over it? The latter is IMO more exciting.

Edit: another related publication
Edit 2: yet another one

[ocramz]

@Consistos is there a quantitative comparison of R1 vs models trained with CoCoNut?

Also, open reproductions are 100% good science and pushing the field forward (more than closed models arguably).

[Consistos]

According to the result table in the pub., the score on ProsQA is ~25 % better than CoT while using ~28% of its tokens. The score on GSM8k is ~21% poorer than CoT while using >3 times less tokens. The one on ProntoQA is ~1% better while using ~10 times less tokens.

I reformulated my last sentence according to your remark.

Some excerpts from the pub.:

This modification frees the reasoning from being within the language space, and the system can be optimized end-to-end by gradient descent, as continuous thoughts are fully differentiable.

Unlike language-based reasoning, continuous thoughts in Coconut can encode multiple potential next steps simultaneously, allowing for a reasoning process akin to breadth-first search (BFS). While the model may not initially make the correct decision, it can maintain many possible options within the continuous thoughts and progressively eliminate incorrect paths through reasoning, guided by some implicit value functions.

In our analysis, we find that after removing the constraint of a language space, a new reasoning pattern similar to BFS emerges, even though the model is not explicitly trained in this way.