Skip to content

Conversation

ggerganov
Copy link

@ggerganov ggerganov commented Oct 17, 2024

target: ggml-org#9833

@clarismiranda
Copy link
Owner

Looks great, thank you!

@clarismiranda clarismiranda merged commit 34fc44d into clarismiranda:grammar-memo Oct 17, 2024
new_stack.push_back(pos);
}
llama_grammar_advance_stack_memo(rules, new_stack, stacks_new, stacks_cache);
llama_grammar_advance_stack_memo(grammar->rules, new_stack, stacks_new, grammar->stacks_cache);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the signature simplification that you made for llama_grammar_accept -- passing around the grammar structure alone (rather than all of the individual pieces of the structure) feels good.

Should we do that to llama_grammar_advance_stack / llama_grammar_advance_stack_memo as well?

I.E. something more like:

static void llama_grammar_advance_stack_memo(
        struct llama_grammar * grammar,
        const llama_grammar_stack  & stack,
              llama_grammar_stacks & new_stacks) {

?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should give it a try. I think I briefly tried it again, but there was some issue. Will retake a look tomorrow and see if we can simplify further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants