[Feature Request]: Support for structured text generation via outlines lib.

[Tanmaypatil123]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Outlines provides ways to control the generation of language models to make their output more predictable.

following is the example of how outlines can be used for structured text generation task :

import outlines

model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")

prompt = "<s>result of 9 + 9 = 18</s><s>result of 1 + 2 = "
answer = outlines.generate.format(model, int)(prompt)
print(answer)
# 3

prompt = "sqrt(2)="
generator = outlines.generate.format(model, float)
answer = generator(prompt, max_tokens=10)
print(answer)
# 1.41421356

Github repo link : outlines

Motivation

current implementation of llamaindex model wrappers for OpenAI gpt's and HuggingFace's open source models doesn't support structured output generation

it will be helpful for generating more robust, formatted and structured output by RAG .

Proposal (If applicable)

No response

[Aillian]

Very interesting idea, would be a great addition to the framework

[shroominic]

I think it would be insteresting having outlines as ChatModel Interface with additional methods that expose the features of outlines but also implementing all Runnable methods. This allowes to use the fast structured output engine of outlines for local funtion calling or grammars.

[Tanmaypatil123]

can anyone let me know how can i raise feature request ... no one from langchain staff responding to this discussion. I can also contribute it

[Jofthomas]

Hey ! In InferecenClient, we do support grammar to have some output format : please see this link

I'm pretty sure it use outlines under the hood. I can work on implementing it in langchain_huggingface if it can solve your use-cases

[Tanmaypatil123]

Hey ! In InferecenClient, we do support grammar to have some output format : please see this link

I'm pretty sure it use outlines under the hood. I can work on implementing it in langchain_huggingface if it can solve your use-cases

yeah i guess it seems similar