Return answer with sources using LCEL

[alice17]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

chain = (
            {"context": retriever, "question": RunnablePassthrough()}
            | RunnablePassthrough.assign(sources=(lambda x: [x.metadata["source"] for x in x["context"]]))
            | prompt
            | llm
            | StrOutputParser()
    )

    add_routes(app, chain)

Description

I would like to parse an answer on the playground with this format:

Hi this is a test answer.

Sources:
- www.example.com/test
- www.example.com/documentation

I was able to extract the links through RunnablePassthrough somewhere in the chain but I can't concatenate the list of links to the answer.
Is it possible? I find LCEL extremely complicated. I add that I would also like to stream the answer to the playground.

System Info

langchain==0.2.7
langchain-community==0.2.7
langchain-core==0.2.32
langchain-huggingface==0.0.3
langchain-ollama==0.1.0
langchain-openai==0.1.14
langchain-text-splitters==0.2.2

[dosubot[bot]]

Hey @alice17! I'm here to help you with your LangChain issue. I'm a bot designed to assist with bug fixes, answer questions, and guide you on becoming a contributor. Let's tackle this together!

To return an answer with sources using LCEL and stream it to the playground, you can follow these steps:

1. Set up the environment and dependencies:

   %pip install --upgrade --quiet langchain langchain-community langchainhub langchain-openai langchain-chroma beautifulsoup4

   Set the environment variables:

   import getpass
   import os
   
   os.environ["OPENAI_API_KEY"] = getpass.getpass()

2. Load, chunk, and index the contents of the blog:

   import bs4
   from langchain.chains import create_retrieval_chain
   from langchain.chains.combine_documents import create_stuff_documents_chain
   from langchain_chroma import Chroma
   from langchain_community.document_loaders import WebBaseLoader
   from langchain_core.prompts import ChatPromptTemplate
   from langchain_openai import OpenAIEmbeddings
   from langchain_text_splitters import RecursiveCharacterTextSplitter
   
   # Load, chunk and index the contents of the blog to create a retriever.
   loader = WebBaseLoader(
       web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
       bs_kwargs=dict(
           parse_only=bs4.SoupStrainer(
               class_=("post-content", "post-title", "post-header")
           )
       ),
   )
   docs = loader.load()
   
   text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
   splits = text_splitter.split_documents(docs)
   vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())
   retriever = vectorstore.as_retriever()
   
   # Incorporate the retriever into a question-answering chain.
   system_prompt = (
       "You are an assistant for question-answering tasks. "
       "Use the following pieces of retrieved context to answer "
       "the question. If you don't know the answer, say that you "
       "don't know. Use three sentences maximum and keep the "
       "answer concise."
       "\n\n"
       "{context}"
   )
   
   prompt = ChatPromptTemplate.from_messages(
       [
           ("system", system_prompt),
           ("human", "{input}"),
       ]
   )
   
   question_answer_chain = create_stuff_documents_chain(llm, prompt)
   rag_chain = create_retrieval_chain(retriever, question_answer_chain)

3. Stream the final outputs:

   for chunk in rag_chain.stream("What is Task Decomposition"):
       print(chunk)

4. Compile the stream as it's being returned:

   output = {}
   curr_key = None
   for chunk in rag_chain.stream("What is Task Decomposition"):
       for key in chunk:
           if key not in output:
               output[key] = chunk[key]
           else:
               output[key] += chunk[key]
           if key != curr_key:
               print(f"\n\n{key}: {chunk[key]}", end="", flush=True)
           else:
               print(chunk[key], end="", flush=True)
           curr_key = key
   output

5. Use Output Parsers to Structure the Response:

   from langchain_core.output_parsers import PydanticOutputParser
   from langchain_core.prompts import PromptTemplate
   from langchain_core.pydantic_v1 import BaseModel, Field, validator
   from langchain_openai import OpenAI
   
   model = OpenAI(model_name="gpt-3.5-turbo-instruct", temperature=0.0)
   
   # Define your desired data structure.
   class AnswerWithSources(BaseModel):
       answer: str = Field(description="The answer to the question")
       sources: str = Field(description="Sources for the answer")
   
   # Set up a parser + inject instructions into the prompt template.
   parser = PydanticOutputParser(pydantic_object=AnswerWithSources)
   
   prompt = PromptTemplate(
       template="Answer the user query.\n{format_instructions}\n{query}\n",
       input_variables=["query"],
       partial_variables={"format_instructions": parser.get_format_instructions()},
   )
   
   # And a query intended to prompt a language model to populate the data structure.
   prompt_and_model = prompt | model
   output = prompt_and_model.invoke({"query": "What is Task Decomposition?"})
   structured_output = parser.invoke(output)
   print(structured_output)

6. Using LCEL for Streaming:

   from langchain_core.output_parsers import StrOutputParser
   from langchain_core.prompts import ChatPromptTemplate
   
   prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
   parser = StrOutputParser()
   chain = prompt | model | parser
   
   async for chunk in chain.astream({"topic": "parrot"}):
       print(chunk, end="|", flush=True)

This code will parse and stream the answer with sources to the playground, showing the intermediate steps and final output, and use output parsers to structure the response. The LCEL chain ensures that streaming works seamlessly, even when using parsers at the end of the chain ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[1nterstellar-JD]

Hello, have you solved it? Can you tell me the solution?

[alice17]

No, I haven't solved it yet.

[1nterstellar-JD]

I found a compromise solution

from langchain_core.runnables import RunnableParallel

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

rag_chain_from_docs = (
    RunnablePassthrough.assign(context=(lambda x: format_docs(x["context"])))
    | QA_CHAIN_PROMPT_2
    | sql_llm
    | StrOutputParser()
)

rag_chain_with_source = RunnableParallel(
    {"context": documents_retriever, "question": RunnablePassthrough()}
).assign(answer=rag_chain_from_docs)

result = rag_chain_with_source.invoke("example query")

This code outputs a json result,

We can then format this json result and output it as expected
This is provided that metadata has been entered when building the vector database

Hi this is a test answer.

Sources:
- www.example.com/test
- www.example.com/documentation