What is the proper way to implement follow-up queries natively?

[rosegarden-coder]

PR #911 exposes the prior answer for enabling asking follow up queries.

I wonder what is the proper way to implement when running LLMs locally? (PQASession and PaperQAEnvironment, perhaps?)

[dosubot[bot]]

To implement native follow-up queries—especially when running LLMs locally—use the PQASession and PaperQAEnvironment abstractions to manage session state and context. These classes track and expose prior answers and contexts, enabling seamless chaining of queries without relying on remote API state.

A recent update (see PR #1032) introduced the ability to group contexts by question, which is particularly useful for follow-up queries. You can enable this by setting answer.group_contexts_by_question to True in your AnswerSettings. This causes the answer context to be organized by the originating question, making it easier for the model to reference prior answers and context in subsequent queries. The test_aquery_groups_contexts_by_question demonstrates this pattern in practice, showing how PQASession can be used to tag and group contexts for follow-up queries. This approach works the same way for both local and remote LLMs, as the session and environment abstractions handle context management internally, regardless of where the model is hosted.

For local LLMs, ensure your environment is configured to use a local backend (like ollama), and then use PQASession to persist and chain context and answers as needed for your workflow. For more details, see the documentation and code changes in PR #1032.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[rosegarden-coder]

@dosu can you provide some code snippet or examples on how to implement follow--up queries?

[dosubot[bot]]

Here's a minimal example of how to implement follow-up queries using PQASession with a local LLM in paper-qa. The key is to pass a PQASession containing the prior question and answer to the query method, so the model can build on previous responses:

from paperqa.types import PQASession
from paperqa.docs import Docs
from paperqa.settings import Settings

# Assume you have a Docs object with your documents loaded
docs = Docs()
settings = Settings()
llm = settings.get_llm()

# First, answer an initial question
question1 = "Is XAI usable in chemistry?"
answer1 = "No, it isn't usable in chemistry."
prior_session = PQASession(question=question1, answer=answer1)

# Now, ask a follow-up query using the prior session
result = await docs.aquery(prior_session, llm_model=llm, settings=settings)
print(result.answer)  # The model's answer, with prior context included

This pattern ensures the prior answer is included in the prompt for the follow-up query, allowing the LLM to reference earlier context. You can see this approach in the test code for follow-up queries (source).

For more complex workflows or agentic flows, use PaperQAEnvironment to manage session state and chain queries. It initializes with a query and settings, and maintains a PQASession internally for context (source).

If you want to group contexts by question (helpful for multi-turn Q&A), set answer.group_contexts_by_question = True in your settings. This organizes the context in the prompt by originating question, making it easier for the model to reference prior answers.

Let me know if you want a more detailed example or have a specific workflow in mind!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}