Prevent duplicate quiz questions by removing duplicates and adding history - #281
Conversation
…e_context and adding generation history Agent-Logs-Url: https://github.com/Leadernelson/DeepTutor/sessions/98d69209-ee73-4cd7-832f-eac0650b097e Co-authored-by: Leadernelson <184500545+Leadernelson@users.noreply.github.com>
…tions fix: prevent duplicate quiz questions caused by duplicated knowledge_context and missing generation history
There was a problem hiding this comment.
Pull request overview
This PR updates the quiz question generation pipeline to reduce repeated/near-duplicate questions within a single generation session by adding “previously generated questions” history into the prompts and coordinator logic.
Changes:
- Extend EN/ZH generator prompt templates to include
{history_context}as a “previously generated questions” block and explicitly forbid duplicates. - Accumulate generated question texts during
_generation_loopand pass them into the generator as part ofhistory_context. - Reduce prompt bloat by serializing
QuestionTemplatewithoutmetadata.knowledge_context(since knowledge context is already injected via{knowledge_context}).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
deeptutor/agents/question/prompts/zh/generator.yaml |
Adds a session history block and anti-duplicate instruction to the ZH generator prompt. |
deeptutor/agents/question/prompts/en/generator.yaml |
Adds a session history block and anti-duplicate instruction to the EN generator prompt. |
deeptutor/agents/question/coordinator.py |
Tracks generated questions and appends them into the context passed to the generator. |
deeptutor/agents/question/agents/generator.py |
Reframes history_context as “previously generated questions” and strips knowledge_context from template JSON serialization to avoid duplication. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "Template: {template}\n" | ||
| "User topic: {user_topic}\n" | ||
| "Preference: {preference}\n" | ||
| "Conversation context: {history_context}\n" | ||
| "Previously generated questions (do not repeat):\n{history_context}\n" | ||
| "Knowledge context: {knowledge_context}\n" |
There was a problem hiding this comment.
history_context is used upstream as general conversation context (e.g., context.metadata['conversation_context_text'] in deeptutor/capabilities/deep_question.py), but this prompt text now frames it as “Previously generated questions (do not repeat)”. With the current call chain, that means the model will be instructed not to be similar to the conversation as well, which is a semantics mismatch and can hurt quality.
Consider introducing a dedicated placeholder/parameter for “previously generated questions” (e.g., generated_questions_context) and keep history_context for conversation context, or update all callers so history_context contains only prior generated questions.
| # Build cumulative context from questions already generated in this | ||
| # session so the Generator can avoid producing duplicates. | ||
| cumulative_history = history_context | ||
| if generated_questions: | ||
| previous = "\n".join( | ||
| f"Q{i}: {q}" for i, q in enumerate(generated_questions, start=1) | ||
| ) | ||
| if cumulative_history: | ||
| cumulative_history = f"{cumulative_history}\n\nQuestions generated in this session:\n{previous}" | ||
| else: | ||
| cumulative_history = f"Questions generated in this session:\n{previous}" |
There was a problem hiding this comment.
This cumulative history grows unbounded with num_questions (and API callers don’t appear to enforce a strict upper limit), which can quickly bloat prompts and exceed model context limits / increase cost. Consider clipping this to the most recent N questions and/or enforcing a character/token budget when building cumulative_history.
| # Build cumulative context from questions already generated in this | ||
| # session so the Generator can avoid producing duplicates. | ||
| cumulative_history = history_context | ||
| if generated_questions: | ||
| previous = "\n".join( | ||
| f"Q{i}: {q}" for i, q in enumerate(generated_questions, start=1) | ||
| ) | ||
| if cumulative_history: | ||
| cumulative_history = f"{cumulative_history}\n\nQuestions generated in this session:\n{previous}" | ||
| else: | ||
| cumulative_history = f"Questions generated in this session:\n{previous}" | ||
|
|
There was a problem hiding this comment.
cumulative_history is constructed with an extra English header (“Questions generated in this session: …”) even though the generator prompt templates already label {history_context}, and the coordinator runs in multiple languages. This creates redundant/awkward prompt text (and mixes languages for zh). Consider passing just the enumerated questions (or a structured list) and letting the language-specific prompt template provide the surrounding label.
| # Serialize the template without the bulky knowledge_context in metadata — | ||
| # it is already included via the dedicated {knowledge_context} placeholder | ||
| # and its presence inside the template JSON would cause it to appear twice, | ||
| # dominating the prompt and making the LLM generate identical questions for | ||
| # every template regardless of their individual concentrations. | ||
| template_dict = template.__dict__.copy() | ||
| if isinstance(template_dict.get("metadata"), dict): | ||
| stripped_metadata = { | ||
| k: v | ||
| for k, v in template_dict["metadata"].items() | ||
| if k != "knowledge_context" | ||
| } | ||
| template_dict["metadata"] = stripped_metadata | ||
|
|
There was a problem hiding this comment.
The template serialization strips metadata.knowledge_context to avoid duplicating large context in the generation prompt, but the repair path still embeds json.dumps(template.__dict__) (which includes metadata.knowledge_context) while also separately appending Knowledge context:. This reintroduces the same prompt bloat risk during repair. Consider reusing the same stripped template_dict for repair prompts as well.
|
Yeah! Thanks for that! |
…neration Fixes semantic overloading introduced by PR #281 — history_context (conversation history) is no longer mixed with the generated-questions dedup list. - Add dedicated `previous_questions` parameter through Generator pipeline - Extract `_strip_template_knowledge_context()` reused by both generate and repair paths - Add `_format_previous_questions()` with MAX_PREVIOUS_QUESTIONS=20 cap - Move language labels into YAML templates (en/zh) to avoid language mixing - Simplify coordinator to pass raw list instead of string concatenation Made-with: Cursor
Prevent duplicate quiz questions by removing duplicates and adding history
…neration Fixes semantic overloading introduced by PR HKUDS#281 — history_context (conversation history) is no longer mixed with the generated-questions dedup list. - Add dedicated `previous_questions` parameter through Generator pipeline - Extract `_strip_template_knowledge_context()` reused by both generate and repair paths - Add `_format_previous_questions()` with MAX_PREVIOUS_QUESTIONS=20 cap - Move language labels into YAML templates (en/zh) to avoid language mixing - Simplify coordinator to pass raw list instead of string concatenation Made-with: Cursor
Description
This pull request enhances the question generation process to ensure diversity by preventing duplicate or highly similar questions from being generated within the same session. It achieves this by tracking previously generated questions and updating both the prompt templates and the generator logic to reference this history.
Key changes:
generator.yaml) and Chinese (generator.yaml) prompt templates to explicitly instruct the model not to generate questions that are identical or nearly identical to any previously generated questions in the session, using the{history_context}placeholder.history_context.generator.pyto clarify the meaning ofhistory_contextand to serialize the template without duplicating the bulkyknowledge_contextin the metadata, preventing prompt bloat and improving question diversity.Related Issues
Module(s) Affected
coreknowledgeagentsapiconfigloggingservicestoolsutilsweb(Frontend)docs(Documentation)scriptstests...Checklist
pre-commit run --all-filesand fixed any issues.Additional Notes
This improvement ensures better question variety across multi-turn tutoring sessions without increasing prompt size or complexity. No breaking changes; backward compatibility is fully maintained.