Skip to content

Commit bf980bb

Browse files
CLoaKY233CLoaKY233
committed
Add detailed metadata and unify citation instructions in prompts
1 parent f3b21e0 commit bf980bb

File tree

6 files changed

+271
-54
lines changed

6 files changed

+271
-54
lines changed

‎prompts/evaluation/follow_up_generation.yaml‎

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,50 @@
1+
---
12
metadata:
23
name: "follow_up_generation"
3-
version: "2.0.0"
4-
description: "Generate precise follow-up search queries"
4+
version: "1.0.0"
5+
description: "Generate follow-up queries for reflexion cycles"
6+
author: "RAG Team"
7+
created_date: "2025-06-10"
8+
last_modified: "2025-06-10"
9+
tags: ["evaluation", "follow-up", "query-generation"]
10+
511
config:
6-
temperature: 0.0
7-
max_tokens: 100
12+
temperature: 0.5
13+
max_tokens: 500
14+
model_type: "evaluation"
15+
816
variables:
917
- name: "original_query"
1018
type: "string"
1119
required: true
20+
description: "Original user question"
21+
- name: "partial_answer"
22+
type: "string"
23+
required: true
24+
description: "Current partial answer"
1225
- name: "missing_aspects"
1326
type: "array"
1427
required: true
28+
description: "List of missing aspects to address"
29+
1530
prompt_template: |
16-
Produce 1–2 concise search queries to find missing info on:
17-
{{original_query}}
31+
Generate 1-2 specific follow-up queries to address missing information.
32+
33+
Original Question: {{original_query}}
34+
35+
Current Answer: {{partial_answer}}
36+
37+
Missing Aspects: {{missing_aspects}}
38+
39+
Requirements:
40+
- Create specific, searchable queries
41+
- Focus on the most important missing information
42+
- Make queries standalone (no pronouns)
43+
- Prioritize factual, retrievable information
44+
- Keep queries concise and focused
1845
19-
Missing:
20-
{{missing_aspects}}
46+
Format as numbered list:
47+
1. [First follow-up query]
48+
2. [Second follow-up query]
2149
22-
Do NOT introduce unrelated topics.
50+
Follow-up queries:

‎prompts/evaluation/response_evaluation.yaml‎

Lines changed: 56 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,18 @@
1+
---
12
metadata:
23
name: "response_evaluation"
3-
version: "2.0.0"
4-
description: "Evaluate answer quality and completeness"
4+
version: "1.0.0"
5+
description: "Evaluate response quality and completeness"
6+
author: "RAG Team"
7+
created_date: "2025-06-10"
8+
last_modified: "2025-06-10"
9+
tags: ["evaluation", "reflexion", "quality-control"]
10+
511
config:
6-
temperature: 0.0
7-
max_tokens: 300
12+
temperature: 0.3
13+
max_tokens: 1000
14+
model_type: "evaluation"
15+
816
variables:
917
- name: "query"
1018
type: "string"
@@ -15,16 +23,53 @@ variables:
1523
- name: "docs_summary"
1624
type: "string"
1725
required: true
26+
- name: "cycle_number"
27+
type: "integer"
28+
required: true
29+
- name: "confidence_threshold"
30+
type: "float"
31+
required: true
32+
1833
prompt_template: |
19-
Evaluate the answer below for question {{query}}:
34+
You are an expert evaluator assessing the quality and completeness of AI responses.
2035
21-
Answer:
36+
EVALUATION TASK:
37+
Assess if the following response sufficiently answers the user's question.
38+
39+
Original Question: {{query}}
40+
41+
Current Response (Cycle {{cycle_number}}):
2242
{{partial_answer}}
2343
24-
Context Summary:
25-
{{docs_summary}}
44+
Available Context: {{docs_summary}}
45+
46+
EVALUATION CRITERIA:
47+
1. Completeness: Does the response address all aspects of the question?
48+
2. Accuracy: Is the response supported by the available documents?
49+
3. Confidence: Does the response contain uncertain or vague language?
50+
4. Specificity: Are there specific sub-questions that need more detail?
51+
52+
RESPONSE FORMAT (JSON):
53+
{
54+
"confidence_score": 0.35,
55+
"decision": "continue|refine_query|complete|insufficient_data",
56+
"reasoning": "Detailed explanation of the assessment",
57+
"covered_aspects": ["aspect1", "aspect2"],
58+
"missing_aspects": ["missing1", "missing2"],
59+
"uncertainty_phrases": ["phrase1", "phrase2"],
60+
"specific_gaps": ["What specific details are missing?"]
61+
}
62+
63+
DECISION GUIDELINES:
64+
- confidence_score: 0.0-1.0 (how well the question is answered)
65+
- "complete": confidence >= {{confidence_threshold}} and no major gaps
66+
- "continue": confidence < {{confidence_threshold}} but retrievable information exists
67+
- "refine_query": need more specific queries for missing aspects
68+
- "insufficient_data": fundamental information is missing from knowledge base
2669
27-
Provide JSON with:
28-
{"confidence":0.0-1.0,"decision":"continue|complete|insufficient_data","reason":"brief"}
70+
INSTRUCTION:
71+
1. Be very strict in the process
72+
2. Always lower confidence on mistakes
73+
3. Ensure that you respond with a stricter and hard honest response so that application can improve it's replies.
2974
30-
- If unsure, choose “insufficient_data.”
75+
Provide your evaluation as valid JSON:

‎prompts/generation/initial_generation.yaml‎

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,67 @@
1+
---
12
metadata:
23
name: "initial_generation"
3-
version: "2.0.0"
4-
description: "First-cycle answer with strict source citations"
4+
version: "1.0.0"
5+
description: "Initial response generation in reflexion cycle"
6+
author: "RAG Team"
7+
created_date: "2025-06-10"
8+
last_modified: "2025-06-10"
9+
tags: ["generation", "reflexion", "initial"]
10+
511
config:
612
temperature: 0.7
7-
max_tokens: 2500
13+
max_tokens: 3000
14+
model_type: "generation"
15+
816
variables:
917
- name: "query"
1018
type: "string"
1119
required: true
20+
description: "User's question"
1221
- name: "context"
1322
type: "string"
1423
required: true
24+
description: "Retrieved document context"
25+
- name: "cycle_number"
26+
type: "integer"
27+
required: true
28+
description: "Current reflexion cycle number"
29+
1530
prompt_template: |
16-
You are an expert research assistant. Answer the question below using only the documents provided.
31+
You are an expert AI assistant providing detailed, accurate answers with proper source citations.
32+
33+
This is the initial response. Provide a comprehensive answer with proper source citations.
1734
1835
Question: {{query}}
1936
20-
Context:
37+
Available Documents:
2138
{{context}}
2239
23-
Instructions:
24-
- Base every fact on context; if unsupported, state “Not enough information.”
25-
- Cite each fact as [Source: filename.ext].
26-
- Be concise and organized.
40+
IMPORTANT: Multiple document entries may be from the SAME SOURCE FILE but different sections/chunks. When citing:
41+
- If multiple "Doc X" entries share the same filename, they are from the SAME document
42+
- Use the source filename as the primary citation reference
43+
- You can reference specific sections if needed, but treat same-named files as one source
44+
45+
CITATION REQUIREMENTS:
46+
- Use [Source: filename] format for inline citations (e.g., [Source: batman.md], [Source: interstellar.md])
47+
- When multiple chunks are from the same file, cite the filename once, not each chunk separately
48+
- Include creation dates when referencing information
49+
- At the end of your response, provide a "Sources" section with unique filenames only
50+
- If information comes from multiple sections of the same document, mention "multiple sections"
51+
- Be specific about which source file supports each claim
52+
53+
RESPONSE STRUCTURE:
54+
1. Provide a comprehensive answer with inline citations using source filenames
55+
2. Use clear, professional language
56+
3. Organize information logically with headers if needed
57+
4. Include a "Sources" section at the end with unique source files only
58+
59+
IMPORTANT GUIDELINES:
60+
- Base your response ONLY on the provided documents
61+
- Use inline citations [Source: filename] after each factual claim
62+
- Treat multiple chunks from the same file as ONE source document
63+
- If information is incomplete, clearly state what's missing and from which sources
64+
- Maintain professional tone throughout
65+
- In Sources section, list each unique filename only once with its full path
2766
2867
Answer:

‎prompts/generation/reflexion_generation.yaml‎

Lines changed: 46 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,18 @@
1+
---
12
metadata:
23
name: "reflexion_generation"
3-
version: "2.0.0"
4-
description: "Follow-up cycle focusing on missing aspects"
4+
version: "1.0.0"
5+
description: "Follow-up response generation in reflexion cycles"
6+
author: "RAG Team"
7+
created_date: "2025-06-10"
8+
last_modified: "2025-06-10"
9+
tags: ["generation", "reflexion", "follow-up"]
10+
511
config:
612
temperature: 0.7
7-
max_tokens: 2000
13+
max_tokens: 3000
14+
model_type: "generation"
15+
816
variables:
917
- name: "query"
1018
type: "string"
@@ -15,12 +23,42 @@ variables:
1523
- name: "cycle_number"
1624
type: "integer"
1725
required: true
26+
1827
prompt_template: |
19-
Cycle {{cycle_number}}: Refine the previous answer on {{query}} with missing details.
28+
You are an expert AI assistant providing detailed, accurate answers with proper source citations.
29+
30+
This is cycle {{cycle_number}} of a reflexion loop. Focus on addressing specific aspects that may have been missed, with accurate citations.
31+
32+
Question: {{query}}
2033
21-
Use only:
34+
Available Documents:
2235
{{context}}
2336
24-
Instructions:
25-
- Fill specific gaps; do NOT introduce new topics.
26-
- Cite every claim [Source: filename.ext], or say “No information available.”
37+
IMPORTANT: Multiple document entries may be from the SAME SOURCE FILE but different sections/chunks. When citing:
38+
- If multiple "Doc X" entries share the same filename, they are from the SAME document
39+
- Use the source filename as the primary citation reference
40+
- You can reference specific sections if needed, but treat same-named files as one source
41+
42+
CITATION REQUIREMENTS:
43+
- Use [Source: filename] format for inline citations (e.g., [Source: batman.md], [Source: interstellar.md])
44+
- When multiple chunks are from the same file, cite the filename once, not each chunk separately
45+
- Include creation dates when referencing information
46+
- At the end of your response, provide a "Sources" section with unique filenames only
47+
- If information comes from multiple sections of the same document, mention "multiple sections"
48+
- Be specific about which source file supports each claim
49+
50+
RESPONSE STRUCTURE:
51+
1. Provide a comprehensive answer with inline citations using source filenames
52+
2. Use clear, professional language
53+
3. Organize information logically with headers if needed
54+
4. Include a "Sources" section at the end with unique source files only
55+
56+
IMPORTANT GUIDELINES:
57+
- Base your response ONLY on the provided documents
58+
- Use inline citations [Source: filename] after each factual claim
59+
- Treat multiple chunks from the same file as ONE source document
60+
- If information is incomplete, clearly state what's missing and from which sources
61+
- Maintain professional tone throughout
62+
- In Sources section, list each unique filename only once with its full path
63+
64+
Answer:

‎prompts/generation/simple_generation.yaml‎

Lines changed: 36 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,53 @@
1+
---
12
metadata:
23
name: "simple_generation"
3-
version: "2.0.0"
4-
description: "Fallback single-pass RAG answer"
4+
version: "1.0.0"
5+
description: "Simple RAG generation without reflexion"
6+
author: "RAG Team"
7+
created_date: "2025-06-10"
8+
last_modified: "2025-06-10"
9+
tags: ["generation", "simple", "fallback"]
10+
511
config:
612
temperature: 0.7
7-
max_tokens: 2000
13+
max_tokens: 3000
14+
model_type: "generation"
15+
816
variables:
917
- name: "question"
1018
type: "string"
1119
required: true
20+
description: "User's question"
1221
- name: "context"
1322
type: "string"
1423
required: true
24+
description: "Retrieved document context"
25+
1526
prompt_template: |
16-
Answer the question using only the context below.
27+
You are an AI assistant that provides accurate answers with proper source citations.
1728
1829
Question: {{question}}
1930
20-
Context:
31+
Available Documents:
2132
{{context}}
2233
23-
- Cite each fact as [Source: filename.ext].
24-
- If you lack information, state “Insufficient data.”
34+
IMPORTANT: Multiple document entries may be from the SAME SOURCE FILE but different sections. When citing:
35+
- Use the source filename for citations, not individual document numbers
36+
- Multiple "Doc X" entries with the same filename are from the SAME document
37+
- Treat same-named files as one source in your response
38+
39+
INSTRUCTIONS:
40+
- Answer based ONLY on the provided documents
41+
- Use inline citations [Source: filename] after each factual statement
42+
- Include creation dates when referencing information
43+
- Be comprehensive and accurate
44+
- If context is insufficient, state what information is missing
45+
- End with a "Sources" section listing unique source files only
46+
47+
CITATION FORMAT:
48+
- Use [Source: filename] for inline citations (NOT [Doc X])
49+
- Reference specific source files that support each claim
50+
- Include file creation dates in your references
51+
- List each unique source file only once in Sources section
52+
53+
Answer with proper source-based citations:

0 commit comments

Comments
 (0)