Update: [AEA-5919] - Split reformulation and rag prompts, ai model and add citation buttons (#202)

## Summary

Updates prompt and message structure for easier understanding and
providing context

### Details

- Updated formatting prompting to be shorter and stricter
- Increase max tokens to 1500, to account for citation information
- Stricter handling of data relevance to improve bot confidence
- Add Citation buttons for easy viewing of extracted data
- Stricter handling of citation formatting to allow lambda to extract
data
- Remove breakdown steps
   - Reduces token usage
   - Form formatting creates "natural" path of creation

---------

Co-authored-by: Bence Gadanyi <[email protected]>

Author: kieran-wilkinson-4

packages/cdk/prompts/systemPrompt.txt

@@ -1,47 +1,49 @@
-<SystemInstructions>
-  You are an AI assistant designed to provide helpful information and guidance related to healthcare systems,
-  data integration and user setup.
-  
-  <Requirements>
-    1. Break down the question(s) based on the context
-    2. Examine the information provided in the question(s) or requirement(s).
-    3. Refer to your knowledge base to find relevant details, specifications, and useful references/ links.
-    4. The knowledge base is your source of truth before anything else
-    5. Acknowledge explicit and implicit evidence
-       5a. If no explicit evidence is available, state implicit evidence with a caveat
-    6. Provide critical thinking before replying to make the direction actionable and authoritative
-    7. Provide a clear and comprehensive answer by drawing inferences,
-     making logical connections from the available information, comparing previous messages,
-      and providing users with link and/ or references to follow.
-    8. Be clear in answers, direct actions are preferred (eg., "Check Postcode" &gt; "Refer to documentation") 
-  </Requirements>
-  
-  <Constraints>
-    1. Quotes should be italic
-    2. Document titles and document section names should be bold
-    3. If there is a single question, or the user is asking for direction, do not list items
-    4. If the query has multiple questions *and* the answer includes multiple answers for multiple questions
-     (as lists or bullet), the list items must be formatted as \`*<question>*
-     - <answer(s)>\`.
-      4a. If there are multiple questions in the query, shorten the question to less than 50 characters
-  </Constraints>
-  
-  <Output>
-    - Use Markdown, avoid XML
-    - Structured, informative, and tailored to the specific context of the question. 
-    - Provide evidence to support results
-    - Acknowledging any assumptions or limitations in your knowledge or understanding.
-    - Text structure should be in Markdown
-  </Output>
-  
-  <Tone> 
-    Professional, helpful, authoritative.
-  </Tone>
-  
-  <Examples>
-    <Example1>
-      Q: Should alerts be automated?
-      A: *Section 1.14.1* mentions handling rejected prescriptions, which implies automation.
-    </Example1>
-  </Examples>
-</SystemInstructions>
+You are an AI assistant designed to provide guidance and references from your knowledge base to help users make decisions when onboarding. It is *VERY* important you return *ALL* references, for user examination. 
+
+# Response
+## Response Structure
+- *Summary*: 100 characters maximum, capturing core answer
+- *Answer* (use "mrkdown") (< 800 characters)
+- Page break (use `------`)
+- \[Bibliography\]
+
+## Formatting ("mrkdwn")
+  a. *Bold* for:
+    - Headings, subheadings: *Answer:*, *Bibliography:*
+    - Source names: *NHS England*, *EPS*
+  b. _Italic_ for:
+    - Citations, references, document titles
+  c. Block Quotes for:
+    - Direct quotes >1 sentence
+    - Technical specifications, parameters
+    - Examples
+  d. `Inline code` for:
+    - System names, field names: `PrescriptionID`
+    - Short technical terms: `HL7 FHIR`
+  e. Links:
+    - Do not provide links
+
+# Thinking
+## Question Handling
+- Detect whether the query contains one or multiple questions
+- Split complex queries into individual sub-questions
+- Identify question type: factual, procedural, diagnostic, troubleshooting, or clarification-seeking
+- For multi-question queries: number sub-questions clearly (Q1, Q2, etc)
+
+## RAG & Knowledge Base Integration
+- Relevance threshold handling:
+  - Score > 0.85 (High confidence)
+  - Score 0.70 - 0.85 (Medium confidence)
+  - Score < 0.70 (Low confidence)
+
+## Corrections
+- Change _National Health Service Digital (NHSD)_ references to _National Health Service England (NHSE)_
+
+# Bibliography
+## Format
+<cit>source number||summary title||link||filename||text snippet||reasoning</cit>\n
+
+## Requirements
+- Return **ALL** retrieved documents, their name and a text snippet, from "CONTEXT"
+- Get full text references from search results for Bibliography
+- Title should be less than 50 characters

packages/cdk/prompts/userPrompt.txt

@@ -1,7 +1,6 @@
-- Using your knowledge around the National Health Service (NHS), Electronic Prescription Service (EPS) and the Fast Healthcare Interoperability Resources' (FHIR) onboarding, Supplier Conformance Assessment List (SCAL), APIs, developer guides and error resolution; please answer the following question and cite direct quotes and document sections.
-- If my query is asking for instructions (i.e., "How to...", "How do I...") provide step by steps instructions
-- Do not provide general advice or external instructions
+# QUERY
+{{user_query}}
 
-<SearchResults>$search_results$</SearchResults>
-
-<UserQuery>{{user_query}}</UserQuery>`
+# CONTEXT
+## Results $search_results$
+## LIST ALL RESULTS IN TABLE

packages/cdk/resources/BedrockPromptResources.ts

@@ -18,12 +18,14 @@ export class BedrockPromptResources extends Construct {
   constructor(scope: Construct, id: string, props: BedrockPromptResourcesProps) {
     super(scope, id)
 
-    const claudeHaikuModel = BedrockFoundationModel.ANTHROPIC_CLAUDE_HAIKU_V1_0
-    const claudeSonnetModel = BedrockFoundationModel.ANTHROPIC_CLAUDE_SONNET_V1_0
+    // Nova Pro is recommended for text generation tasks requiring high accuracy and complex understanding.
+    const novaProModel = BedrockFoundationModel.AMAZON_NOVA_PRO_V1
+    // Nova Lite is recommended for tasks
+    const novaLiteModel = BedrockFoundationModel.AMAZON_NOVA_LITE_V1
 
     const queryReformulationPromptVariant = PromptVariant.text({
       variantName: "default",
-      model: claudeHaikuModel,
+      model: novaLiteModel,
       promptVariables: ["topic"],
       promptText: props.settings.reformulationPrompt.text
     })
@@ -37,7 +39,7 @@ export class BedrockPromptResources extends Construct {
 
     const ragResponsePromptVariant = PromptVariant.chat({
       variantName: "default",
-      model: claudeSonnetModel,
+      model: novaProModel,
       promptVariables: ["query", "search_results"],
       system: props.settings.systemPrompt.text,
       messages: [props.settings.userPrompt]

packages/cdk/resources/BedrockPromptSettings.ts

@@ -35,7 +35,7 @@ export class BedrockPromptSettings extends Construct {
     this.inferenceConfig = {
       temperature: 0,
       topP: 1,
-      maxTokens: 512,
+      maxTokens: 1500,
       stopSequences: [
         "Human:"
       ]

packages/slackBotFunction/app/services/ai_processor.py

@@ -21,6 +21,11 @@ def process_ai_query(user_query: str, session_id: str | None = None) -> AIProces
     # session_id enables conversation continuity across multiple queries
     kb_response = query_bedrock(reformulated_query, session_id)
 
+    logger.info(
+        "response from bedrock",
+        extra={"response_text": kb_response},
+    )
+
     return {
         "text": kb_response["output"]["text"],
         "session_id": kb_response.get("sessionId"),

packages/slackBotFunction/app/services/bedrock.py

@@ -25,7 +25,7 @@ def query_bedrock(user_query: str, session_id: str = None) -> RetrieveAndGenerat
     inference_config = prompt_template.get("inference_config")
 
     if not inference_config:
-        default_values = {"temperature": 0, "maxTokens": 512, "topP": 1}
+        default_values = {"temperature": 0, "maxTokens": 1500, "topP": 1}
         inference_config = default_values
         logger.warning(
             "No inference configuration found in prompt template; using default values",
@@ -43,6 +43,7 @@ def query_bedrock(user_query: str, session_id: str = None) -> RetrieveAndGenerat
             "knowledgeBaseConfiguration": {
                 "knowledgeBaseId": config.KNOWLEDGEBASE_ID,
                 "modelArn": config.RAG_MODEL_ID,
+                "retrievalConfiguration": {"vectorSearchConfiguration": {"numberOfResults": 5}},
                 "generationConfiguration": {
                     "guardrailConfiguration": {
                         "guardrailId": config.GUARD_RAIL_ID,
@@ -57,6 +58,16 @@ def query_bedrock(user_query: str, session_id: str = None) -> RetrieveAndGenerat
                         }
                     },
                 },
+                "orchestrationConfiguration": {
+                    "inferenceConfig": {
+                        "textInferenceConfig": {
+                            **inference_config,
+                            "stopSequences": [
+                                "Human:",
+                            ],
+                        }
+                    },
+                },
             },
         },
     }
@@ -79,7 +90,7 @@ def query_bedrock(user_query: str, session_id: str = None) -> RetrieveAndGenerat
     response = client.retrieve_and_generate(**request_params)
     logger.info(
         "Got Bedrock response",
-        extra={"session_id": response.get("sessionId"), "has_citations": len(response.get("citations", [])) > 0},
+        extra={"session_id": response.get("sessionId")},
     )
     return response
 

packages/slackBotFunction/app/services/prompt_loader.py

@@ -106,7 +106,7 @@ def load_prompt(prompt_name: str, prompt_version: str = None) -> dict:
         actual_version = response.get("version", "DRAFT")
 
         # Extract inference configuration with defaults
-        default_inference = {"temperature": 0, "topP": 1, "maxTokens": 512}
+        default_inference = {"temperature": 0, "topP": 1, "maxTokens": 1500}
         raw_inference = response["variants"][0].get("inferenceConfiguration", {})
         raw_text_config = raw_inference.get("textInferenceConfiguration", {})
         inference_config = {**default_inference, **raw_text_config}