Document how one can provide a system prompt in RetrieveAndGenerateCommand.

[sandrooco]

Describe the issue

I'm using the @aws-sdk/client-bedrock-agent-runtime library to try building a RAG system. I'd like to provide a system prompt to provide some context for the model. The docs from RetrieveAndGenerateCommand don't provide any information about system prompts. How to do this?

Example code:

// This code works, but where can I put the system prompt?
const input = {
        sessionId: requestSessionId,
        input: {
          text: question,
        },
        retrieveAndGenerateConfiguration: {
          type: "KNOWLEDGE_BASE",
          knowledgeBaseConfiguration: {
            knowledgeBaseId: process.env.KNOWLEDGE_BASE_ID,
            modelArn: `arn:aws:bedrock:eu-central-2::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0`,
          },
        },
      };
const command = new RetrieveAndGenerateCommand(input);
const response = await client.send(command);

Links

Docs: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/bedrock-agent-runtime/command/RetrieveAndGenerateCommand/
Code is based on this example repo: https://github.com/aws-samples/amazon-bedrock-rag

[aBurmeseDev]

Hi @sandrooco - thanks for checking in.

The system prompt or context setting is typically handled through the Knowledge Base configuration in the AWS Console rather than in the API calls directly.

You can set these contextual elements by: going to the Amazon Q Knowledge Base settings in the AWS Console, under "Additional settings" or "Prompt configuration", you can configure:

• Knowledge Base instructions
• Additional context
• Custom prompts
  If you specifically need to include additional context for a particular query, you can add it to the text field along with your question. For example:

const input = {
  sessionId: requestSessionId,
  input: {
    text: `Context: [Your context here]
    
Question: ${question}`,
  },
  retrieveAndGenerateConfiguration: {
    type: "KNOWLEDGE_BASE",
    knowledgeBaseConfiguration: {
      knowledgeBaseId: process.env.KNOWLEDGE_BASE_ID,
      modelArn: `ModelARN`,
    },
  },
};

According to service API docs, it's recommended to use the Knowledge Base configuration in the AWS Console as this will apply consistently across all queries to that Knowledge Base.

Queries a knowledge base and generates responses based on the retrieved results and using the specified foundation model or inference profile. The response only cites sources that are relevant to the query.

Hope that helps!
Best,
John

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.