Bedrock Retriever / Synthesizer error

[brandon-orkestus]

I'm following the example here https://docs.llamaindex.ai/en/stable/examples/retrievers/bedrock_retriever/

This is my code

LLM_NAME = "anthropic.claude-3-sonnet-20240229-v1:0"
NUM_RESULTS = 10
MAX_TOKENS = 4_096
retriever = AmazonKnowledgeBasesRetriever(
            knowledge_base_id=aws_knowledge_base_id,
            retrieval_config={"vectorSearchConfiguration": {"numberOfResults": num_results}},
        )
retrieved_results = retriever.retrieve(query)

llm = Bedrock(model=model, temperature=temperature, max_tokens=MAX_TOKENS)
response_synthesizer = get_response_synthesizer(response_mode=ResponseMode.COMPACT, llm=llm, verbose=True)
response = str(response_synthesizer.synthesize(query, retrieved_results))

The last line gives me this error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/base.py", line 241, in synthesize
    response_str = self.get_response(
                   ^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py", line 43, in get_response
    return super().get_response(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/refine.py", line 172, in get_response
    response = self._give_response_single(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/refine.py", line 227, in _give_response_single
    program(
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/refine.py", line 79, in __call__
    answer = self._llm.predict(
             ^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/llms/llm.py", line 575, in predict
    chat_response = self.chat(messages)
                    ^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/llms/callbacks.py", line 172, in wrapped_llm_chat
    f_return_val = f(_self, messages, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/llms/bedrock/base.py", line 294, in chat
    completion_response = self.complete(prompt, formatted=True, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/llms/callbacks.py", line 426, in wrapped_llm_predict
    f_return_val = f(_self, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/llms/bedrock/base.py", line 236, in complete
    request_body = self._provider.get_request_body(prompt, all_kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/llms/bedrock/utils.py", line 157, in get_request_body
    if len(prompt) > 0 and prompt[0]["role"] == "system":
                           ~~~~~~~~~^^^^^^^^
TypeError: string indices must be integers, not 'str'

Does anybody else facing this issue?

Im using this dependencies

llama-index = "^0.11.1"
llama-index-retrievers-bedrock = "^0.2.0"
llama-index-llms-bedrock = "^0.2.0"

[dosubot[bot]]

Hey there, @brandon-orkestus! I'm here to help you out with any bugs or questions you have while waiting for a human maintainer. If you need assistance, feel free to ask.

The error you're encountering, TypeError: string indices must be integers, not 'str', indicates that the prompt being passed to the complete method is expected to be a list of dictionaries, but it is currently a string. To resolve this, you need to format the query correctly before passing it to the synthesize method.

Here's how you can modify your code to ensure the query is formatted correctly:

1. Wrap your query in a list of dictionaries with the appropriate roles.
2. Pass the formatted query to the synthesize method.

Here's the updated code:

LLM_NAME = "anthropic.claude-3-sonnet-20240229-v1:0"
NUM_RESULTS = 10
MAX_TOKENS = 4_096

retriever = AmazonKnowledgeBasesRetriever(
    knowledge_base_id=aws_knowledge_base_id,
    retrieval_config={"vectorSearchConfiguration": {"numberOfResults": NUM_RESULTS}},
)

retrieved_results = retriever.retrieve(query)

llm = Bedrock(model=LLM_NAME, temperature=temperature, max_tokens=MAX_TOKENS)
response_synthesizer = get_response_synthesizer(response_mode=ResponseMode.COMPACT, llm=llm, verbose=True)

# Format the query as a list of dictionaries
formatted_query = [{"role": "user", "content": query}]

response = str(response_synthesizer.synthesize(formatted_query, retrieved_results))

This should resolve the TypeError by ensuring the query is in the expected format when passed to the complete method of the Bedrock class ^[1].

To continue talking to Dosu, mention @dosu.

[brandon-orkestus]

The synthethize method receives a string, here is the traceback with your proposed solution

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 261, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.12/site-packages/llama_index/core/response_synthesizers/base.py", line 207, in synthesize
    SynthesizeStartEvent(
  File ".venv/lib/python3.12/site-packages/pydantic/main.py", line 175, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for SynthesizeStartEvent
query.str
  Input should be a valid string [type=string_type, input_value=[{'role': 'user', 'conten...rmas mas importantes?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.7/v/string_type
query.QueryBundle
  Input should be a dictionary or an instance of QueryBundle [type=dataclass_type, input_value=[{'role': 'user', 'conten...rmas mas importantes?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.7/v/dataclass_type