Prompt Engineering for RAG via Context Transformations - PII Example

[mraguth]

I am using code given in the example: Context Transformations - PII Example
from the web page: https://docs.llamaindex.ai/en/stable/examples/prompts/prompts_rag/

I am getting lot better response, however I am also getting this warning. How I rectify that error? please advise:

Some weights of the model checkpoint at dbmdz/bert-large-cased-finetuned-conll03-english were not used when initializing BertForTokenClassification: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
C:\Users\a024377\AppData\Roaming\Python\Python311\site-packages\transformers\pipelines\token_classification.py:168: UserWarning: `grouped_entities` is deprecated and will be removed in version v5.0.0, defaulted to `aggregation_strategy="AggregationStrategy.SIMPLE"` instead.
  warnings.warn(
No model was supplied, defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
Some weights of the model checkpoint at dbmdz/bert-large-cased-finetuned-conll03-english were not used when initializing BertForTokenClassification: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
No model was supplied, defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
Some weights of the model checkpoint at dbmdz/bert-large-cased-finetuned-conll03-english were not used when initializing BertForTokenClassification: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
C:\Users\a024377\AppData\Roaming\Python\Python311\site-packages\transformers\pipelines\token_classification.py:168: UserWarning: `grouped_entities` is deprecated and will be removed in version v5.0.0, defaulted to `aggregation_strategy="AggregationStrategy.SIMPLE"` instead.
  warnings.warn(
No model was supplied, defaulted to dbmdz/bert-large-cased-finetuned-conll03-english and revision f2482bf (https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english).
Using a pipeline without specifying a model name and revision in production is not recommended.

[dosubot[bot]]

To rectify the warning messages you're encountering, you can take the following steps:

1. Specify the Model Name and Revision: Explicitly specify the model name and revision when creating the pipeline to avoid warnings about defaulting to a model.

2. Handle Deprecation of grouped_entities: Use the aggregation_strategy parameter instead of grouped_entities to avoid deprecation warnings.

Here is an updated version of your code snippet:

from transformers import pipeline

# Specify the model name and revision
model_name = "dbmdz/bert-large-cased-finetuned-conll03-english"
revision = "f2482bf"

# Create the NER pipeline with the specified model and revision
ner = pipeline("ner", model=model_name, revision=revision, aggregation_strategy="simple")

# Example text
text = """
Hello Paulo Santos. The latest statement for your credit card account \
1111-0000-1111-0000 was mailed to 123 Any Street, Seattle, WA 98109.
"""

# Process the text
response = ner(text)
print(response)

This should help in addressing the warnings related to the model checkpoint and the deprecation of grouped_entities ^[1][2].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu. Thank you and it did work. However, I like to know some additional details on 'Context Transformations - PII Example from this link... https://docs.llamaindex.ai/en/stable/examples/prompts/prompts_rag/

Can you expalin what this code really doing for prompt? I am sure I am getting good response, but I like to know under the hood what it is doing for the prompt. The combined code in this example is given below....

from llama_index.core.postprocessor import (
    NERPIINodePostprocessor,
    SentenceEmbeddingOptimizer,
)
from llama_index.core import QueryBundle
from llama_index.core.schema import NodeWithScore, TextNode

pii_processor = NERPIINodePostprocessor(llm=gpt4_llm)

def filter_pii_fn(**kwargs):
    # run optimizer
    query_bundle = QueryBundle(query_str=kwargs["query_str"])

    new_nodes = pii_processor.postprocess_nodes(
        [NodeWithScore(node=TextNode(text=kwargs["context_str"]))],
        query_bundle=query_bundle,
    )
    new_node = new_nodes[0]
    return new_node.get_content()

qa_prompt_tmpl_str = (
    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Given the context information and not prior knowledge, "
    "answer the query.\n"
    "Query: {query_str}\n"
    "Answer: "
)
qa_prompt_tmpl = PromptTemplate(
    qa_prompt_tmpl_str, function_mappings={"context_str": filter_pii_fn}
)
query_engine.update_prompts(
    {"response_synthesizer:text_qa_template": qa_prompt_tmpl}
)
# take a look at the prompt
retrieved_nodes = vector_retriever.retrieve(query_str)
context_str = "\n\n".join([n.get_content() for n in retrieved_nodes])

print(qa_prompt_tmpl.format(query_str=query_str, context_str=context_str))

response = query_engine.query(query_str)
print(str(response))

[dosubot[bot]]

The provided code is designed to filter out personally identifiable information (PII) from context strings in a prompt for a question-answering (QA) system using the NERPIINodePostprocessor. Here's a detailed breakdown of what each part of the code does:

1. PII Processing:

   • The NERPIINodePostprocessor is initialized with a language model (gpt4_llm).
   • The filter_pii_fn function is defined to process nodes containing text and remove any PII before they are included in the prompt. It creates a QueryBundle from the input query string and processes the context string to return a sanitized version.

   pii_processor = NERPIINodePostprocessor(llm=gpt4_llm)
   
   def filter_pii_fn(**kwargs):
       query_bundle = QueryBundle(query_str=kwargs["query_str"])
       new_nodes = pii_processor.postprocess_nodes(
           [NodeWithScore(node=TextNode(text=kwargs["context_str"]))],
           query_bundle=query_bundle,
       )
       new_node = new_nodes[0]
       return new_node.get_content()

2. Prompt Template:

   • The qa_prompt_tmpl_str defines the structure of the prompt, which includes placeholders for the context and query.
   • The PromptTemplate is instantiated with this string and the filter_pii_fn for dynamic context processing.

   qa_prompt_tmpl_str = (
       "Context information is below.\n"
       "---------------------\n"
       "{context_str}\n"
       "---------------------\n"
       "Given the context information and not prior knowledge, "
       "answer the query.\n"
       "Query: {query_str}\n"
       "Answer: "
   )
   qa_prompt_tmpl = PromptTemplate(
       qa_prompt_tmpl_str, function_mappings={"context_str": filter_pii_fn}
   )

3. Updating the Query Engine:

   • The prompt template is registered with the query engine using query_engine.update_prompts, allowing it to be used in subsequent queries.

   query_engine.update_prompts(
       {"response_synthesizer:text_qa_template": qa_prompt_tmpl}
   )

4. Execution:

   • The context is retrieved and formatted into the prompt, which is then used to generate a response from the query engine.

   retrieved_nodes = vector_retriever.retrieve(query_str)
   context_str = "\n\n".join([n.get_content() for n in retrieved_nodes])
   
   print(qa_prompt_tmpl.format(query_str=query_str, context_str=context_str))
   
   response = query_engine.query(query_str)
   print(str(response))

Under the Hood:

• PII Filtering: The NERPIINodePostprocessor uses a Named Entity Recognition (NER) model to identify and mask PII in the text nodes. This ensures that any sensitive information is removed before the context is included in the prompt.
• Dynamic Context Processing: The filter_pii_fn function dynamically processes the context string to remove PII using the NERPIINodePostprocessor before it is formatted into the prompt.
• Prompt Template: The PromptTemplate allows for the dynamic insertion of the sanitized context and query into the prompt structure.
• Query Engine Integration: The updated prompt template is integrated into the query engine, enabling it to use the sanitized context for generating responses.

This setup ensures that the context used in the QA system is free of PII, enhancing privacy and compliance while maintaining the quality of the responses ^[1][2].

To continue talking to Dosu, mention @dosu.