How might we support filtering conversation state?

[stevepolitodesign]

When working with conversation state you should filter every message before including it in the request. This ensures no sensitive data slips through from previous messages.

Here's a distilled example:

instructions = <<~TEXT
  I'm going to send filtered information to you in the form of free text.
  If you need to refer to the filtered information in a response, just reference it by the filter.
TEXT

messages = [
  { role: "system", content: instructions },
  { role: "user",   content: "Hello, my name is Ralph." },
  { role: "assistant", content: "Hi Ralph! How can I help you today?" }
]

filtered_messages = messages.map do |message|
  result = TopSecret::Text.filter(message[:content])
  message.merge(content: result.output, mapping: result.mapping)
end

The problem with this first pass is that it could result in inconsistent mappings. The example below would result in "Ruby" being mapped to PERSON_2 and PERSON_1.

[
  { role: "system", content: instructions },
  { role: "user",   content: "Hello, my name is Ralph. I have a friend named Ruby." },
  { role: "user",   content: "My friend Ruby works at thoughtbot" },
]

This would lead to inconsistent results when "restoring" the response from an LLM.

[MatheusRich]

I think it should refer to the mapping to check if there's an entry there already before adding a new one

[MatheusRich]

There might be an API here to filter multiple records like this:

results = TopSecret::Text.filter_all(messages)

# or maybe something more specific

messages = TopSecret::Text.filter_all(messages, key: :content)

[stevepolitodesign]

This has been resolved in #54