How might we account for deny lists?

[stevepolitodesign]

There might be times where two filters share the same label. One example is if you need to create a deny list, to account for limitations with a NER filter.

For example, if the current people_filter does not filter a name, you might do something like this:

known_people = TopSecret::Filters::Regex.new(label: "PERSON", regex: /Steve Polito/i)

TopSecret.configure do |config|
  config.custom_filters << known_people
end

[stevepolitodesign]

Another solution to this problem is to just train your existing model.

[MatheusRich]

Isn't this the same as a custom filter?

[stevepolitodesign]

@MatheusRich at this point, yes. Here's a real-world example:

result = TopSecret::Text.filter("My name is Ralph")

# MITIE does not actually recognize this as a name
result.output # => "My name is Ralph"

known_people = TopSecret::Filters::Regex.new(label: "PERSON", regex: /Ralph/i)
result = TopSecret::Text.filter("My name is Ralph", custom_filters: [known_people])

result.output # => "My name is [PERSON_1]"

However, this requires you to ensure the label matches.

[MatheusRich]

I like that being explicit.

[stevepolitodesign]

I should add this to the README.