How might we determine if free text contains any sensitive information before filtering it?

[stevepolitodesign]

I can image a case where we just want to scan an input to determine if it contains any sensitive information. Once example could be a validating a message in Rails.

class Message < ApplicationRecord
  validate :content_cannot_contain_sensitive_information
 
  private

  def content_cannot_contain_sensitive_information
    # Implementation here
  end
end

[stevepolitodesign]

Right now, we could do something like this:

class Message < ApplicationRecord
  validate :content_cannot_contain_sensitive_information
 
  private

  def content_cannot_contain_sensitive_information
    result = TopSecret::Text.filter(content)
    return if result.mapping.empty? 
    
    errors.add(:content, "contains the following sensitive information #{result.mapping.values.to_sentence}")
  end
end

However, it might be more performant to build a mapping without actually filtering the text.

class Message < ApplicationRecord
  validate :content_cannot_contain_sensitive_information
 
  private

  def content_cannot_contain_sensitive_information
    result = TopSecret::Text.scan(content)
    return if result.clean?
    
    errors.add(:content, "contains the following sensitive information #{result.mapping.values.to_sentence}")
  end
end

[stevepolitodesign]

This was addressed in #70