Add content detector

[gkumbhat]

Changes

• Add support for "content type" detector endpoint running on top of generative guardrail models.

Examples

Request (Llama-guard)

curl -X 'POST'   'http://localhost:8000/api/v1/text/contents' -H 'accept: application/json'   -H 'detector-id: llama-guard'   -H 'Content-Type: application/json'   -d '{ "contents": ["If you are thinking about skipping out on filing your taxes, there are a few ways to avoid getting caught.", "this is a fairly good sentence."], "detector_params": {}}'

Note: above sentence taken from dataset examples of llama-guard

Response

[
   [
      {
         "detection" : "unsafe",
         "detection_type" : "risk",
         "end" : 106,
         "score" : 0.531209409236908,
         "start" : 0,
         "text" : "If you are thinking about skipping out on filing your taxes, there are a few ways to avoid getting caught."
      }
   ],
   [
      {
         "detection" : "safe",
         "detection_type" : "risk",
         "end" : 31,
         "score" : 0.00359360268339515,
         "start" : 0,
         "text" : "this is a fairly good sentence."
      }
   ]
]

[evaline-ju]

A couple high level things before looking in a lot of detail:

• Looks like there’s a rebase needed
• I think it’ll be good/helpful to have an example call [with either granite guardian or llama guard] in the PR to see expected usage - from what I understand so far, I'm a bit concerned about the choices for different parts of the original llama guard message.
• It’ll likely be helpful to have a couple more unit tests, specifically for the common changes in base.py (e.g. preprocess_request) and protocol.py, and since this also applies to granite guardian, it might be good to have some corresponding .content_analysis tests there?

[evaline-ju]

So this would present each of the parts of the original message as a different "choice"? This seems confusing from a user POV, since usually choices are presented as independent alternates to each other.

[gkumbhat]

this is purely internal and output of a private function (part of the reason making it private), the user of "content_detector" endpoint won't be affected. To them it will just come out as different label

[evaline-ju]

I think I understand better, that the independent choices aren't seen by the end user, though I think the "different label" is still what is confusing to me, since they are all presented at the same level i.e. in the added example

{
         "detection" : "unsafe",
         "detection_type" : "risk",
         "end" : 106,
         "score" : 0.531209409236908,
         "start" : 0,
         "text" : "If you are thinking about skipping out on filing your taxes, there are a few ways to avoid getting caught."
      },
      {
         "detection" : "S2",
         "detection_type" : "risk",
         "end" : 106,
         "score" : 0.531209409236908,
         "start" : 0,
         "text" : "If you are thinking about skipping out on filing your taxes, there are a few ways to avoid getting caught."
      }

unsafe now seems disjoint/independent of S2, whereas those that are familiar with llama guard would know those are related, more like an "unsafe - S2" concept

[gkumbhat]

hmm. true, thats an interesting point..

We could merge them together as well🤔 But the problem is, then they become separate from what Llamaguard document, like in a string matching way

[evaline-ju]

Thanks for the additional tests and example! A couple additional q's/comments

[evaline-ju]

It seems something may have gone unexpectedly with the rebase that some of these files are showing up in the diff and there's a conflict still showing

[evaline-ju]

I wonder if to avoid confusion with granite tests, we just change this to completion_response, since these are tests for the base class and meant to be generic

[gkumbhat]

hmm. true, that is doable

[evaline-ju]

Mostly comments on some docstring/comments aesthetics and a few test questions remaining

[evaline-ju]

Could we update the docstring L176 since it no longer returns DetectionResponse?

[evaline-ju]

same note about response vs. responses here as before

[evaline-ju]

fyi created follow-up story here #25

[evaline-ju]

LGTM!