Initial NEMO implementation

[RobGeada]

Adds initial NEMO implementation:

Example deployment resource

• Replace $YOUR_MODEL_PREDICTOR_SERVICE as needed (e.g., http://phi3-predictor.model-namespace.svc.cluster.local):

apiVersion: v1
kind: ConfigMap
metadata:
  name: nemo-config
data:
  config.yaml: |
    models:
      - type: main
        engine: vllm_openai
        parameters:
          openai_api_base: "$YOUR_MODEL_PREDICTOR_SERVICE/v1"
          model_name: "llm"
          api_key: "None"
    rails:
      config:
        sensitive_data_detection:
          input:
            entities:
              - PERSON
              - EMAIL_ADDRESS
          output:
            entities:
              - PERSON
      input:
        flows:
          - detect sensitive data on input
          - check message length
          - check forbidden words
      output:
        flows:
          - detect sensitive data on output
  rails.co: |
    define flow check message length
      $length_result = execute check_message_length
      if $length_result == "blocked_too_long"
        bot inform message too long
        stop
      if $length_result == "warning_long" 
        bot warn message long

    define bot inform message too long
      "Please keep your message under 100 words for better assistance."

    define bot warn message long
      "That's quite detailed! I'll help as best I can."
    
    define flow check forbidden words
      $forbidden_result = execute check_forbidden_words
      if $forbidden_result != "allowed"
        bot inform forbidden content
        stop

    define bot inform forbidden content
      "I can't help with that type of request. Please ask something else."
    

  actions.py: |
    from typing import Optional
    from nemoguardrails.actions import action

    @action(is_system_action=True)
    async def check_message_length(context: Optional[dict] = None) -> str:
        """Check if user message is within acceptable length limits."""
        user_message = context.get("user_message", "")
        word_count = len(user_message.split())
        MAX_WORDS = 100
        if word_count > MAX_WORDS:
            return "blocked_too_long"
        elif word_count > MAX_WORDS * 0.8:
            return "warning_long"
        return "allowed"

    @action(is_system_action=True)
    async def check_forbidden_words(context: Optional[dict] = None) -> str:
        """Check for forbidden words or topics."""
        user_message = context.get("user_message", "").lower()
        forbidden_topics = {
            "security": ["password", "hack", "exploit", "vulnerability"],
            "inappropriate": ["violence", "illegal", "harmful"],
            "competitors": ["chatgpt", "openai", "claude", "anthropic"],
        }
        for category, words in forbidden_topics.items():
            for word in words:
                if word in user_message:
                    return f"blocked_{category}_{word}"
        return "allowed"
---
apiVersion: trustyai.opendatahub.io/v1alpha1
kind: NemoGuardrails
metadata:
  name: nemo-guardrails
spec:
  nemoConfig: "nemo-config"

Example curl:

GUARDRAILS_ROUTE=https://$(oc get routes/nemo-guardrails -o json  -o jsonpath='{.status.ingress[0].host}')
curl -k -X POST $GUARDRAILS_ROUTE/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $(oc whoami -t)" \
  -d '{"messages":[{"role":"user","content":"How does this compare to ChatGPT?"}]}'

returns:

{"messages":[{"role":"assistant","content":"I can't help with that type of request. Please ask something else."}]}

[github-actions]

PR image build and manifest generation completed successfully!

📦 PR image: quay.io/trustyai/trustyai-service-operator:3004d4a73bee74eb6fb8adab02f86cbb45e36ca2

📦 LMES driver image: quay.io/trustyai/ta-lmes-driver:latest

📦 LMES job image: quay.io/trustyai/ta-lmes-job:latest

📦 Guardrails orchestrator image: quay.io/trustyai/ta-guardrails-orchestrator:latest

🗂️ CI manifests

      devFlags:
        manifests:
          - contextDir: config
            sourcePath: ''
            uri: https://api.github.com/repos/trustyai-explainability/trustyai-service-operator-ci/tarball/operator-3004d4a73bee74eb6fb8adab02f86cbb45e36ca2

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@RobGeada: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/trustyai-service-operator-e2e	3004d4a	link	true	/test trustyai-service-operator-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.