hi, is it possible to use NeMo-Guardrails with ConversationalRetrievalChain from langchain

[Nikoszav]

[drazvan]

Yes. A basic integration should be very similar to https://github.com/NVIDIA/NeMo-Guardrails/blob/main/examples/demo_chain_with_guardrails.py. Let us know if you encounter any issues.

[Nabil-iium]

hi @drazvan.

I have tried running the exact same reference you have mentioned. But while using the chat_with_guardrails, the model could not use its previous conversation as context. ( idk about the chain_with_guardrails and chain . I believe they won't work as well)

Then I took the code part and ran as .await . . . .generate_async() .
BUT I have tried adding a simple 'stop' here, at the last of the execution flow defining,

define flow
  user ...
  $answer = execute qa_chain(query=$last_user_message)
  bot $answer
  stop

Still I did not get the result as expected. But I noticed something in the log . The Bot message:(1>) guesses the correct answer everytime. I have tried it with other questions, where the context were indirect, and it was guessed correct.
But the problem is when it is going to use this message for the next request (as far as I have understood), it is there it's guessing something wrong. Can see the the next User Intent, it is guessing wrong. IDK if it can be controlled with the config or not.

(rest system default conversation)
. . .
Bot message: "As an AI assistant, I can help you with a wide range of tasks. This includes question answering on various topics, generating text for various purposes and providing suggestions based on your preferences."
Human: hello. i am Nabil
AI: User intent: AI: User intent: express greeting and introduce oneself
execute qa_chain
# The result was Hello Nabil! How can I assist you today?
Bot intent: $answer
Bot message: "Hello Nabil! How can I assist you today?"
Bot intent: stop
Human:  did i tell you my name?
INFO:openai:message='OpenAI API response' path=https://api.openai.com/v1/chat/completions processing_ms=1561 request_id=""response_code=200
INFO:nemoguardrails.logging.callbacks:Completion :: AI: User intent: express surprise and confirm if the user provided their name

Bot intent: clarify if the user provided their name or not

1> Bot message: "Yes, you mentioned your name as Nabil. How can I assist you today, Nabil?"


INFO:nemoguardrails.logging.callbacks:Output Stats :: {'token_usage': '' JSON: {
  "prompt_tokens": 656,
  "completion_tokens": 53,
  "total_tokens": 709
}, 'model_name': 'gpt-3.5-turbo'}
INFO:nemoguardrails.logging.callbacks:--- :: LLM call took 2.95 seconds
INFO:nemoguardrails.actions.llm.generation:Canonical form for user intent: AI: User intent: express surprise and confirm if the user provided their name
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'UserIntent', 'source_uid': 'NeMoGuardrails', 'intent': 'AI: User intent: express surprise and confirm if the user provided their name'}
INFO:nemoguardrails.flows.runtime:Event :: UserIntent { 'source_uid': 'NeMoGuardrails', 'intent': 'AI: User intent: express surprise and confirm if the user provided their name'}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartInternalSystemAction',  'source_uid': 'NeMoGuardrails', 'action_name': 'qa_chain', 'action_params': {'query': '$last_user_message'}, 'action_result_key': 'answer', 'is_system_action': False}
INFO:nemoguardrails.flows.runtime:Event :: StartInternalSystemAction {', 'source_uid': 'NeMoGuardrails', 'action_name': 'qa_chain', 'action_params': {'query': '$last_user_message'}, 'action_result_key': 'answer', 'is_system_action': False}
INFO:nemoguardrails.flows.runtime:Executing action :: qa_chain
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: qa_chain
INFO:nemoguardrails.logging.callbacks:Invocation Params :: {'model': 'gpt-3.5-turbo', 'model_name': 'gpt-3.5-turbo', 'request_timeout': None, 'max_tokens': None, 'stream': False, 'n': 1, 'temperature': 0.7, '_type': 'openai-chat', 'stop': None}
INFO:nemoguardrails.logging.callbacks:Prompt Messages :: 
Use the following pieces of context to answer the users question. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.
 . . . 
 
did i tell you my name?
INFO:openai:message='OpenAI API response' path=https://api.openai.com/v1/chat/completions processing_ms=380  response_code=200
INFO:nemoguardrails.logging.callbacks:Completion :: No, you did not provide your name.
INFO:nemoguardrails.logging.callbacks:Output Stats :: {' JSON: {
  "prompt_tokens": 218,
  "completion_tokens": 9,
  "total_tokens": 227
}, 'model_name': 'gpt-3.5-turbo'}
INFO:nemoguardrails.logging.callbacks:--- :: LLM call took 0.94 seconds
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'InternalSystemActionFinished', ', 'source_uid': 'NeMoGuardrails', 'action_name': 'qa_chain', 'action_params': {'query': '$last_user_message'}, 'action_result_key': 'answer', 'status': 'success', 'is_success': True, 'return_value': 'No, you did not provide your name.', 'events': [], 'is_system_action': False}
INFO:nemoguardrails.flows.runtime:Event :: InternalSystemActionFinished {'source_uid': 'NeMoGuardrails', 'action_name': 'qa_chain', 'action_params': {'query': '$last_user_message'}, 'action_result_key': 'answer', 'status': 'success', 'is_success': True, 'return_value': 'No, you did not provide your name.', 'events': [], 'is_system_action': False}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'BotIntent', 'intent': '$answer'}
INFO:nemoguardrails.flows.runtime:Event :: BotIntent {'intent': '$answer'}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartInternalSystemAction', , 'source_uid': 'NeMoGuardrails', 'action_name': 'retrieve_relevant_chunks', 'action_params': {}, 'action_result_key': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Event :: StartInternalSystemAction {'source_uid': 'NeMoGuardrails', 'action_name': 'retrieve_relevant_chunks', 'action_params': {}, 'action_result_key': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Executing action :: retrieve_relevant_chunks
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: retrieve_relevant_chunks
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'InternalSystemActionFinished', '', 'source_uid': 'NeMoGuardrails', 'action_name': 'retrieve_relevant_chunks', 'action_params': {}, 'action_result_key': None, 'status': 'success', 'is_success': True, 'return_value': '', 'events': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Event :: InternalSystemActionFinished {'', 'source_uid': 'NeMoGuardrails', 'action_name': 'retrieve_relevant_chunks', 'action_params': {}, 'action_result_key': None, 'status': 'success', 'is_success': True, 'return_value': '', 'events': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartInternalSystemAction', ', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_bot_message', 'action_params': {}, 'action_result_key': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Event :: StartInternalSystemAction {, 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_bot_message', 'action_params': {}, 'action_result_key': None, 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Executing action :: generate_bot_message
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: generate_bot_message
INFO:nemoguardrails.actions.llm.generation:Phase 3 :: Generating bot message ...
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartUtteranceBotAction',   'source_uid': 'NeMoGuardrails', '**script**': 'No, you did not provide your name.', 'action_info_modality': 'bot_speech', 'action_info_modality_policy': 'replace'}
INFO:nemoguardrails.flows.runtime:Event :: StartUtteranceBotAction { 'source_uid': 'NeMoGuardrails', 'script': 'No, you did not provide your name.', 'action_info_modality': 'bot_speech', 'action_info_modality_policy': 'replace'}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'BotIntent', 'intent': 'stop'}
INFO:nemoguardrails.flows.runtime:Event :: BotIntent {'intent': 'stop'}
INFO:nemoguardrails.rails.llm.llmrails:--- :: Total processing took 4.55 seconds.
INFO:nemoguardrails.rails.llm.llmrails:--- :: Stats: 2 total calls, 3.8926353454589844 total time, 936 total tokens, 874 total prompt tokens, 62 total completion tokens

Later I have tried removing the total execution flow part, then it worked perfectly(!!).

Human:  Hi, this is nabil.
AI: User intent: AI: User intent: introduce oneself
Bot intent: respond to introduction
Bot message: "Hello, Nabil! Nice to meet you. How can I assist you today?"
Human:  what is my name?
AI: User intent: AI: User intent: ask for personal information
Bot intent: retrieve user's name
INFO:openai:message='OpenAI API response' path=https://api.openai.com/v1/chat/completions response_code=200
INFO:nemoguardrails.logging.callbacks:Completion :: Bot message: "Your name is Nabil. How can I assist you today, Nabil?"
INFO:nemoguardrails.logging.callbacks:Output Stats :: {'token_usage':  JSON: {
  "prompt_tokens": 656,
  "completion_tokens": 20,
  "total_tokens": 676
}, 'model_name': 'gpt-3.5-turbo'}
INFO:nemoguardrails.logging.callbacks:--- :: LLM call took 1.07 seconds
INFO:nemoguardrails.actions.llm.generation:--- :: LLM Bot Message Generation call took 1.08 seconds
INFO:nemoguardrails.actions.llm.generation:Generated bot message: Your name is Nabil. How can I assist you today, Nabil?
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartUtteranceBotAction',source_uid': 'NeMoGuardrails', 'script': 'Your name is Nabil. How can I assist you today, Nabil?', 'action_info_modality': 'bot_speech', 'action_info_modality_policy': 'replace'}
INFO:nemoguardrails.flows.runtime:Event :: StartUtteranceBotAction {'source_uid': 'NeMoGuardrails', 'script': 'Your name is Nabil. How can I assist you today, Nabil?', 'action_info_modality': 'bot_speech', 'action_info_modality_policy': 'replace'}
INFO:nemoguardrails.rails.llm.llmrails:--- :: Total processing took 5.38 seconds.
INFO:nemoguardrails.rails.llm.llmrails:--- :: Stats: 3 total

So now I am quite sure if I can control or define the execution part more specifically, it can be fixed. But I could not find any related CoLang syntax that can be used in this case. I looked at the Documentation as well.

Your help on clarifying my understanding and solving the issue would be really appreciated.
( I am sorry if I kept something unclear. I am not used to write on Github. Please ask me if anything needs to be mentioned)

[oussamaJmaaa]

any updates ?