AlignScore factcheck.co is not working when accuracy is lower than threshold

[Ryan-ZL-Lin]

How to reproduce the issue?

1. setup AlignScore server and use large model

2. Reference Configuration: https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/examples/configs/rag/fact_checking
   config:
   

   rails:
   factcheck.co and
   general.co

Who could help?
@drazvan

Description

Hi
I tried out the whole setup, and it seemed work well initially. I got an successful testing result by making bot to use "inform answer unknown" when the accuracy from AlignScore was lower than 0.4.

However, when I tried to adjust factcheck.co to have different bot response, it's not working as expected.
For example, I changed the threshold to make the Bot to use "inform answer potentially inaccurate" as follows, but I still got "inform answer unknown" even the score is above 0.4.

Does one who could provide some tips to better address this issue?

here is what I found in colag_history