Faithfulness prompt update to avoid having single quotes in response (#1874)

The error described in [this
comment](#1688 (comment))
is not resolved when computing Faithfulness.

After digging in judge LLMs responses, there is a JSON parse error
happening when parsing the output of the judge. For instance, it happens
for the below answer:

```
{
  "statements": [
    {
      "statement": "The Norwegian Dawn cruise ship was denied access to Mauritius.",
      "reason": "The context explicitly states that local authorities denied permission for the Norwegian Dawn ship to access the Mauritius capital of Port Louis.",
      "verdict": 1
    },
    {
      "statement": "The denial of access was due to potential health risks.",
      "reason": "The context directly mentions that the ship was denied access \"citing \\\"potential health risks.\\\"\"",
      "verdict": 1
    },
    {
      "statement": "The specific health risk was a potential cholera outbreak on the Norwegian Dawn cruise ship.",
      "reason": "While the context mentions fears of a potential cholera outbreak in the title, it does not explicitly state that cholera was the specific health risk on the Norwegian Dawn. The context only mentions \'stomach-related illness\' without specifying cholera.",
      "verdict": 0
    }
  ]
}
```

The error is that the generated context has single quotes as you can see
in the last reason. This is not allowed in JSON, and it happens
frequently with the current prompt for Faithfulness as it often cite
elements from the retrieved context to explain its verdict.

I tried to change the `PydanticOutputParser` logic in Langchain Core,
the one that Ragas uses to parse JSON. I tried to replace single quotes
by double quotes with simple string replace but it did not work.

Thus, an immediate solution that worked for me was to specifically ask
the judge LLM to only output double quotes, not single quotes, and the
error disappears.

Author: michaelromagne

src/ragas/prompt/pydantic_prompt.py

@@ -46,6 +46,8 @@ def _generate_output_signature(self, indent: int = 4) -> str:
             f"Please return the output in a JSON format that complies with the "
             f"following schema as specified in JSON Schema:\n"
             f"{self.output_model.model_json_schema()}"
+            "Do not use single quotes in your response but double quotes,"
+            "properly escaped with a backslash."
         )
 
     def _generate_examples(self):