Skip to content

Commit b7f552a

Browse files
authored
Merge pull request #39 from ibm-ecosystem-engineering/prompt_udpated_sim_llama3.1
updated prompt
2 parents 1c17b20 + 4eba66f commit b7f552a

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

Framework/answer_similarity.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@
1313
2. **Initial Setup**: Begin by carefully reviewing the Golden Text to understand the key information, entities, and intents it contains. The Golden Text is considered fully correct and comprehensive. Then, examine the Generated Text that needs evaluation.
1414
3. **Evaluation Criteria**: Evaluate the Generated Text based on the following criteria:
1515
- Output {{"Grade": "1"}} if:
16-
a) The Generated Text matches the Golden Text closely in terms of key entities and intents. Note that these may be worded differently but convey the same meaning.
16+
a) The Generated Text matches the Golden Text closely in terms of key entities and intents. Note that these may be worded differently but convey the same meaning contextually.
1717
b) The Generated Text contains all the essential information from the Golden Text, even if presented in a different order or with slight variations in phrasing.
18-
c) The Generated Text includes the core information from the Golden Text and may contain additional relevant details or expansions that don't contradict the original.
18+
c) The Generated Text includes the core information from the Golden Text or may contain additional relevant, concise details or expansions that don't contradict the contextual meaning of the Golden Text.
1919
- Output {{"Grade": "0"}} if:
2020
a) The Generated Text is missing critical entities or intents that are present in the Golden Text.
2121
b) The Generated Text contains significant factual errors or contradictions when compared to the Golden Text.

0 commit comments

Comments
 (0)