Fixed a typo in evaluation function

[BastienZim]

As I was going through the code to understand what the evaluation was doing I encountered this typo.
This affects the whole evaluation process and might bias the results.

This is why I would consider incorporating this change.

[kedarchandrayan]

Hello @BastienZim , please resolve the conflict.

[LazaroHurtado]

Hey @BastienZim, it seems like there were some issues while resolving the conflicts and your original typo fix is no longer in this PR. Could you please update this PR one more time resolving this issue?

[BastienZim]

It seems the evaluation function has been moved to the src/evaluators/openai.py file. Any tips of how i could change my PR to shift my change to only this file or should I open a new PR ?

[BastienZim]

@LazaroHurtado

[BastienZim]

We could also discuss whether a LLM call is the best evaluation function for the needle task. As the task is quite straightforward, resolving to the few shot-learning capabilities of LLMs might not be the best solution. I would be interested in implementing something like a rouge based evaluation or any hard coded evaluation such as whether the answer contains the exact string expected (a boolean indicating if the needle is there or not).

[LazaroHurtado]

@BastienZim, these might help you resolve all the merge conflicts:

1. Go to your forked repo and sync with the root repo, there should be a "sync" button
2. In your terminal checkout to the main branch: git checkout main
3. pull the latest changes coming in from the sync: git pull
4. Now you should have the latest changes without any conflicts, so you can make your changes to the evaluators/openai.py file
5. commit and push

I like your idea of using other algorithms for evaluation, like ROUGE and cosine similarity. Would be awesome if you can add those features!

[BastienZim]

Thanks for the tips @lazaro. Your instructions were good. However, the sync button seemed to have cancelled the PR. I opened a new one, "Fixed a typo in evaluation function #29".

[BastienZim]

I will come back soon with a base to work on different metrics !