Include Vespa Lexical Search as an option to BEIR benchmark

[thigm85]

@NThakur20 could you take a look at this PR? The idea is to make it easier to benchmark Vespa applications using the BEIR datasets and framework. We started with Lexical Search but will make it more general later.

[thakur-nandan]

Hi @thigm85, thank you for providing this PR!

in the next upcoming days, I will have a look at the PR. I'm happy to see Vespa being included within BEIR!

I was busy shifting the repository last few weeks. Will soon merge with the development branch and later release with the next version of beir updates!

Kind Regards,
Nandan Thakur

[thakur-nandan]

Hi @thigm85, I went through your PR. Thanks for all the code added and it looks good.

I have a few updates mentioned here.

1. in setup.py can you add pyvespa and tenacity as optional dependencies something similar to what I have done for tensorflow? Something like:

optional_packages = {
    "tf" : ['tensorflow>=2.2.0', 'tensorflow-text', 'tensorflow-hub'],
    "vespa": ["pyvespa", "tenacity"]
}

2. Can you create a tests folder within the main directory of BEIR and place test_retrieval_lexical_vespa.py inside this tests folder. I currently do not have any unittests implemented for other methods, this will unify in future all unittests at one place.

3. could you shift your example benchmark_lexical_vespa.py from examples/benchmarking to examples/retrieval/evaluation/lexical because the other place contains all sample scripts to evaluate different lexical search and will be easier for the user to find it.

4. Could you mention briefly on top of benchmark_lexical_vespa.py a few steps on how to run vespa lexical search? What must a user should have in place to run vespa search? Or how to download and run the vespa application? You can have a look at evaluate_bm25.py for reference.

Thank you! Will merge once it the small updates mentioned has been resolved.

Kind Regards,
Nandan Thakur