Unable to install annoy which is a dependency of capreolus

[RiverDong]

Hi, I am trying to run the code. And I need to install annoy which is a depedency of capreolus. However, I am unable to install it for reasons I cannot debug below. I wonder if you encountered similar issues or know some environment settings that you used. (Like python versions, package versions, etc.) I also posted the same question in capreolus and hopefully get some help there. Thank you for your help!

<img width="957" alt="Screen Shot 2022-06-04 at 9 00 43 AM" src="https://user-images.githubusercontent.com

/54600128/172014039-2857aabb-2494-4f05-a5e1-7bc56554a42c.png">

[RiverDong]

Hi I tried again and annoy works now. But when I run the command in section 3. I will get this. I think it's the spacy version that is incorrect? Now I am using 3.2.4.

I also tried to instead of doing pip install git+https://github.com/capreolus-ir/capreolus.git@d7d2631994dcf56e3920fd5294e1758fd798544b, I did pip install capreolus. And it's getting me this error.

Let me know if you see which one you have encountered before. Thank you!

[crystina-z]

Hi @RiverDong, for spacy version, I'm using 2.3.7;

The second error was because that some config name and also their implementation (e.g. warmupsteps) has changed after the commit d7d26.

I hvn't tried to run the code with the up-to-date Capreolus. Lmk if change the spacy version helps, if not I can try to see if I can adapt this repo to reproduce with the latest Capreolus.

[RiverDong]

Hi Crystina, no it does not help. There are more dependency issues. I think it's because the default version is incorrect. Is that possible for you to just run "conda env export | grep -v "^prefix: " > environment.yml" and share the enviorment file. Thank you

[crystina-z]

Hi @RiverDong! So sorry for missing the reply here again, and thank you really for reaching out via email.
I tried to export the conda environment but conda throws error like TypeError: '<' not supported between instances of 'str' and 'NoneType', let me try to go through the environment set up and see if I can reach your issue and find a solution. If not, I'll start testing the code on the latest Capreolus.

Thanks again for your patience.

[crystina-z]

Hi @RiverDong, I tried to set up the conda environment from scratch and started with
pip install git+https://github.com/capreolus-ir/capreolus.git@d7d2631994dcf56e3920fd5294e1758fd798544b

After the spacy version issues above, I met a package issue related to tensorflow and tf-ranking, which seemingly can be solved by pip install tensorflow-ranking==0.3.

Seems that after re-installing tf-ranking, the script for section 1 could start running on my end. still waiting for the results tho. lmk if this solves the issue on your end.

[RiverDong]

Hi @crystina-z, that doesn't work. There's more and more wierd version issues that I don't want to go into details. I think the most efficient way is to export the enviornment file. Could you try other commands herehttps://stackoverflow.com/questions/41274007/anaconda-export-environment-file and see if it works?

[RiverDong]

If the latest Capreolus works that would be also a good solution I believe

[crystina-z]

sadly the conda list gave the same error. I put the results of pip freeze here. lmk if that helps

[RiverDong]

Nice, it seems that python 3,7 and Tensorflow-ranking 0.3.0 moves a bit further. But then I saw this error. I cannot solve it. Have you ever met this error (know the reason)? Thank you!

[RiverDong]

The previous one is simple, I forget to export JAVA_HOME and path. But this one is quite wierd. The .so file is there but the code cannot access it b/c it's turned to binary somehow. Did you got this error before? Thank you! (If this still doesn't work, I guess I will have to go to something else. Did you know similar code that do what you did in section 3?)

[crystina-z]

Hi @RiverDong, sadly I didn't see this error before. not sure if this would be helpful?

Sadly I don't really know the other codebase that was comparing the deep and shallow labeling. but I guess if you want to try it on your dataset, the implementation was straightforward. What i was doing is basically sampling the queries / the judged documents per query. With the dataset prepared, all the training and inference are same.