Understand PMC_128 data

[james20141606]

Hi, I downloaded the chunked PMC dataset with link http://nlp.dmis.korea.edu/projects/selfbiorag-jeong-et-al-2024/data/retriever/PMC_128.tar.gz
I found that there are files

PMC_128_Abs_Articles.json  PMC_128_Main_Articles.json
PMC_128_Abs_Embeds.npy     PMC_128_Main_Embeds.npy
PMC_128_idx_array.npy

I assume that contain's everything?

For the other small files under PMC_128_temporary, they should be the same as the above merged file?

[minstar]

@jw-sohn Could you please help @james20141606 to understand the following files?

[jw-sohn]

Hello @james20141606 !
Thank you for your interest in our work!
Please place the two files:
PMC_128_Abs_Embeds.npy and PMC_128_Main_Embeds.npy in the retriever/embeddings/pmc directory,
and the other two:
PMC_128_Abs_Articles.json and PMC_128_Main_Articles.json in the retriever/articles/pmc directory.

Once that’s done, everything should be ready to go!