IRP dataset?

[i-am-neo]

@Teddy-Li @Nighttime
Thanks for your interesting paper! I'm interested in having another language model take a test using your methods.

Can you point me to datasets for Random Premise task (IRP ) and typed identifiers (IRP _TA)?

Thanks for your assistance.

[Teddy-Li]

Hello! Thanks for your interest! Apologizes for the previously slightly outdated repository structure; I have recently restructured the codes and data and results files so they are hopefully clearer for use. The I_{RP} and I_{RP_TA} files now sit in this folder; particularly, loading the type-argument variant requires both files, handled by the load_typed_general_entries function in utils.py. More detailed instructions on analyzing the results will shortly become available on the README of this repo. Do let me know if you have more questions :)

Cheers

[iBibek]

Thank you @Teddy-Li