Important context information missing for some Qasper tasks in LongBench v1

[VisionTheta]

Hi team,

I would like to report a potential data integrity issue concerning the Qasper-derived tasks in LongBench v1.

Case Details

• LongBench item _id: d022dfe02fb2a55b4baa40fe436f616aecd3f3ced1a58d7c
• Original Qasper validation set ID: 1801.05147
• Question: "What accuracy does the proposed system achieve?"
• Issue: In the original paper, the answer can be found in Table 2 and Table 3. However, neither the original Qasper nor the LongBench dataset version for this item provides the actual content of those tables. As a result, it is impossible for any LLM to answer this question correctly based on the provided context.

Could you confirm whether there is an updated version of the LongBench dataset that addresses this or similar issues? Is there a known list of dataset integrity issues, or a process for reporting and tracking such problems?

Thank you for your attention, and for your work on LongBench!