Skip to content

Missing IC-SWE instances #61

@xuyabo0035

Description

@xuyabo0035

Hi, thanks for releasing this benchmark! I have two questions:

  1. The paper mentions the Diamond subset contains 237 IC-SWE instances, but in the repo at https://github.com/openai/preparedness/blob/main/project/swelancer/all_swelancer_tasks.csv, there are only 198. Where are the remaining examples?
  2. How can I get access to the full dataset?

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions