Code success outcomes

[LukasWallrich]

As promised, here the success outcomes.

• success when the authors interpret their replication as successful (e.g., “we successfully replicated…”)
• failure when the authors interpret their replication as unsuccessful (e.g., “we failed to replicate…”)
• mixed when the authors describe their results as mixed or as a partial replication. If they report a detail that was diffferent but still conclude success or failure overall, we trust them on the overall judgement.

There are a three more specific categories that are used less frequently:

• uninformative when the authors explicitly describe their results as uninformative, e.g. due to a small sample size
• statistically successful but flawed when the authors replicate the original finding but primarily do so to show that the method is flawed, and that an alternative analysis yields different results
• descriptive only when authors describe patterns but do not draw comparative conclusions.

[LukasWallrich]

@ebel-frank tagging you to make sure you see this :)

[LukasWallrich]

@ebel-frank here the full list of coded outcomes - not that any where the source is not abstract can be ignored here:
https://docs.google.com/spreadsheets/d/1xoqdkhKTVtSAe_Gs-DcHEGB4FdDn4kUYz-xjuyfezYY/edit?gid=984458430#gid=984458430