Reproducibility Bugs

Every time we encounter a reproducibility related bug/issue (in our attempts to reproduce others' work), we'll document it here. To the best of our abilities.

• README doesn't tell me where to start.
• Chicken and egg issue with environments. No environment.yml file or the like. (Even if there are some instructions in a notebook).
• Versions not pinned. E.g. uses a dev branch without a clear indication of when it became released.
• No data license.
• No repo license. Continued with expressed permission.
• No data versioning/hash. No way to tell if the data changed since it was originally accessed.
• Variable undefined. (Hidden state error, cells run out of sequence, copy notebook error?)
• Markdown cell(s) copied and wrong. (comments in copied notebooks that weren't updated)
• "We'll overwrite our variable (which is generally bad) in order to save on code reuse from the last notebook."
• Copied notebooks for code reuse, instead of generalizing to functions/module
• Fixed random seed, different result. This time, different results on different architectures.
• Ran notebooks out of order. No indication of where to start, or where to go next.
• Markdown cell(s) copied and wrong. (comments in copied notebooks that weren't updated)
• Markdown cell(s) copied and wrong. (comments in copied notebooks that weren't updated)