Artifact Automation

[jon-bell]

Should an artifact automatically generate any tables/graphs that appear in a paper, or is it OK (or, perhaps even better, given the potential for mistakes in these scripts) for this to be a somewhat manual process?

[sbaltes]

Regardless of whether the process is manual, semi-automatic, or automatic, it should be documented in a way that allows reviewers to understand and reproduce how the tables/graphs were created from the data to be able to verify their correctness.

[piskachev]

I think the "call for artifacts" should encourage the authors to make the steps automatic as much as possible. However, for manual steps, authors should be aware that reviewers should understand the artifact and judge its quality in a "reasonable" amount of time.