Regarding open review, the most USELESS thing was....

[timm]

No description provided.

[emhill]

When authors had multiple READMEs on the internet for related or the same artifact. Sometimes there would be this web of links to different directions, some of which conflicted, or were for an artifact that the submission was based on. I had trouble knowing if I had successfully run the submitted artifact, or if I had run an artifact that had been previously published.

[mcmillco]

It was not all that helpful to review the "insightfulness", since that was often evaluated by reviewers of a related research paper. Also, it is hard to see the benefit sometimes when you are not actively working in an area.

I would instead focus on how usable and available the tools/data are.

Though I suppose this has more to do with the Artifacts track specifically, and not open review.