ENH: Decouple the benchmark runner from viewer

[ev-br]

Consider a use case: I've a test suite which uses, say, pytest-benchmark and I'd like to be able to reuse asv publish to web display.

A first step could be to document the data format asv expects. This includes json file names, formats and locations. (also #1410 ). Then writing an adapter from whatever json pytest-benchmark or other test runner produces would be easier.

[HaoZeke]

IMO this is probably a better use case for a a plugin like the memray-plugin where say, instead of using pytest-benchmark directly and trying to match results, the benchmarks would be written using asv calling pytest-benchmark and then plugging seamlessly into the web display.

[ev-br]

Such plugin would be interesting indeed. Got to admit I've no idea how to implement one though, FWIW.

That said, I'd argue going the plugin route adds tighter coupling between different thing ASV insists on doing itself: managing the repository, managing the environments, running benchmarks and visualizing them, instead of making it looser.