Replace Highcharts with fully open-source charting library

[brylie]

Context

Orange version

Orange 2.x and 3.x

Expected behavior

Orange would comply with license terms of all internal components, and would integrate only fully open-source components.

Actual behavior

The Highcharts library is not fully open-source.

Additional info

Orange is likely in violation of the Highcharts license. Full details supporting this conclusion are available in a discussion in the orange repository.

Proposal

Replace Highcharts with a fully open-source charting library. Specifically, the following libraries are strong candidates:

• Altair - Altair is a declarative statistical visualization library for Python, based on the powerful Vega-Lite visualization grammar.
• Bokeh - Bokeh is a Python interactive visualization library that targets modern web browsers for presentation.
• Matplotlib D3 (mpld3) - The mpld3 project brings together Matplotlib, the popular Python-based graphing library, and D3js, the popular JavaScript library for creating interactive data visualizations for the web.

[kernc]

Replacing Highcharts with Vega/Plotly. The add-ons that still use Highcharts have been relicensed to CC-BY-NC interim eventual porting. Making Vega nice and interactive looks something to swallow.

A Plotly-based implementation of parallel coordinates in #2327.

[brylie]

Does vega/vega-lite use CSS for styling? Vega seems to have a well concieved, mostly declarative, API.

Plot.ly does look nice. Are there any pros/cons to its Python/JS APIs ?

Also, which specific addons use Highcharts?

[brylie]

Jake VanderPlas (@jakevdp) gave a talk at PyCon 2017, entitled The Python Visualization Landscape. The presentation compares several major projects within the Python Visualization ecocystem, and how they relate. Projects directly relevant to this discussion include

• Altair (part of University of Washington eScience Institute - @uwescience),
• Bokeh (from Continuum Analytics - @ContinuumIO), and
• plot.ly (@plotly).

[kernc]

Vega / Altair lacks sufficient interactive examples, and interactivity is key in Orange. Besides, I've followed the tutorial and was at the end trying to adapt their Airports example to weigh lines bolder by number of flights; took me a week to figure it out. Vega versions used in the tutorial and in the online editor aren't the same (3 vs 2) so copying code, as is implied in text, doesn't even work. Vega in whole felt like the visualization code will become, albeit safely packed in JSON, impossible to read afterwards and maintain in the long run. Instead of simple imperative JS event handlers anyone can reason about, you get safe-subset-of-JavaScript string expressions scattered all over the place. The community is virtually non-existent (very few questions on StackOverflow compared to competing libs, even though the project is nearly four years old); regardless of its purported eventual heavy use on Wikipedia, and it's "beautiful" declarative nature, I don't think this is something that will stick, or that complex interactive visualizations will be routinely built with it.

That's my current view of Vega. I have it wrapped, though, so it's in general trivial to include, should the need arise.

Bokes was a top contender, but Plotly seemed like a lighter weight solution and more like what we currently rely on. OTOH, for a completely new, browser-based solutions, I'd have no reservations about using Bokeh controls for the UI as well.

Does vega/vega-lite use CSS for styling?

These solutions mostly output SVG, so elements can be styled with CSS to a degree. Browsers' support is currently still limited to a subset of CSS directives (e.g. text-overflow: ellipsis for sure doesn't work).

Plot.ly does look nice. Are there any pros/cons to its Python/JS APIs ?

JS API is JSON, Python API uses thin, arguments-checking dicts. It is declarative in the way of data and options passed. Easy to add JS event handlers. Probably harder to make completely custom visualizations, unless one is proficient in D3 anyway.

Also, which specific addons use Highcharts?

Of public add-ons I know, Timeseries and Educational.

[kernc]

Vega seems to have a well concieved, mostly declarative, API.

Well-conceived? It's a flood of nested directions, few of which intuitively fit together and do exactly what one expects. Much like CSS, except that instead of 25M people knowing (about) it, 25 people do. 😆

[brylie]

I just meant that the Vega grammar describes common characteristics of data visualization such as:

• data
• transformations
• scales
• axes
• legends
• markings

The goal of the Vega specification is to help people think (declaratively) in terms of the visualizations they want to produce, rather than the (imperative) machine steps necessary to produce them.

@jakevdp, are there any examples of interactive visualizations made with Altair/Vega that we can use as reference?

[jakevdp]

FYI: vega/vega-lite just released a backward-incompatible update, and because I've been on parental leave I've not had time to update Altair, or update some of the confused JS library links that were changed in the release. For the same reason, Altair currently has some hiccups. It also doesn't currently support the interaction that was introduced a few weeks ago in the Vega-Lite release. Many of the difficulties you encountered are due to that unfortunate coincidence of timing (the vega-lite release & my parental leave) and I'll work on addressing them when I'm back to work in June.

I guess my request is: please don't judge the libraries on difficulties they present right now – it's quite temporary.

I think @kanitw and @domoritz might be able to point to to interactive features on the vega/vega-lite side, but they are not yet implemented in Altair.

[jakevdp]

Same goes for documentation. I put some simple Altair docs out last fall, and decided to wait on putting concerted effort into them until after the impeding vega-lite 2.0 release. It turns out that release, for various reasons, was gradually pushed back for about six months, so in retrospect I probably should have been working on the Altair docs in the meantime. Hindsight, 20/20, and all that.

[brylie]

Thank you so much for your response @jakevdp, and for your continued work on Altair. Regards to you and your family 😃

[domoritz]

@kernc Thank you for the interesting discussion. We are currently preparing the new releases and so some of the documentation is out of sync (we fixed the issue you pointed out in the docs at vega/vega#821). You can try Vega 3 and Vega-Lite 2 in the online editor with those libraries installed. For example, at https://vega.github.io/new-editor/?mode=vega-lite&spec=brush&showEditor=1 you brush and select points (this example is Vega-Lite, the high-level language that compiles to Vega).

Vega-Lite is more concise and so you wouldn't have to write (and maintain) large Vega specs. However, if you need to customize a visualization, you can always post-process the generated Vega spec. This is, in my option, really nice as it allows you to switch between abstraction levels as needed.

We agree that the community is not that large but we have an active community at https://groups.google.com/forum/#!forum/vega-js. Our lab has built tools around authoring Vega and Vega-Lite and we will continue to work on these libraries.

Does vega/vega-lite use CSS for styling?

Vega-Lite compiles to Vega and Vega has different renders (SVG and Canvas). Because of this, you cannot reliably style with CSS. However, Vega supports all common styling options directly. For example, you can truncate text with limit (see example at https://vega.github.io/vega/docs/marks/text/).

[fab6]

I am far from being a expert, but bokeh seems to me quite powerful also in the long-run due to the continuum support.

[brylie]

I am wondering, why was this issue closed? I think we have a good discussion here, and it might be useful to start a prototype. E.g. now discuss the details of working with Bokeh or Altair at least as Orange add-ons.

[astaric]

Orange visualizations do not rely on highcharts (and never did). The issues was closed since highcharts libraries are no longer part of orange distribution so the licence compliance should be ok now.

There are two add-ons I am aware of that still rely on highcharts: Timeseries and Educational. If anyone wants to start working on this, that would be IMHO the best place to start. Isuees can be opened there, where alternative libraries can be evaluated based on how well they are suited to reimplement existing visualizations.

If you would rather reuse this issue, feel free to reopen it.

[brylie]

OK, cool. I would just like to discuss more about using a Bokeh or Altair to build visualization widgets for Orange. So,it could be a separate issue, I reckon. Thanks for your reply.