html fit report appearance/behavior in JupyterLab

[simonkeys]

The html repr for the fit report uses html h2 tags, which have a predefined semantic meaning in JupyterLab as section headings. JupyterLab looks for header tags to generate the table of contents for a notebook. This results in the fit report table titles ("Fit Statistics", "Variables", "Correlations") appearing as notebook sections in the table of contents (left sidebar), as shown in the following screenshot.

I don't know in what other contexts the html fit reports are intended to be used, so it may be that h2 tags are needed for some reason. However, there is an html tag intended for labeling tables, namely the caption tag. I propose that the caption tag be used here instead. This fixes the issue with the table of contents, as shown in the second screenshot.

I will revise my pull request (#883) to implement whatever solution is agreed upon.

[newville]

@simonkeys Thanks -- that does helps show what is going on. I don't have a strong opinion, as I don't use JupyterLab. A suggestion: maybe have one "h2"-level heading (perhaps "Fit Report"), and then all the other report sections as "h3" or "strong"?

[reneeotten]

@simonkeys @newville I do like the idea of having "Fit report" as a header so that it shows up as a section and then the others can be h3, strong, or caption. I don't have a strong preference either. Again, perhaps @Tillsten can comment here on what his preference is, but if there is no response I would say that @simonkeys should feel free to update the PR so we can merge this as I agree the current situation doesn't look great in JupyterLab.

[Tillsten]

I just used what looked nice. Maybe make it customizeable? I still like the idea of using semantic tags.

[newville]

@Tillsten Thanks - do you think we should merge #883 and then leave semantic tags as a follow-up fix, or should we hold #883 until it has semantic tags? I worry that the PR could fall through the cracks if we wait for "perfect"

[reneeotten]

No, we should decide on a solution, implement that and merge. I agree with @simonkeys that currently it looks weird in JupyterLab and not like something many people would want. I don't know why semantic tags would be preferred but if so, then perhaps pick something that will not show up as a "main" header in the JupyterLab sidebar. I don't know how <h3> or <h4> would look like, but if that's something that makes everyone happy please change accordingly.

I am against merging a PR for which there is no consensus and then hope that someone will follow-up on it at some point in the (far) future. I am not worried about PRs falling to the cracks as there are currently only two open ;) Finally, I am personally not in favor of making a tag or caption customizable; I see no value in that.

[newville]

@simonkeys @Tillsten @reneeotten how about H2 for Model and H3 for the others? That renders a nested (and collapsable) set of headings. HN is mostly semantic, right?

(yes, that means I managed to run jupyter lab!)

[simonkeys]

Using H tags, even H2 and H3, doesn't work great for me personally because I generally use the headings to form an outline of the whole document. The chosen headings for the fit report wouldn't necessarily show up at the correct outline level for a particular document.

Accordingly, I would vote for caption tags, maybe plus strong for aesthetics.

However, I can always just monkeypatch for my own purposes, so I'm happy to implement the consensus of the devs.

[eendebakpt]

The problem @simonkeys mentions in the first post is bothering us as the tags generated by lmfit show in our table of contents. So I support changing the h2/h3 headers into caption (or any other way of preventing the lmfit output adding entries in the table of contents).

[newville]

Hm, if I understand correctly, the whole point of _repr_html_() is for Jupyter and its derivatives to be able to display HTML formatted representations. H1, H2, ... are the way to specify section headings with HTML. Using "<b>" or "<strong>" are about formatting, not about hierarchy. I think we want semantic tags.

But the complaints here are that people do not like the choices that JupyterLab makes with the HTML.

Well, what does JupyterLab want? Do they have guidance on what _html_repr_() should produce? It seems perfectly reasonable to me that the representations should start with "<H2>Minimizer Result</H2>" or "<H2>Model Result</H2>", and then use H3, etc for sub-sections.

I'm kind of baffled by this. If people don't like how JupyterLab is rendering HTML representations, why are we even generating them?

[eendebakpt]

Hm, if I understand correctly, the whole point of _repr_html_() is for Jupyter and its derivatives to be able to display HTML formatted representations. H1, H2, ... are the way to specify section headings with HTML. Using "<b>" or "\<strong\>" are about formatting, not about hierarchy. I think we want semantic tags.

This is exactly the point. I would rather not have sematic tags. The current representation of the fitting results looks good, but I want to define my own semantic interpretation. If I want a section (like h2 or h3), I can add that myself. In the current implementation showing an lmfit result adds the h2, and I cannot remove it.

Well, what does JupyterLab want? Do they have guidance on what _html_repr_() should produce? It seems perfectly reasonable to me that the representations should start with "<H2>Minimizer Result</H2>" or "<H2>Model Result</H2>", and then use H3, etc for sub-sections.

This is not about what JupyterLab wants, but what a user wants from the lmfit result representation. A single h2 with h3 subsections is indeed reasonable (and better than the current representation with multiple h2 tags). But not having header tags at all (or configurable) would be my first choice.

[newville]

@eendebakpt I think "What does Jupyter / JupyterLab want" is relevant for the _repr_html_() methods for lmfit. I do not know that we can guess what "the user" wants with much accuracy ;).

For some reason, JupyterLab chooses to render some portions of the output (stdout and stderr? Not sure, not relevant here) that are in HTML in a couple of different places: Headings go in two places or something. Like, we're not doing anything but sending HTML to stdout. So, yeah, what is Jupyter expecting to do with that?

Honestly, I do not see many examples of _repr_html_() methods. I do agree that tables can/should have captions and have no objection to using those. But, if JupyterLab is doing something with "<H2>" that you don't like, does it also do something with
"<caption>" too?

Anyway, if there are guidelines about what Jupyter and JupyterLab want, that would be helpful to know.

I think another perfectly reasonable option is to remove _repr_html_. Like, you are saying you do not like its results. ;)

[eendebakpt]

@newville

• I guess you are looking for a specification or guidelines on what the output of _repr_html_() should be. There is some documentation, but it does not mention any good practices of when to include a header and when not, e.g.

  • https://jupyterlab.readthedocs.io/en/latest/user/toc.html.
  • https://ipython.readthedocs.io/en/stable/config/integrating.html

  I agree that it matters what jupyter expects with respect to structure (e.g. _repr_html_() should return valid html, and _repr_png_() a png image), but my claim is that jupyter should have no opinion on the contents of the data. In this case: _repr_html() with or without headers are both valid in jupyter and it is up to the method implementing the representation to make a choice.

• An overview of some nice representation methods can be found here: https://matthewrocklin.com/blog/2019/07/04/html-repr
  (personally I use mainly the xarray and sympy ones)

• There is an option to generate a header in the notebook, but not in the table of contents:

<h2 class="jp-toc-ignore">A H2 header, but no toc entry</h2>
Yeah!
<h2>Same style, but not we do have a toc entry</h2>
Cool!

@simonkeys Perhaps that would be an option to consider for #883

• I hope enough consensus can be reached to change the current status. Any of the options mentioned so far (except removal of the repr_html) would be an improvement for me.

[newville]

@eendebakpt thanks for those links. I think that jupyterlab TOC doc is exactly what we need to know.

My instinct would be to have the HTML reprs use an H2 tag that is intended for the TOC, but maybe with that heading labeled tweaked to be "Minimizer Result" or "Model Result". Using H3 with "toc-ignore" for everything else seems OK to me.

Isn't that sort of what you would want if one asks for the formatted results within a JupyterLab session?

I guess the next question is whether for ModelResult to also include a matplotlib-PNG generated from .plot or a plot using plotly.

[reneeotten]

My instinct would be to have the HTML reprs use an H2 tag that is intended for the TOC, but maybe with that heading labeled tweaked to be "Minimizer Result" or "Model Result". Using H3 with "toc-ignore" for everything else seems OK to me.

I agree, that sounds like the best solution. Perhaps just using "Fit Result" as the header so that it would apply to both Minimizer and Model class?

I guess the next question is whether for ModelResult to also include a matplotlib-PNG generated from .plot or a plot using plotly.

well, that's another discussion and thus perhaps something for another PR, at least if you want to keep things to the point and easily merge-able? ;)

[newville]

@reneeotten @simonkeys Yes, I agree. Let's get something that works and merge in....

And, if a suggestion for an automated plot is coming from me, I'm definitely willing to say it is not necessary! If someone wants to add that, it can come later...

[newville]

@simonkeys @reneeotten @Tillsten I think that #902 addresses these issues, using both table captions and class="jp-toc-ignore" where appropriate.

[simonkeys]

Great, thanks for the work on this!