8.0 Release plans

[jasongrout]

We are well on our way to an 8.0 release. The milestone is at https://github.com/jupyter-widgets/ipywidgets/milestone/30

I've tagged backwards-incompatible issues/PRs in the 8.0 milestone with a label. I consider those criticial blockers for 8.0. Other issues/PRs could come in an 8.1 if needed since they are not backwards incompatible. Please let us know if you think otherwise, or if you feel an issue/PR has been mislabeled.

Since JupyterLab 2.0 (shipping by Feb 2020) requires ipywidgets 8.0 (and not ipywidgets 7.x), I suggest we have a tentative release date of the first or second week of February.

Update: We have now released 8.0.0rc2 (5 Aug 2022), which we anticipate being the final RC release. Items still to do:

• Review PRs in the cookiecutter to help authors porting widgets to 8.0:
  • Automatically set public path widget-cookiecutter#103
  • Update packaging to use more modern python practices widget-cookiecutter#87
  • Note in our migration guide these PRs to help widget authors incorporate their changes as they upgrade their packages
• Add release on GH
• Add blog post

[jasongrout]

FYI, JLab just released an RC, so it will probably be about a week until JLab 2.0.

[vidartf]

Let's try to do a call (this week) to evaluate the backwards incompatible changes that have already been merged, and whether any need to / should be reverted before 8.0. Given that 8.0 is now a requisite for lab 2.0, we might want to give it a double check, as my initial understanding would be that we would have longer to evaluate the consequences of those changes.

[jasongrout]

How about Wednesday at 8am PT?

[jasongrout]

I think some of the more controversial changes (like the rewrite of the data layer) has been postponed past 8.0 given the tight timeline we are on. My guess is that we won't have to revert anything, it's more a question of how much more to merge in.

[vidartf]

How about Wednesday at 8am PT?

I have a conflicting meeting at that time, and I'm not sure the reset of @jupyter-widgets/jupyter-widgets has noticed :)

[vidartf]

I think some of the more controversial changes (like the rewrite of the data layer) has been postponed past 8.0 given the tight timeline we are on.

I think you are right, at least on the Python/spec side of things (that matters the most w.r.t. stability).

[jasongrout]

@jupyter-widgets/jupyter-widgets, @zerline, and anyone else interested in participating: I propose that we have a weekly meeting (30-45 minutes) to maintain momentum towards releasing 8.0 in the next month or so.

Anyone interested in joining, please mark times that work for you at http://whenisgood.net/9eftj5s

[jasongrout]

CC @ibdafna @mbektasbbg - please see the message above about a weekly meeting

[jasongrout]

I think some of the more controversial changes (like the rewrite of the data layer) has been postponed past 8.0 given the tight timeline we are on. My guess is that we won't have to revert anything, it's more a question of how much more to merge in.

For completeness - we released a new version of the jlab widget manager compatible with ipywidgets 7.5 and jupyterlab 2.0, so the time pressure is off for shipping ipywidgets 8.0 with jlab 2.0.

[vidartf]

Note that if we were to have it bi-weekly, we could have it in the same slot as the Voila call (alternating weeks).

[jasongrout]

Since there are no times that work for the three people responding to the survey, let's try @vidartf's idea, and try it in the Voila meeting timeslot in the off-weeks from the voila meeting.

Looking at https://voila-dashboards.github.io/, that would mean the first meeting would be 9:30-10:30 PDT Monday, March 30, 2020.

Would that work better for @zerline or @jupyter-widgets/jupyter-widgets?

[zerline]

Ok for me

[vidartf]

Note that the voila meet is specified in central EU timezone that switched to DST this weekend, but @jasongrout specified the call in PDT above, so I'm confused what time we are operating with here.

[jasongrout]

I'm happy to leave the call anchored to a EU timezone. What time is the Voila call if we anchor to Europe? I can meet either now or in an hour, which I think is what the two possibilities are.

[jtpio]

We can use the Voila Weekly Whereby room for these calls: https://whereby.com/voila-weekly

Time is 17:30 CET: https://voila-dashboards.github.io

[jasongrout]

@jupyter-widgets/jupyter-widgets and others that want to help with ipywidgets 8: @ianhi is setting up a meeting for us to restart planning for ipywidgets 8:

If you're interesting joining a ipywidgets meeting please fill out this whenisgood http://whenisgood.net/hbbqe7h

The broad goals of this meeting are to talk briefly about high level goals and to figure out the state of an 8.0 release.

See https://hackmd.io/V2rlXrDQRzKH3lyTgh0ZIw for full details.

[jasongrout]

As @willingc requested: FYI, @willingc, @MSeal, @captainsafia - we are arranging an ipywidgets 8.0 planning meeting soon. See the comment just above this for details.

[ianhi]

Hi all,

Following up on this. Unfortunately there was no single time that worked for everyone. So the time that was settled on (https://gitter.im/jupyter-widgets/Lobby?at=600ae1028816425540e2ac46) is:
Tuesday, Feb 2nd, at 3 pm UTC (10 am EST)

@jasongrout how do links for things like this get generated? Zoom links? Generated by whoever?

[jasongrout]

We can use the same Zoom as the JupyterLab weekly meetings: Zoom link

[willingc]

@jasongrout @ianhi It would be great if we can record this meeting for those unable to attend. Thanks for organizing.

[jtpio]

Thanks all for organizing.

Here is a link to easily get the time and date of the meeting for your time zone: Tuesday, Feb 2nd, at 3 pm UTC

(also added to the hackmd)

[jasongrout]

FYI, the meeting starts in about 10.5 hours (Local time)

Zoom
Agenda

[jasongrout]

Thanks everyone who participated in the meeting today. It was really great to see everyone.

I've added notes at a new team-compass repo: jupyter-widgets/team-compass#1

[jasongrout]

I've released ipywidgets 8.0.0a3, widgetsnbextension 4.0.0a1, jupyterlab_widgets 2.0.0a0. This works in JupyterLab 3.

[vidartf]

The current plan is a first beta release of 8.0 next week, with an RC phase starting around the beginning of December. If no major disruptions are encountered, the plan is then to do a final release of 8.0 in early/mid January.

[vidartf]

Due to some delays in the release schedules, the RC phase seems likely to not start until early January, with a subsequent shift of the final release to late January.

[frenzymadness]

I'm testing RPM packages in Fedora with IPython 8.0.0rc1, ipywidgets 8.0.0b1, and widgetsnbextension 4.0.0b1 and so far I've discovered only one minor test incompatibility in nbclient, reported: jupyter/nbclient#191

[martinRenou]

Commented on the nbclient issue: jupyter/nbclient#191 (comment)

[ruomad]

Any news on final RC/release plans ? The target seems to have shifted somehow
Thanks for your work !

[vidartf]

It was initially delayed by a last-minute request to resolve #3111. A suggested solve to that issue was merged in #3195. The final piece currently missing for the RC is the attempt to make that implementation be backwards compatible on the protocol level (i.e. avoid needing to bump the widget messaging protocol to a new major version) in #3394.

[ruomad]

OK thanks a lot for these insights.
My concern for the time being is that voila isn't compatible with ipywidgets 8 so far and they wouldn't publish a compatible version until final release.
Any chance that your last piece would eliminate the need of a new version of voila ?

[vidartf]

@ruomad The current plan for the remaining steps are:

1. Release an RC of 8.0. That includes all the python and JS packages.
2. Finish any documentation work related to the 8.0 changes. This includes a migration guide for both widget users, and for third party widget library authors (Add migration guide for 8.0 #3405).
3. Test and fix other packages and libraries to add support for ipywidgets 8. This can of course not be exhaustive, but ideally as many libraries as possible would be included in this. This way, we would have final input on the changes, and a chance to use that input to re-evaluate and/or adjust them.
4. Do the final release.

Regarding your question about voila, I'm not familiar with this issue. Could you cross-link any issues detailing the problem you are referring to?

[martinRenou]

My concern for the time being is that voila isn't compatible with ipywidgets 8 so far and they wouldn't publish a compatible version until final release.
Any chance that your last piece would eliminate the need of a new version of voila ?

We'll make sure to publish a Voila release compatible with ipywidgets 8 soon after ipywidgets 8 is out.

[ruomad]

Thanks a lot for your detailed reply ! As for the problem with voila I did not see any reported issue but I will try to post one, hopefully with a minimal example. Basically I have a cell running a form with a few widgets, works allright with voila and ipywidgets 7, but voila doesn’t display anything with ipywidgets 8.0.0b1. I haven’t seen any hint in logs but I’ll try to lower log level to debug

…

-- Sent from Canary (https://canarymail.io)

On Friday, Mar 11, 2022 at 3:41 PM, Vidar Tonaas Fauske ***@***.*** ***@***.***)> wrote: @ruomad (https://github.com/ruomad) The current plan for the remaining steps are: Release an RC of 8.0. That includes all the python and JS packages. Finish any documentation work related to the 8.0 changes. This includes a migration guide for both widget users, and for third party widget library authors (#3405 (#3405)). Test and fix other packages and libraries to add support for ipywidgets 8. This can of course not be exhaustive, but ideally as many libraries as possible would be included in this. This way, we would have final input on the changes, and a chance to use that input to re-evaluate and/or adjust them. Do the final release. Regarding your question about voila, I'm not familiar with this issue. Could you cross-link any issues detailing the problem you are referring to? — Reply to this email directly, view it on GitHub (#2750 (comment)), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AECOISHQMPVISCPMYHCB7ATU7NLQBANCNFSM4KK4DUCQ). Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub). You are receiving this because you were mentioned.Message ID: ***@***.***>

[ruomad]

I opened an issue on this topic, for the reference
#3413

[jasongrout]

I've released 8.0.0rc0

[jasongrout]

Short update: There are two PRs I'm waiting on reviews/merges before releasing the (hopefully final) 8.0 RC for final testing.

[jasongrout]

8.0.0rc2 is now released! Please test.

[frenzymadness]

I did some testing in Fedora with all packages depending on ipywidgets and I see only one problem in nbclient test suite:

=================================== FAILURES ===================================
______________ test_run_all_notebooks[JupyterWidgets.ipynb-opts7] ______________

input_name = 'JupyterWidgets.ipynb', opts = {'kernel_name': 'python'}

    @pytest.mark.parametrize(
        ["input_name", "opts"],
        [
            ("Other Comms.ipynb", dict(kernel_name="python")),
            ("Clear Output.ipynb", dict(kernel_name="python")),
            ("Empty Cell.ipynb", dict(kernel_name="python")),
            ("Factorials.ipynb", dict(kernel_name="python")),
            ("HelloWorld.ipynb", dict(kernel_name="python")),
            ("Inline Image.ipynb", dict(kernel_name="python")),
            (
                "Interrupt.ipynb",
                dict(kernel_name="python", timeout=1, interrupt_on_timeout=True, allow_errors=True),
            ),
            ("JupyterWidgets.ipynb", dict(kernel_name="python")),
            ("Skip Exceptions with Cell Tags.ipynb", dict(kernel_name="python")),
            ("Skip Exceptions.ipynb", dict(kernel_name="python", allow_errors=True)),
            ("Skip Execution with Cell Tag.ipynb", dict(kernel_name="python")),
            ("SVG.ipynb", dict(kernel_name="python")),
            ("Unicode.ipynb", dict(kernel_name="python")),
            ("UnicodePy3.ipynb", dict(kernel_name="python")),
            ("update-display-id.ipynb", dict(kernel_name="python")),
            ("Check History in Memory.ipynb", dict(kernel_name="python")),
        ],
    )
    def test_run_all_notebooks(input_name, opts):
        """Runs a series of test notebooks and compares them to their actual output"""
        input_file = os.path.join(current_dir, 'files', input_name)
        input_nb, output_nb = run_notebook(input_file, opts, notebook_resources())
>       assert_notebooks_equal(input_nb, output_nb)

nbclient/tests/test_client.py:333: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

expected = {'cells': [{'cell_type': 'code', 'execution_count': 1, 'metadata': {}, 'outputs': [{'data': {'application/vnd.jupyter....del_name': 'LayoutModel', 'state': {}}}, 'version_major': 2, 'version_minor': 0}}}, 'nbformat': 4, 'nbformat_minor': 2}
actual = {'cells': [{'cell_type': 'code', 'metadata': {'execution': {'iopub.status.busy': '2022-08-09T19:28:42.615167Z', 'iopub...oltip': None, 'value': 'Hello World'}}}, 'version_major': 2, 'version_minor': 0}}}, 'nbformat': 4, 'nbformat_minor': 2}

    def assert_notebooks_equal(expected, actual):
        expected_cells = expected['cells']
        actual_cells = actual['cells']
        assert len(expected_cells) == len(actual_cells)
    
        for expected_cell, actual_cell in zip(expected_cells, actual_cells):
            # Uncomment these to help debug test failures better
            # from pprint import pprint
            # pprint(expected_cell)
            # pprint(actual_cell)
            expected_outputs = expected_cell.get('outputs', [])
            actual_outputs = actual_cell.get('outputs', [])
            normalized_expected_outputs = list(map(normalize_output, expected_outputs))
            normalized_actual_outputs = list(map(normalize_output, actual_outputs))
>           assert normalized_expected_outputs == normalized_actual_outputs
E           AssertionError: assert [{'data': {'application/vnd.jupyter.widget-view+json': {'model_id': '<MODEL_ID>',\n                                                        'version_major': 2,\n                                                        'version_minor': 0},\n           'text/plain': 'LabelvalueHelloWorld'},\n  'output_type': 'display_data'}] == [{'data': {'application/vnd.jupyter.widget-view+json': {'model_id': '<MODEL_ID>',\n                                                        'version_major': 2,\n                                                        'version_minor': 0},\n           'text/plain': 'LabelvalueHelloWorld'},\n  'execution_count': 1,\n  'output_type': 'execute_result'}]
E             At index 0 diff: {'data': {'application/vnd.jupyter.widget-view+json': {'model_id': '<MODEL_ID>', 'version_major': 2, 'version_minor': 0}, 'text/plain': 'LabelvalueHelloWorld'}, 'output_type': 'display_data'} != {'output_type': 'execute_result', 'data': {'text/plain': 'LabelvalueHelloWorld', 'application/vnd.jupyter.widget-view+json': {'version_major': 2, 'version_minor': 0, 'model_id': '<MODEL_ID>'}}, 'execution_count': 1}
E             Full diff:
E               [
E                {'data': {'application/vnd.jupyter.widget-view+json': {'model_id': '<MODEL_ID>',
E                                                                       'version_major': 2,
E                                                                       'version_minor': 0},
E                          'text/plain': 'LabelvalueHelloWorld'},
E             +   'output_type': 'display_data'},
E             -   'execution_count': 1,
E             -   'output_type': 'execute_result'},
E               ]

nbclient/tests/test_client.py:280: AssertionError

But it should be easy to fix.

[jasongrout]

I did some testing in Fedora with all packages depending on ipywidgets and I see only one problem in nbclient test suite:

@frenzymadness - thank you very much for testing. Yes, this is an expected difference. It has to do with 8.0 switching to use a more modern and consistent way of displaying widgets (rather than displaying widgets as a side effect). This change was made in #2021

[jasongrout]

I've opened an issue in nbclient to track this: jupyter/nbclient#244

[jasongrout]

FYI, here is a draft paragraph about migration from @vidartf for a potential 8.0 blog post announcement:

In this release of ipywidgets, some of the JavaScript code has been refactored in order to improve
the experience for third-party widget library authors. The main change is that the widget_manager
property on the Widget class in @jupyter-widgets/base is now typed as an interface, which
prevents typings from JupyterLab to bleed through. This should prevent the need for extension
authors to update their extension every time JupyterLab performs a major version update, which
can significantly reduce churn. If you are extending the base widget manager class, it now lives
in the new package @jupyter-widgets/base-manager.