Support Jupyerlab 4.x

[willarmiros]

I tried to follow this guide from Jupyerlab to upgrade our jupyerlab version from 3.x to 4.x and hopefully fix CI failures like this one: https://github.com/protectai/nbdefense-jupyter/actions/runs/5490860882/job/14871335551?pr=16

Overall I tried to keep all the changes from the upgrade script as-is, but I did remove some boilerplate for testing that the script added and some new GH workflows it tried to introduce. When trying to run make install-dev after using the script, I got the below error that seemed to come from changes to pyproject.toml.

Opening this issue to track progress on upgrading to jupyterlab 4.x. After performing the upgrade, we should test:

• All make targets are working as expected
• All GH workflows are passing (including the release workflow)
• The extension works as expected in a real jupyter notebook (sanity check)

Full stack trace

make install-dev
pip uninstall nbdefense_jupyter -y
Found existing installation: nbdefense_jupyter 0.dev0
Uninstalling nbdefense_jupyter-0.dev0:
  Successfully uninstalled nbdefense_jupyter-0.dev0
pip install -e .[dev]
Obtaining file:///Users/williamarmiros/open-source/willarmiros/nbdefense-jupyter
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [245 lines of output]
      INFO:hatch_jupyter_builder.utils:Running jupyter-builder
      INFO:hatch_jupyter_builder.utils:Building with hatch_jupyter_builder.npm_builder
      INFO:hatch_jupyter_builder.utils:With kwargs: {'build_cmd': 'install:extension', 'npm': ['jlpm'], 'source_dir': 'src', 'build_dir': 'nbdefense_jupyter/labextension'}
      INFO:hatch_jupyter_builder.utils:Installing build dependencies with npm.  This may take a while...
      INFO:hatch_jupyter_builder.utils:> /private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/overlay/bin/jlpm install
      ➤ YN0000: ┌ Resolution step
      ➤ YN0002: │ @jupyterlab/services@npm:7.0.3 doesn't provide react (pc77de), requested by @jupyterlab/settingregistry
      ➤ YN0002: │ @jupyterlab/testutils@npm:4.0.3 doesn't provide typescript (p05ec7), requested by @jupyterlab/testing
      ➤ YN0060: │ nbdefense-jupyter@workspace:. provides jest (pe2690) with version 29.6.1, which doesn't satisfy what ts-jest requests
      ➤ YN0060: │ nbdefense-jupyter@workspace:. provides typescript (p51b1a) with version 5.0.4, which doesn't satisfy what ts-jest requests
      ➤ YN0002: │ nbdefense-jupyter@workspace:. doesn't provide webpack (p82f71), requested by css-loader
      ➤ YN0002: │ nbdefense-jupyter@workspace:. doesn't provide webpack (pced70), requested by source-map-loader
      ➤ YN0002: │ nbdefense-jupyter@workspace:. doesn't provide webpack (p3ef88), requested by style-loader
      ➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
      ➤ YN0000: └ Completed
      ➤ YN0000: ┌ Fetch step
      ➤ YN0000: └ Completed
      ➤ YN0000: ┌ Link step
      ➤ YN0000: └ Completed in 0s 248ms
      ➤ YN0000: Done with warnings in 0s 622ms
      INFO:hatch_jupyter_builder.utils:> /private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/overlay/bin/jlpm run install:extension
      node_modules/@types/codemirror/index.d.ts(553,39): error TS2304: Cannot find name 'DocumentAndElementEventHandlersEventMap'.
      node_modules/@types/codemirror/index.d.ts(555,48): error TS2304: Cannot find name 'DocumentAndElementEventHandlersEventMap'.
      node_modules/@types/codemirror/index.d.ts(563,40): error TS2304: Cannot find name 'DocumentAndElementEventHandlersEventMap'.
      node_modules/@types/codemirror/index.d.ts(565,48): error TS2304: Cannot find name 'DocumentAndElementEventHandlersEventMap'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3085,14): error TS2300: Duplicate identifier 'LibraryManagedAttributes'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3096,13): error TS2717: Subsequent property declarations must have the same type.  Property 'a' must be of type 'DetailedHTMLProps<AnchorHTMLAttributes<HTMLAnchorElement>, HTMLAnchorElement>', but here has type 'DetailedHTMLProps<AnchorHTMLAttributes<HTMLAnchorElement>, HTMLAnchorElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3097,13): error TS2717: Subsequent property declarations must have the same type.  Property 'abbr' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3098,13): error TS2717: Subsequent property declarations must have the same type.  Property 'address' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3099,13): error TS2717: Subsequent property declarations must have the same type.  Property 'area' must be of type 'DetailedHTMLProps<AreaHTMLAttributes<HTMLAreaElement>, HTMLAreaElement>', but here has type 'DetailedHTMLProps<AreaHTMLAttributes<HTMLAreaElement>, HTMLAreaElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3100,13): error TS2717: Subsequent property declarations must have the same type.  Property 'article' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3101,13): error TS2717: Subsequent property declarations must have the same type.  Property 'aside' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3102,13): error TS2717: Subsequent property declarations must have the same type.  Property 'audio' must be of type 'DetailedHTMLProps<AudioHTMLAttributes<HTMLAudioElement>, HTMLAudioElement>', but here has type 'DetailedHTMLProps<AudioHTMLAttributes<HTMLAudioElement>, HTMLAudioElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3103,13): error TS2717: Subsequent property declarations must have the same type.  Property 'b' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3104,13): error TS2717: Subsequent property declarations must have the same type.  Property 'base' must be of type 'DetailedHTMLProps<BaseHTMLAttributes<HTMLBaseElement>, HTMLBaseElement>', but here has type 'DetailedHTMLProps<BaseHTMLAttributes<HTMLBaseElement>, HTMLBaseElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3105,13): error TS2717: Subsequent property declarations must have the same type.  Property 'bdi' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3106,13): error TS2717: Subsequent property declarations must have the same type.  Property 'bdo' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3107,13): error TS2717: Subsequent property declarations must have the same type.  Property 'big' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3108,13): error TS2717: Subsequent property declarations must have the same type.  Property 'blockquote' must be of type 'DetailedHTMLProps<BlockquoteHTMLAttributes<HTMLQuoteElement>, HTMLQuoteElement>', but here has type 'DetailedHTMLProps<BlockquoteHTMLAttributes<HTMLQuoteElement>, HTMLQuoteElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3109,13): error TS2717: Subsequent property declarations must have the same type.  Property 'body' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLBodyElement>, HTMLBodyElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLBodyElement>, HTMLBodyElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3110,13): error TS2717: Subsequent property declarations must have the same type.  Property 'br' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLBRElement>, HTMLBRElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLBRElement>, HTMLBRElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3111,13): error TS2717: Subsequent property declarations must have the same type.  Property 'button' must be of type 'DetailedHTMLProps<ButtonHTMLAttributes<HTMLButtonElement>, HTMLButtonElement>', but here has type 'DetailedHTMLProps<ButtonHTMLAttributes<HTMLButtonElement>, HTMLButtonElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3112,13): error TS2717: Subsequent property declarations must have the same type.  Property 'canvas' must be of type 'DetailedHTMLProps<CanvasHTMLAttributes<HTMLCanvasElement>, HTMLCanvasElement>', but here has type 'DetailedHTMLProps<CanvasHTMLAttributes<HTMLCanvasElement>, HTMLCanvasElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3113,13): error TS2717: Subsequent property declarations must have the same type.  Property 'caption' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3114,13): error TS2717: Subsequent property declarations must have the same type.  Property 'cite' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3115,13): error TS2717: Subsequent property declarations must have the same type.  Property 'code' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3116,13): error TS2717: Subsequent property declarations must have the same type.  Property 'col' must be of type 'DetailedHTMLProps<ColHTMLAttributes<HTMLTableColElement>, HTMLTableColElement>', but here has type 'DetailedHTMLProps<ColHTMLAttributes<HTMLTableColElement>, HTMLTableColElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3117,13): error TS2717: Subsequent property declarations must have the same type.  Property 'colgroup' must be of type 'DetailedHTMLProps<ColgroupHTMLAttributes<HTMLTableColElement>, HTMLTableColElement>', but here has type 'DetailedHTMLProps<ColgroupHTMLAttributes<HTMLTableColElement>, HTMLTableColElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3118,13): error TS2717: Subsequent property declarations must have the same type.  Property 'data' must be of type 'DetailedHTMLProps<DataHTMLAttributes<HTMLDataElement>, HTMLDataElement>', but here has type 'DetailedHTMLProps<DataHTMLAttributes<HTMLDataElement>, HTMLDataElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3119,13): error TS2717: Subsequent property declarations must have the same type.  Property 'datalist' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLDataListElement>, HTMLDataListElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLDataListElement>, HTMLDataListElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3120,13): error TS2717: Subsequent property declarations must have the same type.  Property 'dd' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3121,13): error TS2717: Subsequent property declarations must have the same type.  Property 'del' must be of type 'DetailedHTMLProps<DelHTMLAttributes<HTMLModElement>, HTMLModElement>', but here has type 'DetailedHTMLProps<DelHTMLAttributes<HTMLModElement>, HTMLModElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3122,13): error TS2717: Subsequent property declarations must have the same type.  Property 'details' must be of type 'DetailedHTMLProps<DetailsHTMLAttributes<HTMLDetailsElement>, HTMLDetailsElement>', but here has type 'DetailedHTMLProps<DetailsHTMLAttributes<HTMLDetailsElement>, HTMLDetailsElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3123,13): error TS2717: Subsequent property declarations must have the same type.  Property 'dfn' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3124,13): error TS2717: Subsequent property declarations must have the same type.  Property 'dialog' must be of type 'DetailedHTMLProps<DialogHTMLAttributes<HTMLDialogElement>, HTMLDialogElement>', but here has type 'DetailedHTMLProps<DialogHTMLAttributes<HTMLDialogElement>, HTMLDialogElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3125,13): error TS2717: Subsequent property declarations must have the same type.  Property 'div' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLDivElement>, HTMLDivElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLDivElement>, HTMLDivElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3126,13): error TS2717: Subsequent property declarations must have the same type.  Property 'dl' must be of type 'DetailedHTMLProps<HTMLAttributes<HTMLDListElement>, HTMLDListElement>', but here has type 'DetailedHTMLProps<HTMLAttributes<HTMLDListElement>, HTMLDListElement>'.
      ....
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3268,13): error TS2717: Subsequent property declarations must have the same type.  Property 'symbol' must be of type 'SVGProps<SVGSymbolElement>', but here has type 'SVGProps<SVGSymbolElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3269,13): error TS2717: Subsequent property declarations must have the same type.  Property 'text' must be of type 'SVGTextElementAttributes<SVGTextElement>', but here has type 'SVGProps<SVGTextElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3270,13): error TS2717: Subsequent property declarations must have the same type.  Property 'textPath' must be of type 'SVGProps<SVGTextPathElement>', but here has type 'SVGProps<SVGTextPathElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3271,13): error TS2717: Subsequent property declarations must have the same type.  Property 'tspan' must be of type 'SVGProps<SVGTSpanElement>', but here has type 'SVGProps<SVGTSpanElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3272,13): error TS2717: Subsequent property declarations must have the same type.  Property 'use' must be of type 'SVGProps<SVGUseElement>', but here has type 'SVGProps<SVGUseElement>'.
      node_modules/@types/react-dom/node_modules/@types/react/index.d.ts(3273,13): error TS2717: Subsequent property declarations must have the same type.  Property 'view' must be of type 'SVGProps<SVGViewElement>', but here has type 'SVGProps<SVGViewElement>'.
      node_modules/@types/react/ts5.0/index.d.ts(3194,19): error TS2320: Interface 'ElementClass' cannot simultaneously extend types 'Component<any, {}, any>' and 'Component<any, {}, any>'.
        Named property 'context' of types 'Component<any, {}, any>' and 'Component<any, {}, any>' are not identical.
      node_modules/@types/react/ts5.0/index.d.ts(3194,19): error TS2320: Interface 'ElementClass' cannot simultaneously extend types 'Component<any, {}, any>' and 'Component<any, {}, any>'.
        Named property 'props' of types 'Component<any, {}, any>' and 'Component<any, {}, any>' are not identical.
      node_modules/@types/react/ts5.0/index.d.ts(3194,19): error TS2320: Interface 'ElementClass' cannot simultaneously extend types 'Component<any, {}, any>' and 'Component<any, {}, any>'.
        Named property 'refs' of types 'Component<any, {}, any>' and 'Component<any, {}, any>' are not identical.
      node_modules/@types/react/ts5.0/index.d.ts(3202,14): error TS2300: Duplicate identifier 'LibraryManagedAttributes'.
      src/handler.ts(28,45): error TS2345: Argument of type 'unknown' is not assignable to parameter of type 'TypeError'.
      src/utils/editorErrorsUtils.tsx(106,13): error TS2322: Type 'import("/Users/williamarmiros/open-source/willarmiros/nbdefense-jupyter/node_modules/@types/react-dom/node_modules/@types/react/index").ReactPortal | undefined' is not assignable to type 'React.ReactPortal | undefined'.
        Type 'import("/Users/williamarmiros/open-source/willarmiros/nbdefense-jupyter/node_modules/@types/react-dom/node_modules/@types/react/index").ReactPortal' is not assignable to type 'React.ReactPortal'.
          Types of property 'children' are incompatible.
            Type 'import("/Users/williamarmiros/open-source/willarmiros/nbdefense-jupyter/node_modules/@types/react-dom/node_modules/@types/react/index").ReactNode' is not assignable to type 'React.ReactNode'.
              Type '{}' is not assignable to type 'ReactNode'.
      src/utils/editorErrorsUtils.tsx(148,29): error TS2339: Property 'markText' does not exist on type 'Text'.
      src/widgets/NBDefenseWidget.tsx(23,11): error TS2554: Expected 0 arguments, but got 1.
      Traceback (most recent call last):
        File "/Users/williamarmiros/miniconda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 167, in prepare_metadata_for_build_editable
          hook = backend.prepare_metadata_for_build_editable
      AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/Users/williamarmiros/miniconda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/williamarmiros/miniconda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/williamarmiros/miniconda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 176, in prepare_metadata_for_build_editable
          whl_basename = build_hook(metadata_directory, config_settings)
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/overlay/lib/python3.10/site-packages/hatchling/build.py", line 78, in build_editable
          return os.path.basename(next(builder.build(wheel_directory, ['editable'])))
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/overlay/lib/python3.10/site-packages/hatchling/builders/plugin/interface.py", line 150, in build
          build_hook.initialize(version, build_data)
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/normal/lib/python3.10/site-packages/hatch_jupyter_builder/plugin.py", line 87, in initialize
          raise e
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/normal/lib/python3.10/site-packages/hatch_jupyter_builder/plugin.py", line 82, in initialize
          build_func(self.target_name, version, **build_kwargs)
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/normal/lib/python3.10/site-packages/hatch_jupyter_builder/utils.py", line 115, in npm_builder
          run([*npm_cmd, "run", build_cmd], cwd=str(abs_path))
        File "/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/normal/lib/python3.10/site-packages/hatch_jupyter_builder/utils.py", line 229, in run
          return subprocess.check_call(cmd, **kwargs)
        File "/Users/williamarmiros/miniconda/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/private/var/folders/h_/krvxw1n979352w3lz6kmhpdc0000gn/T/pip-build-env-trp69c37/overlay/bin/jlpm', 'run', 'install:extension']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
make: *** [install-dev] Error 1

[badarahmed]

We need to continue to support JL 3 for foreseeable future as most Jupyterlab as a service offerings do not support v4 yet and likely won't for the next 3-6 months. JL 4 support needs to be in addition to JL 3. It might need code in two separate branches.