feat: Documentation update for newer versions Jupyter notebook

CHANGELOG.md

@@ -8,6 +8,7 @@
 - Enable basic DependaBot scanning for pip packages and GitHub actions used in CI
 - Add CI support for release proces with [release.yaml](.github/workflows/release.yml) workflow
   - Release documentation is updated acrodingly in [RELEASING.md](./RELEASING.md)
+- Update README.md with installation methods on newer Jupyter and JupyterLab releases
 
 ## 0.21.0
 

Dockerfile.jupyter

@@ -30,19 +30,15 @@ RUN if [ "$dev_mode" = "true" ]; then \
       cd sparkmagic && pip install -e . && cd ../ ; \
     else pip install sparkmagic ; fi
 
-# Jupyter extensions changed in >7.x.x
-# For now (workaround), let's pin to 6 to avoid breaking things
-# xref: https://github.com/jupyter-incubator/sparkmagic/issues/825
-RUN pip install notebook==6.5.5
 
 RUN mkdir /home/$NB_USER/.sparkmagic
 COPY sparkmagic/example_config.json /home/$NB_USER/.sparkmagic/config.json
 RUN sed -i 's/localhost/spark/g' /home/$NB_USER/.sparkmagic/config.json
-RUN jupyter nbextension enable --py --sys-prefix widgetsnbextension
+RUN pip install ipywidgets
 RUN jupyter-kernelspec install --user $(pip show sparkmagic | grep Location | cut -d" " -f2)/sparkmagic/kernels/sparkkernel
 RUN jupyter-kernelspec install --user $(pip show sparkmagic | grep Location | cut -d" " -f2)/sparkmagic/kernels/pysparkkernel
 RUN jupyter-kernelspec install --user $(pip show sparkmagic | grep Location | cut -d" " -f2)/sparkmagic/kernels/sparkrkernel
-RUN jupyter serverextension enable --py sparkmagic
+RUN jupyter server extension enable --py sparkmagic
 
 USER root
 RUN chown $NB_USER /home/$NB_USER/.sparkmagic/config.json

README.md

@@ -48,20 +48,42 @@ See the [Sending Local Data to Spark notebook](examples/Send%20local%20data%20to
 
 ## Installation
 
+### Jupyter Notebook 7.x / JupyterLab 3.x
+
 1. Install the library
 
         pip install sparkmagic
 
 2. Make sure that ipywidgets is properly installed by running
 
-        jupyter nbextension enable --py --sys-prefix widgetsnbextension 
+        pip install ipywidgets 
+
+3. (Optional) Install the wrapper kernels. Do `pip show sparkmagic` and it will show the path where `sparkmagic` is installed at. `cd` to that location and do:
+        jupyter-kernelspec install sparkmagic/kernels/sparkkernel
+        jupyter-kernelspec install sparkmagic/kernels/pysparkkernel
+        jupyter-kernelspec install sparkmagic/kernels/sparkrkernel
+
+4. (Optional) Modify the configuration file at ~/.sparkmagic/config.json. Look at the [example_config.json](sparkmagic/example_config.json)
+
+5. (Optional) Enable the server extension so that clusters can be programatically changed:
+
+        jupyter server extension enable --py sparkmagic
+
+### Jupyter Notebook 5.2 or earlier / JupyterLab 1 or 2
+
+1. Install the library
+
+        pip install sparkmagic
+
+2. Make sure that ipywidgets is properly installed by running
 
-3. If you're using JupyterLab, you'll need to run another command:
+        jupyter nbextension enable --py --sys-prefix widgetsnbextension 
+
+3. If you're using JupyterLab 1 or 2, you'll need to run another command:
 
         jupyter labextension install "@jupyter-widgets/jupyterlab-manager"
 
 4. (Optional) Install the wrapper kernels. Do `pip show sparkmagic` and it will show the path where `sparkmagic` is installed at. `cd` to that location and do:
-
         jupyter-kernelspec install sparkmagic/kernels/sparkkernel
         jupyter-kernelspec install sparkmagic/kernels/pysparkkernel
         jupyter-kernelspec install sparkmagic/kernels/sparkrkernel