Add 'Electrophysiology' page. #55

JoeZiminski · 2024-05-15T11:26:55Z

As discussed in #53, this PR adds a 'Electrophysiology' page and changes 'Data Analysis' to 'Behaviour' page. The electrophysiology page has added:

getting started - quick overview on resources to get started
examples - python and matlab code from researchers in the building for analysing ephys data
community - some links to various community resources
resources - list of interesting reading.

Reviewing:

I think the changes to the website-generation machinery are quite minimal, bar adding sphinx-gallery
I am interested to know if the text reads okay and example pipeline blurbs are clear. It might easiest to build to read for review.
@neuroinformatics-unit/behaviour this rearranges the way the behaviour section is displayed, let me know if you have any feedback / don't like it.

TODO

just realised one of the example pipeline repos are private, will follow up.

adamltyson

This is excellent @JoeZiminski, I think this will be incredibly useful for SWC (and other) researchers. I have some minor comments, and I think the roadmap should be moved out of this website.

docs/source/behav_at_swc/index.md

docs/source/index.md

docs/source/ephys_at_swc/index.md

docs/source/behav_at_swc/index.md

docs/source/ephys_at_swc/getting_started.md

docs/source/ephys_at_swc/roadmap.md

docs/source/ephys_at_swc/community.md

docs/source/ephys_at_swc/index.md

JoeZiminski · 2024-05-17T12:24:54Z

Hey @adamltyson I've added a very rough version of the example pipelines just to get an idea of the layout. Basically the type of pipeline in use in the building that it would be nice to include roughly falls into three categories:

spikeinterface scripts for preprocessing and sorting (suitable for sphinx-gallery)
full project repos that use python (often spikeinterface) but are fully integrated with multimodal analysis (e.g. Steve's pipeline)
(2) but for MATLAB pipelines (e.g. Mateo's pipeline)

It seems important to include all three categories on the website (e.g. even though we recommend SI there is a lot of valuable methods in those matlab pipelines). But i'm not sure how best to display the information in a non-clunky way. ATM the front page of the sphinx gallery contains preprocessing/sorting scripts in the usual gallery way, but also links to two other pages detailing python and matlab repos. Do you think this works? (and the general approach, giving some backround of named researchers, their setups and studies).

JoeZiminski · 2024-05-17T20:08:12Z

Ah, now understand that incredibly annoying error! Build was failing with unpinned sphinx with the below error. I was lazily ignoring warnings because it was building fine locally, I think because I had contaminated my env with ipykernel. However CI was failing with the below error:

Notebook error:
NoSuchKernel in ephys_at_swc/gallery/sara_mederos.ipynb:
No such kernel named python3

and was fixed by pinning to sphinx<7.2. The problem was I ignored the following error in latest shpinx:

WARNING: multiple files found for the document "ephys_at_swc/gallery/sara_mederos": ['ephys_at_swc/gallery/sara_mederos.py', 'ephys_at_swc/gallery/sara_mederos.ipynb', 'ephys_at_swc/gallery/sara_mederos.rst']
Use '/Users/joeziminski/git_repos/HowTo/docs/source/ephys_at_swc/gallery/sara_mederos.ipynb' for the build.

so it was trying to use .ipynb for the build by default in later sphinx. In earlier sphinx, it uses the .rst by default:

WARNING: multiple files found for the document "ephys_at_swc/gallery/sara_mederos": ['ephys_at_swc/gallery/sara_mederos.py', 'ephys_at_swc/gallery/sara_mederos.ipynb', 'ephys_at_swc/gallery/sara_mederos.rst']
Use '/Users/joeziminski/git_repos/HowTo/docs/source/ephys_at_swc/gallery/sara_mederos.rst' for the build.

and fixing the warning with the below means it builds okay on most recent sphinx 🎉. I guess a good lesson to always fix warnings first even if they seem unrelated to the error!

exclude_patterns = [
   ...
    "ephys_at_swc/gallery/*.ipynb",
    "ephys_at_swc/gallery/*.py",
]

adamltyson · 2024-05-20T09:48:54Z

@JoeZiminski I think the general idea is good, with the obvious caveats that:

Everyone is properly credited
They're happy with this all being online, and they're aware that it's fully public etc.

…dd to gitignore.

adamltyson

Hey @JoeZiminski this is excellent. I've reviewed afresh, sorry if anything I say contradicts my original reviews!

I've left some comments, but I think the main thing to check is conflicts with recent changes to the behavioural guides.

Also, as you mentioned something needs to be done about the private link, either get it made public (ideal) or remove that section for now.

Assuming everyone mentioned is happy for this to be shared, I think this is nearly ready to go!

adamltyson · 2024-09-17T09:40:14Z

.github/workflows/docs_build_and_deploy.yml

@@ -27,7 +27,9 @@ jobs:
    needs: linting
    runs-on: ubuntu-latest
    steps:
-      - uses: neuroinformatics-unit/actions/build_sphinx_docs@v2
+      - uses: neuroinformatics-unit/actions/build_sphinx_docs@main


Is there a reason to use main here? I think in general we want to be using the latest released version (in case a rogue PR goes through). @v2 will resolve to the latest v2.x release.

adamltyson · 2024-09-17T09:40:39Z

.github/workflows/docs_build_and_deploy.yml

-      - uses: neuroinformatics-unit/actions/build_sphinx_docs@v2
+      - uses: neuroinformatics-unit/actions/build_sphinx_docs@main
+        with:
+          python-version: 3.11


Any chance 3.12 works? Just gives us slightly more runway before it needs changing.

adamltyson · 2024-09-17T09:40:46Z

.github/workflows/docs_build_and_deploy.yml

@@ -37,6 +39,6 @@ jobs:
    if: github.event_name == 'push' && github.ref_name == 'main'
    runs-on: ubuntu-latest
    steps:
-      - uses: neuroinformatics-unit/actions/deploy_sphinx_docs@v2
+      - uses: neuroinformatics-unit/actions/deploy_sphinx_docs@main


See above re action version.

adamltyson · 2024-09-17T09:43:24Z

docs/source/behav_at_swc/HPC-module-SLEAP.md

-::: {dropdown} Note on managed Linux desktops
-:color: info
-:icon: info
-
-The SWC's IT team offers managed desktop computers equipped with a Linux image. These machines are already part of SWC's trusted domain and have direct access to SLURM, the HPC modules, and the SWC filesystem.
-
-If you have access to one of these desktops,
-you can skip the pre-requisite steps.
-You may simply open a terminal, type `module load SLEAP`,
-and start using SLEAP directly, as you would on any local
-Linux machine. All SLEAP commands should work as expected,
-including `sleap-label` for launching the GUI.
-
-That said, you may still want to offload GPU-intensive tasks to an HPC node (e.g. because the desktop's GPU is not powerful enough or because you need to run many jobs in parallel). In that case, you may
-still want to read the sections on [model training](sleap-training)
-and [inference](sleap-inference).
-:::
-
-(access-to-the-hpc-cluster)=


Is there a reason why this was removed in this PR?

adamltyson · 2024-09-17T09:44:32Z

docs/source/behav_at_swc/HPC-module-SLEAP.md

@@ -53,17 +33,15 @@ $ module avail
 ...
 SLEAP/2023-03-13
 SLEAP/2023-08-01
-SLEAP/2024-08-14


This PR seems to be undoing some changes made since it was first raised.

adamltyson · 2024-09-17T09:54:05Z

docs/source/ephys_at_swc/community.md

+This site is maintained by the
+[NeuroInformatics Unit](https://neuroinformatics.dev/).
+Please don't hesitate to contact us
+(in particular Joe Ziminski) on Slack with any questions or feedback.


Maybe link to the Zulip here so any external people can get in touch too?

adamltyson · 2024-09-17T09:55:05Z

docs/source/ephys_at_swc/example_pipelines/examples/README.rst

Maybe link each reference to the lab to the official SWC page for that lab? Then there's an easy way to people to find email addresses etc?

adamltyson · 2024-09-17T09:56:01Z

docs/source/ephys_at_swc/example_pipelines/examples/README.rst

+striatum. He is using NeuroPixels 2.0 and SpikeGLX with a SpikeInterface-based
+pipeline; the code can be found `here <https://github.com/stephenlenzi/npix_lse>`__.
+
+Python Scripts


It's not immediately clear how the scripts section is different to the other parts above.

adamltyson · 2024-09-17T09:56:51Z

docs/source/ephys_at_swc/example_pipelines/examples/sara_mederos.py

+
+Sara Mederos (Hofer Lab) performs chronic electrophysiological
+recordings from subcortical and cortical areas using 4-shank
+Neuropixels 2.0 probes (Open Ephys). Recordings are conducted


Just to make it super obvious what this means to anyone new to the field

Suggested change

Neuropixels 2.0 probes (Open Ephys). Recordings are conducted

Neuropixels 2.0 probes (acquired using [Open Ephys](https://open-ephys.org/)). Recordings are conducted

adamltyson · 2024-09-17T09:58:08Z

docs/source/ephys_at_swc/example_pipelines/matlab_examples.rst

+Mateo Velez-Fort (Margie Lab) investigates the integration of visual
+and vestibular information in the visual cortex with the
+Margrie lab's 'Translocator' setup. They use
+Neuropixels 2.0 (SpikeGLX) for acute recordings from the


As above, just to make it super clear.

Suggested change

Neuropixels 2.0 (SpikeGLX) for acute recordings from the

Neuropixels 2.0 (acquired using [SpikeGLX](https://github.com/billkarsh/SpikeGLX)) for acute recordings from the

JoeZiminski changed the title ~~Add 'Ephys at the SWC' page.~~ Add 'Electrophysiology at the SWC' page. May 15, 2024

JoeZiminski marked this pull request as draft May 15, 2024 11:27

JoeZiminski requested review from adamltyson and niksirbi May 15, 2024 11:28

adamltyson reviewed May 15, 2024

View reviewed changes

adamltyson reviewed May 16, 2024

View reviewed changes

docs/source/ephys_at_swc/index.md Outdated Show resolved Hide resolved

niksirbi mentioned this pull request May 20, 2024

Unpin sphinx version in docs requirements neuroinformatics-unit/movement#183

Merged

JoeZiminski mentioned this pull request Jun 10, 2024

Unpin sphinx<7.2 neuroinformatics-unit/datashuttle#397

Closed

JoeZiminski and others added 19 commits September 16, 2024 17:41

Reorganise to ephys and behav at swc pages.

13d3341

Adding content rough draft.

db0a966

Continue to tidy up, small renaming of sections.

c03e8b3

Tidy up roadmap.

e730904

Tidy up getting started.

cc13b67

Fix linting.

e8af5a9

Add examples and sphinx gallery foundation.

ed05f82

Working on sphinx-gallerys.

05cf311

Remove / move roadmap.

add5646

fix linting.

738a754

Add pandoc dependency.

b24b7ee

Try updating github actions for building docs with sphinx.

4dd133f

try uninstall jupyter.

428078c

try ipython dependency.

3e45da1

add ipykernel

fece33a

copy all movement deps.

2c83212

Fix links and example build error.

329ae9d

fix broken link.

308617b

Remove sphinx autodoc as uneeded, probably isn't related to build error.

efadb92

JoeZiminski and others added 23 commits September 16, 2024 17:43

Fix linting.

070cc9a

Remove part about community moving to SI.

5dfc547

Reorder resources / getting started, rename gallery output page and a…

48b7727

…dd to gitignore.

Further reorganise getting started page.

e7daf12

Tidy up getting started, fix external links.

ec38d0b

Small tidy ups, try and remove auto_examples folder.

7905495

Tidy up resources.

a69b1d9

add velez-fort.

d11705e

Add Adrei first draft.

8ec92c5

Add note on papers.

83d9169

Small rewrites for community, getting started, resources.

d8fad36

Restructure sphinx examples.

24c022f

Start to update example entries.

f7e2e5e

Tidy up the blurbs

ccb2e14

Add per-shank CAR to sara example.

b121a9f

update gitignore.

f5fa953

removed auto_examples_ephys from git tracking.

7e90bb1

Update getting started.

aa22edf

Typos and grammer fixes, just resources to do.

4013d11

Final typo fixes.

735a172

Fix warnings.

e7aa661

Small tidy ups.

8840bb4

Check links.

1502263

JoeZiminski requested a review from adamltyson September 16, 2024 17:00

JoeZiminski marked this pull request as ready for review September 16, 2024 17:00

Ignore private repo link.

4521390

JoeZiminski requested a review from a team September 16, 2024 17:39

JoeZiminski changed the title ~~Add 'Electrophysiology at the SWC' page.~~ Add 'Electrophysiology' page. Sep 16, 2024

JoeZiminski force-pushed the add-ephys-at-swc branch from 93cb103 to 1502263 Compare September 16, 2024 17:50

adamltyson requested changes Sep 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 'Electrophysiology' page. #55

Add 'Electrophysiology' page. #55

JoeZiminski commented May 15, 2024 •

edited

Loading

adamltyson left a comment

JoeZiminski commented May 17, 2024

JoeZiminski commented May 17, 2024

adamltyson commented May 20, 2024 •

edited

Loading

adamltyson left a comment

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

adamltyson Sep 17, 2024

	Neuropixels 2.0 probes (Open Ephys). Recordings are conducted
	Neuropixels 2.0 probes (acquired using [Open Ephys](https://open-ephys.org/)). Recordings are conducted

	Neuropixels 2.0 (SpikeGLX) for acute recordings from the
	Neuropixels 2.0 (acquired using [SpikeGLX](https://github.com/billkarsh/SpikeGLX)) for acute recordings from the

Add 'Electrophysiology' page. #55

Are you sure you want to change the base?

Add 'Electrophysiology' page. #55

Conversation

JoeZiminski commented May 15, 2024 • edited Loading

adamltyson left a comment

Choose a reason for hiding this comment

JoeZiminski commented May 17, 2024

JoeZiminski commented May 17, 2024

adamltyson commented May 20, 2024 • edited Loading

adamltyson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeZiminski commented May 15, 2024 •

edited

Loading

adamltyson commented May 20, 2024 •

edited

Loading