Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't perform ICA cleaning on raw data by default when epochs are present #926

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions docs/source/v1.9.md.inc
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,22 @@

### :new: New features & enhancements

- Added number of subject to `sub-average` report (#902, #910 by @SophieHerbst)
- Added number of subjects to `sub-average` report (#902, #910 by @SophieHerbst)
- The type annotations in the default configuration file are now easier to read: We
replaced `Union[X, Y]` with `X | Y` and `Optional[X]` with `X | None`. (#908, #911 by @hoechenberger)
- Added a new configuration option [`spatial_filter_raw`][mne_bids_pipeline._config.spatial_filter_raw]
to control whether to create cleaned raw data via ICA or SSP. Previously, we'd only clean epochs.
(#840, #926 by @larsoner and @hoechenberger)

### :warning: Behavior changes

- All ICA HTML reports have been consolidated in the standard subject `*_report.html`
file instead of producing separate files (#899 by @larsoner).
- Changed default for `source_info_path_update` to `None`. In `_04_make_forward.py`
- When using ICA or SSP with resting-state data, we now automatically produce cleaned raw data files. This
behavior can be controlled via the new [`spatial_filter_raw`][mne_bids_pipeline._config.spatial_filter_raw]
configuration option. (#840, #926 by @larsoner and @hoechenberger)
Comment on lines +16 to +18
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The introduction of writing raw data was actually in 1.6 / #840. So the behavior change here is actually to stop writing files by default. So maybe we should leave the default as True for backward compat. Then this PR is just to add the ability to have False | None.

Personally I think True is the most reasonable default anyway. If you ever want to re-epoch your data (I often do!) then you'll want those preprocessed raw files. And you can cross-check individual runs against the produced epochs, it's more complete to have raw data report at each stage, etc. even though it makes the report longer.

- Changed default for [`source_info_path_update`][mne_bids_pipeline._config.source_info_path_update]
to `None`. In `_04_make_forward.py`
and `_05_make_inverse.py`, we retrieve the info from the file from which
the `noise_cov` is computed (#919 by @SophieHerbst)
- The [`depth`][mne_bids_pipeline._config.depth] parameter doesn't accept `None`
Expand Down
7 changes: 7 additions & 0 deletions mne_bids_pipeline/_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -1138,6 +1138,13 @@
ICA fitting – this file will need to be updated!
"""

spatial_filter_raw: bool | None = None
"""
Whether to clean the raw data with the spatial filter. If `True`, creates cleaned data
for all raw data. If `False`, no raw data is written. If `None`, creates
cleaned raw data for resting-state and empty-room runs if present.
"""

min_ecg_epochs: Annotated[int, Ge(1)] = 5
"""
Minimal number of ECG epochs needed to compute SSP projectors.
Expand Down
9 changes: 9 additions & 0 deletions mne_bids_pipeline/_config_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -658,3 +658,12 @@ def _proj_path(
suffix="proj",
check=False,
)


def _which_spatial_filter_raw(*, config: SimpleNamespace) -> tuple[str]:
if config.spatial_filter_raw:
return ("runs", "noise", "rest")
elif config.spatial_filter_raw is None:
return ("noise", "rest")
else:
return ()
7 changes: 6 additions & 1 deletion mne_bids_pipeline/_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
"_bids_kwargs": ("get_task",),
"_import_data_kwargs": ("get_mf_reference_run",),
"get_runs": ("get_runs_all_subjects",),
"_which_spatial_filter_raw": ("spatial_filter_raw",),
}


Expand Down Expand Up @@ -144,6 +145,10 @@ def __init__(self, force_empty=None):
for call in ast.walk(func):
if not isinstance(call, ast.Call):
continue
# e.g., which = _which_spatial_filter_raw(config=config)
if isinstance(call.func, ast.Name):
for attr in _EXTRA_FUNCS.get(call.func.id, ()):
self._add_step_option(step, attr)
for keyword in call.keywords:
if not isinstance(keyword.value, ast.Attribute):
continue
Expand Down Expand Up @@ -226,7 +231,7 @@ def __init__(self, force_empty=None):
if isinstance(keyword.value, ast.Name):
key = f"{where}:{keyword.value.id}"
if key in _MANUAL_KWS:
for option in _MANUAL_KWS[f"{where}:{keyword.value.id}"]:
for option in _MANUAL_KWS[key]:
self._add_step_option(step, option)
continue
raise RuntimeError(f"{where} cannot handle Name {key=}")
Expand Down
3 changes: 3 additions & 0 deletions mne_bids_pipeline/steps/preprocessing/_08a_apply_ica.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from mne_bids import BIDSPath

from ..._config_utils import (
_which_spatial_filter_raw,
get_runs_tasks,
get_sessions,
get_subjects,
Expand Down Expand Up @@ -276,6 +277,7 @@ def main(*, config: SimpleNamespace) -> None:
parallel, run_func = parallel_func(
apply_ica_raw, exec_params=config.exec_params
)
which = _which_spatial_filter_raw(config=config)
logs += parallel(
run_func(
cfg=get_config(
Expand All @@ -294,6 +296,7 @@ def main(*, config: SimpleNamespace) -> None:
config=config,
subject=subject,
session=session,
which=which,
)
)
save_logs(config=config, logs=logs)
3 changes: 3 additions & 0 deletions mne_bids_pipeline/steps/preprocessing/_08b_apply_ssp.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

from ..._config_utils import (
_proj_path,
_which_spatial_filter_raw,
get_runs_tasks,
get_sessions,
get_subjects,
Expand Down Expand Up @@ -181,6 +182,7 @@ def main(*, config: SimpleNamespace) -> None:
parallel, run_func = parallel_func(
apply_ssp_raw, exec_params=config.exec_params
)
which = _which_spatial_filter_raw(config=config)
logs += parallel(
run_func(
cfg=get_config(
Expand All @@ -199,6 +201,7 @@ def main(*, config: SimpleNamespace) -> None:
config=config,
subject=subject,
session=session,
which=which,
)
)
save_logs(config=config, logs=logs)
Loading