diff --git a/CHANGELOG.md b/CHANGELOG.md index b977bad..2f3d8fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,40 @@ Versions are in format: `[MAJOR.MINOR.BUGFIX]` Dates are in format: `YYYY-MM-DD` +## [0.2.2] - 2022-08-09 + +Various improvements to the logic of the Data Visualization module and Dialog-based form components. + +### Fixed + +- Fixed a bug where multiple dialog windows could be opened for dialog-based components. A useRef was used to track + the dialog's open state to prevent this. + +- Fixed a bug in Data Visualization where changing between plots did not reset all the plotting variables. + +- Fixed a bug where additional hover data permitted the selection of variables that were already dedicated to X/Y axis, + causing a doubling effect to occur. This has been fixed twofold: + - Variables that are already dedicated to X/Y are disabled in the "additional hover data" section. + - Selecting a variable for X/Y that is already in "additional hover data" will cause it to be removed from the latter. + +- Fixed a bug in Data Visualization Swarmplot where changing the orientation field did not change said orientation. + +### Added + +- Help sections have been added for the first two steps in the Data Visualization module. The main plotting page itself + hasn't been given any help buttons for the time being. + +- The main plotting page's typography has been wrapped in a CardHeader component for a more aesthetically-pleasing look + and consistency with the previous steps. + +### Changed + +- Hovering over datapoints in Data Visualization now has several changes: + - Subject and session are explicitly stated in the hoverdata instead of the previous ambiguous "ID" label + - Instead of X and Y, the variable names are now explicitly stated in the hoverlabel + +- Swarmplot's continuous var axis is now better scaled so that the maximum/minimum values do not appear at the edges. + --- ## [0.2.1] - 2022-08-08 @@ -14,11 +48,14 @@ Additional information added to README and reformatted RunImportModule visuals. ### Fixed -- Fixed a bug where saving a DataPar.json file in Define Parameters resulting in snackbar feedback that did not respect OS-specific filepath delimiters +- Fixed a bug where saving a DataPar.json file in Define Parameters resulting in snackbar feedback that did not respect + OS-specific filepath delimiters -- Fixed a bug in MRIViewSettings where the maximum value exceeded the possible indexable value of MRI slices by 1 due to Javascript's zero-based indexing vs MNI space indexing discrepancy, causing crashes. +- Fixed a bug in MRIViewSettings where the maximum value exceeded the possible indexable value of MRI slices by 1 due + to Javascript's zero-based indexing vs MNI space indexing discrepancy, causing crashes. -- Fixed a bug in StructureByParts where selecting a new value in the Select components did not trigger validation for that field. Changing any of the selects now forces validation. +- Fixed a bug in StructureByParts where selecting a new value in the Select components did not trigger validation for + that field. Changing any of the selects now forces validation. ### Added @@ -26,7 +63,8 @@ Additional information added to README and reformatted RunImportModule visuals. - Started adding some configuration for the auto-update feature in `package.json`. -- Modules that fail to complete will now give some feedback to the particular subjects/visits/sessions and the step where they did not complete. +- Modules that fail to complete will now give some feedback to the particular subjects/visits/sessions and the step + where they did not complete. ### Changed @@ -40,11 +78,13 @@ Important bugfixes and successful deployment on Windows 10. ### Fixed -- Fixed bugs in DataVisualization where previous graphical settings would not be reset if the user backtracked to load a new dataframe +- Fixed bugs in DataVisualization where previous graphical settings would not be reset if the user backtracked to load + a new dataframe - Fixed `calculateASLWorkload` to anticipate proper lock paths. Previously, it would early-exit due to issues with the session regex and a ES6 parsing error involving an if statement. -- Fixed `calculateStructuralWorkload` which made the mistake of using an async forEach loop, causing early return of the function. This has been fixed to use a traditional for-loop. +- Fixed `calculateStructuralWorkload` which made the mistake of using an async forEach loop, causing early return of + the function. This has been fixed to use a traditional for-loop. - Fixed a bug in which the StepRunImportModule component did not communicate the channel name when pausing/resuming/terminating @@ -54,13 +94,15 @@ Important bugfixes and successful deployment on Windows 10. - Added scripts for generating an Inno wizard installer for the program on Windows as an alternative to Squirrel -- Added a compiled node mode `win32-x64_lib.node` to backend. This should allow for circumventing the ntsuspend optional dep. +- Added a compiled node mode `win32-x64_lib.node` to backend. This should allow for circumventing the ntsuspend + optional dep. - Added legend settings to Scatterplot. Users can now alter legend symbol size, label fontsize, item spacing, etc. ### Changed -- Change the DrawerItems in GUIFrame such that the parent item of a nested structure will now be a ListItemButton which, when clicked, will expand/collapse the nested structure +- Change the DrawerItems in GUIFrame such that the parent item of a nested structure will now be a ListItemButton + which, when clicked, will expand/collapse the nested structure --- diff --git a/README.md b/README.md index 9358ec6..0171d22 100644 --- a/README.md +++ b/README.md @@ -238,7 +238,7 @@ Additional features to keep in mind: - [x] Create a Data Visualization Module. - [x] Add plot settings for plot legends (i.e. legend text fontsize, positioning, etc.) within the DataVisualization Module. - [x] Add more helpful information in the Process Studies module for when a study does not fully complete. -- [ ] Add help information on the steps within the Data Visualization Module. +- [x] Add help information on the steps within the Data Visualization Module. - [ ] Add plot settings for renaming axis main labels within the Data Visualization Module. - [ ] Add Multiprocessing Capability to the Import Module as well. \*\* - [ ] Add a submodule to Process Studies where users can pin-point change the JSON sidecars of individual subjects/visits/sessions. diff --git a/package.json b/package.json index 4f47688..41bc802 100755 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "ExploreASLJS", "productName": "ExploreASLJS", - "version": "0.2.1", + "version": "0.2.2", "description": "A user interface for the analysis of arterial spin labeling (ASL) images", "repository": "https://github.com/MauricePasternak/ExploreASLJS", "main": ".webpack/main", diff --git a/src/assets/img/HelperImages/HelperImage__DataVizRerunPopulationModule.png b/src/assets/img/HelperImages/HelperImage__DataVizRerunPopulationModule.png new file mode 100644 index 0000000..f48a14f Binary files /dev/null and b/src/assets/img/HelperImages/HelperImage__DataVizRerunPopulationModule.png differ diff --git a/src/common/utilityFunctions/dataFrameFunctions.ts b/src/common/utilityFunctions/dataFrameFunctions.ts index 86c52e6..b055591 100755 --- a/src/common/utilityFunctions/dataFrameFunctions.ts +++ b/src/common/utilityFunctions/dataFrameFunctions.ts @@ -440,7 +440,7 @@ export function toNivoSwarmPlotDataSingle(df: DataFrame, id: string, value: stri export function toNivoSwarmPlotDataGroupBy( df: DataFrame, - id: string, + id: string | string[], value: string, group: string, ...other: string[] @@ -448,14 +448,20 @@ export function toNivoSwarmPlotDataGroupBy( console.log(`toNivoSwarmPlotDataGroupBy invoked with id: ${id}, value: ${value}, group: ${group}, other: ${other}`); const groups = df.getSeries(group).distinct().toArray(); + const iid = Array.isArray(id) ? id : [id]; const data = df - .subset([id, value, group, ...other]) + .subset([...iid, value, group, ...other]) .toArray() .map(obj => { const groupName = obj[group]; // const result = { ...lodashPick(obj, other), id: obj[id], value: obj[value], group: groupName }; // return result; - return { ...lodashPick(obj, other), id: obj[id], value: obj[value], group: groupName }; + return { + ...lodashPick(obj, other), + id: Array.isArray(id) ? `${obj[id[0]]}_${obj[id[1]]}` : obj[id], + value: obj[value], + group: groupName, + }; }); return [groups, data] as const; } diff --git a/src/components/AtomicSnackbarMessage.tsx b/src/components/AtomicSnackbarMessage.tsx index 5be2e4d..cec5775 100644 --- a/src/components/AtomicSnackbarMessage.tsx +++ b/src/components/AtomicSnackbarMessage.tsx @@ -88,12 +88,17 @@ function AtomicSnackbarMessage({ return message.map((msg, idx) => { const key = `AtomicSnackbarMessageLine_${idx}`; // Array of strings - if (typeof msg === "string") return /^\s+$/gm.test(msg) ?
: {msg}; + if (typeof msg === "string") + return /^\s+$/gm.test(msg) ?
: {msg}; // Array of SnackbarMessageType objects return msg; }); } else if (typeof message === "string") { - return /^\s+$/gm.test(message) ?
: {message}; + return /^\s+$/gm.test(message) ? ( +
+ ) : ( + {message} + ); } else { return message; } diff --git a/src/components/FormComponents/RHFFilepathDropzone.tsx b/src/components/FormComponents/RHFFilepathDropzone.tsx index 6ceed49..8ed1ace 100644 --- a/src/components/FormComponents/RHFFilepathDropzone.tsx +++ b/src/components/FormComponents/RHFFilepathDropzone.tsx @@ -162,6 +162,7 @@ function ControlledFilepathDropzone { + if (isDialogOpened.current) return; if (!dialogOptions) throw new Error("dialogOptions were not specified"); // Get the filepaths from the dialog and early exit if no files were selected + isDialogOpened.current = true; const { canceled, filePaths } = await api.invoke("Dialog:OpenDialog", dialogOptions); + isDialogOpened.current = false; if (canceled) return; // Filter the filepaths and update the innerVal state diff --git a/src/components/FormComponents/RHFFilepathTextfield.tsx b/src/components/FormComponents/RHFFilepathTextfield.tsx index 100139b..c575f4f 100644 --- a/src/components/FormComponents/RHFFilepathTextfield.tsx +++ b/src/components/FormComponents/RHFFilepathTextfield.tsx @@ -1,4 +1,4 @@ -import React, { useCallback, useState, useEffect } from "react"; +import React, { useCallback, useState, useEffect, useRef } from "react"; import TextField, { TextFieldProps } from "@mui/material/TextField"; import { Controller, FieldValues, Path } from "react-hook-form"; import { useDebouncedCallback } from "use-debounce"; @@ -57,7 +57,7 @@ export function ControlledFilepathTextField { + if (isDialogOpened.current) return; + isDialogOpened.current = true; const { canceled, filePaths } = await api.invoke("Dialog:OpenDialog", dialogOptions); + isDialogOpened.current = false; !canceled && handleChange(filePaths[0]); }; diff --git a/src/components/FormComponents/RHFInterDepFilepathTextfield.tsx b/src/components/FormComponents/RHFInterDepFilepathTextfield.tsx index bbc4d1a..def545a 100644 --- a/src/components/FormComponents/RHFInterDepFilepathTextfield.tsx +++ b/src/components/FormComponents/RHFInterDepFilepathTextfield.tsx @@ -5,7 +5,7 @@ import FormHelperText from "@mui/material/FormHelperText"; import IconButton from "@mui/material/IconButton"; import TextField, { TextFieldProps } from "@mui/material/TextField"; import { OpenDialogOptions } from "electron"; -import React, { useCallback, useEffect, useState } from "react"; +import React, { useCallback, useEffect, useRef, useState } from "react"; import { Controller, FieldValues, Path } from "react-hook-form"; import { useDebouncedCallback } from "use-debounce"; import { RHFFieldAndFieldStateType, RHFInterDepBaseProps, RHFTriggerType } from "../../common/types/formTypes"; @@ -64,6 +64,7 @@ export function InterDepControlledFilepathTextField< const [innerVal, setInnerVal] = useState(field.value); const hasError = !!fieldState.error; const errorMessage = hasError ? fieldState.error?.message : ""; + const isDialogOpened = useRef(false); // Setup default dialog options if a dialogOptions object was not provided and a button is to be presented if (!dialogOptions && includeButton) { @@ -171,7 +172,10 @@ export function InterDepControlledFilepathTextField< ); const handleDialogueClick = async () => { + if (isDialogOpened.current) return; + isDialogOpened.current = true; const { canceled, filePaths } = await api.invoke("Dialog:OpenDialog", dialogOptions); + isDialogOpened.current = false; !canceled && handleChange(filePaths[0]); }; diff --git a/src/pages/DataVisualization/DataVizPlots/EASLScatterplot.tsx b/src/pages/DataVisualization/DataVizPlots/EASLScatterplot.tsx index a33289c..616d91a 100644 --- a/src/pages/DataVisualization/DataVizPlots/EASLScatterplot.tsx +++ b/src/pages/DataVisualization/DataVizPlots/EASLScatterplot.tsx @@ -39,20 +39,23 @@ function EASLScatterplot() { const setMRIDataStats = useSetAtom(atomMRIDataStats); const [currentMRIViewSubject, setMRICurrentViewSubject] = useAtom(atomCurrentMRIViewSubject); const scatterplotSettings = useAtomValue(atomEASLScatterplotSettings); + + const SubjectSessionSpread = dataFrame.hasSeries("session") ? ["SUBJECT", "session"] : ["SUBJECT"]; + const data = dataVarsSchema.GroupingVar ? toNivoScatterPlotDataGroupBy( dataFrame, dataVarsSchema.XAxisVar, dataVarsSchema.YAxisVar, dataVarsSchema.GroupingVar, - "SUBJECT", + ...SubjectSessionSpread, ...dataVarsSchema.HoverVariables ) : toNivoScatterPlotDataSingle( dataFrame, dataVarsSchema.XAxisVar, dataVarsSchema.YAxisVar, - "SUBJECT", + ...SubjectSessionSpread, ...dataVarsSchema.HoverVariables ); @@ -71,7 +74,7 @@ function EASLScatterplot() { if (!(await api.path.filepathExists(popPath.path))) return; // console.log("handleLoadSubject -- found popPath: ", popPath.path); - const qCBFFiles = await api.path.glob(popPath.path, `qCBF_${subjectToLoad}_*.nii*`); + const qCBFFiles = await api.path.glob(popPath.path, `qCBF_${subjectToLoad}*.nii*`); if (qCBFFiles.length === 0) return; // console.log("handleLoadSubject -- found qCBFFiles: ", qCBFFiles); @@ -147,9 +150,20 @@ function EASLScatterplot() { return ( - ID: {data["SUBJECT" as "x"]} - X: {data.x} - Y: {data.y} + {SubjectSessionSpread.length === 1 ? ( + Subject/Visit: {data["SUBJECT" as "x"]} + ) : ( + <> + Subject/Visit: {data["SUBJECT" as "x"]} + Session: {data["session" as "x"]} + + )} + + {dataVarsSchema.XAxisVar}: {data.x} + + + {dataVarsSchema.YAxisVar}: {data.y} + {...dataVarsSchema.HoverVariables.map(varName => ( {varName}: {data[varName as "x"]} @@ -193,12 +207,15 @@ function EASLScatterplot() { legends: { text: { fontSize: scatterplotSettings.theme.legendTextFontSize, - } - } + }, + }, }} onClick={async ({ data }) => { if (!("SUBJECT" in data)) return; - await handleLoadSubject(data["SUBJECT" as "x"]); + const subjectToLoad = + "session" in data ? `${data["SUBJECT" as "x"]}_${data["session" as "y"]}` : data["SUBJECT" as "x"]; + console.log("Scatterplot trying to load in subject/session: ", subjectToLoad); + await handleLoadSubject(subjectToLoad); }} /> ); diff --git a/src/pages/DataVisualization/DataVizPlots/EASLSwarmplot.tsx b/src/pages/DataVisualization/DataVizPlots/EASLSwarmplot.tsx index bed3f52..4a67cc8 100644 --- a/src/pages/DataVisualization/DataVizPlots/EASLSwarmplot.tsx +++ b/src/pages/DataVisualization/DataVizPlots/EASLSwarmplot.tsx @@ -3,13 +3,14 @@ import Stack from "@mui/material/Stack"; import { useTheme } from "@mui/material/styles"; import Typography from "@mui/material/Typography"; import { ResponsiveSwarmPlotCanvas } from "@nivo/swarmplot"; -import { useAtomValue, useSetAtom, atom } from "jotai"; +import { useAtomValue, useSetAtom, atom, useAtom } from "jotai"; import React from "react"; import { toNivoSwarmPlotDataGroupBy, toNivoSwarmPlotDataSingle, } from "../../../common/utilityFunctions/dataFrameFunctions"; import { + atomCurrentMRIViewSubject, atomDataVizLoadSettings, atomDataVizSubsetDF, atomEASLSwarmplotSettings, @@ -23,6 +24,7 @@ import { niftiToNivoCoronal, niftiToNivoSagittal, } from "../../../common/utilityFunctions/nivoFunctions"; +import { getMinMaxCountSum } from "../../../common/utilityFunctions/arrayFunctions"; function EASLSwarmplot() { const { api } = window; @@ -32,13 +34,21 @@ function EASLSwarmplot() { const dataLoadSettings = useAtomValue(atomDataVizLoadSettings); const setMRIData = useSetAtom(atomOfAtomMRIData); const setMRIDataStats = useSetAtom(atomMRIDataStats); + const [currentMRIViewSubject, setMRICurrentViewSubject] = useAtom(atomCurrentMRIViewSubject); const swarmplotSettings = useAtomValue(atomEASLSwarmplotSettings); + + const [YMin, YMax, YCount, YSum] = getMinMaxCountSum(dataFrame.getSeries(dataVarsSchema.YAxisVar).toArray()); + const YMean = YSum / YCount; + + const SubjectSessionSpread = dataFrame.hasSeries("session") ? ["SUBJECT", "session"] : ["SUBJECT"]; + const [groups, data] = dataVarsSchema.XAxisVar ? toNivoSwarmPlotDataGroupBy( dataFrame, - "SUBJECT", + SubjectSessionSpread, dataVarsSchema.YAxisVar, dataVarsSchema.XAxisVar, + ...SubjectSessionSpread, ...dataVarsSchema.HoverVariables ) : toNivoSwarmPlotDataSingle( @@ -46,6 +56,7 @@ function EASLSwarmplot() { "SUBJECT", dataVarsSchema.YAxisVar, dataVarsSchema.XAxisVar, + ...SubjectSessionSpread, ...dataVarsSchema.HoverVariables ); @@ -53,11 +64,13 @@ function EASLSwarmplot() { console.log("EASLSwarmplot rendered with data:", data); async function handleLoadSubject(subjectToLoad: string) { + if (subjectToLoad === currentMRIViewSubject) return; // Early return if the subject is already loaded + const popPath = api.path.asPath(dataLoadSettings.StudyRootPath, "derivatives", "ExploreASL", "Population"); if (!(await api.path.filepathExists(popPath.path))) return; console.log("handleLoadSubject -- found popPath: ", popPath.path); - const qCBFFiles = await api.path.glob(popPath.path, `qCBF_${subjectToLoad}_*.nii*`); + const qCBFFiles = await api.path.glob(popPath.path, `qCBF_${subjectToLoad}*.nii*`); if (qCBFFiles.length === 0) return; console.log("handleLoadSubject -- found qCBFFiles: ", qCBFFiles); @@ -73,6 +86,7 @@ function EASLSwarmplot() { setMRIData([atom(axialData), atom(coronalData), atom(sagittalData)]); setMRIDataStats({ max: maximumValue, min: minimumValue }); + setMRICurrentViewSubject(subjectToLoad); } return ( @@ -88,6 +102,12 @@ function EASLSwarmplot() { from: "color", modifiers: [["darker", 0.6]], }} + valueScale={{ + type: "linear", + min: YMin - YMean * 0.1, + max: YMax + YMean * 0.1, + }} + layout={swarmplotSettings.plotLayout} size={swarmplotSettings.nodeSize} enableGridY={swarmplotSettings.enableGridY} enableGridX={swarmplotSettings.enableGridX} @@ -97,7 +117,7 @@ function EASLSwarmplot() { tickSize: swarmplotSettings.axisBottom.tickHeight, tickPadding: swarmplotSettings.axisBottom.tickLabelPadding, tickRotation: swarmplotSettings.axisBottom.tickLabelRotation, - legend: dataVarsSchema.XAxisVar, + legend: swarmplotSettings.plotLayout === "vertical" ? dataVarsSchema.XAxisVar : dataVarsSchema.YAxisVar, legendPosition: "middle", legendOffset: swarmplotSettings.axisBottom.axisLabelTextOffset, }} @@ -105,7 +125,7 @@ function EASLSwarmplot() { tickSize: swarmplotSettings.axisLeft.tickHeight, tickPadding: swarmplotSettings.axisLeft.tickLabelPadding, tickRotation: swarmplotSettings.axisLeft.tickLabelRotation, - legend: dataVarsSchema.YAxisVar, + legend: swarmplotSettings.plotLayout === "vertical" ? dataVarsSchema.YAxisVar : dataVarsSchema.XAxisVar, legendPosition: "middle", legendOffset: swarmplotSettings.axisLeft.axisLabelTextOffset, }} @@ -113,9 +133,20 @@ function EASLSwarmplot() { tooltip={({ data }) => ( - ID: {data.id} - X: {data.group} - Y: {data.value} + {SubjectSessionSpread.length === 1 ? ( + Subject/Visit: {data["SUBJECT" as "value"]} + ) : ( + <> + Subject/Visit: {data["SUBJECT" as "value"]} + Session: {data["session" as "value"]} + + )} + + {dataVarsSchema.XAxisVar}: {data.group} + + + {dataVarsSchema.YAxisVar}: {data.value} + {...dataVarsSchema.HoverVariables.map(varName => ( {varName}: {data[varName as "group"]} @@ -157,7 +188,12 @@ function EASLSwarmplot() { }, }} onClick={async ({ data }) => { - await handleLoadSubject(data["id"]); + console.log(data); + if (!("SUBJECT" in data)) return; + const subjectToLoad = + "session" in data ? `${data["SUBJECT" as "id"]}_${data["session" as "id"]}` : data["SUBJECT" as "id"]; + console.log("Swarmplot trying to load in subject/session: ", subjectToLoad); + await handleLoadSubject(subjectToLoad); }} /> ); diff --git a/src/pages/DataVisualization/DataVizSettings/PlotTypeSettings.tsx b/src/pages/DataVisualization/DataVizSettings/PlotTypeSettings.tsx index 84a77a7..ace8f7b 100644 --- a/src/pages/DataVisualization/DataVizSettings/PlotTypeSettings.tsx +++ b/src/pages/DataVisualization/DataVizSettings/PlotTypeSettings.tsx @@ -20,10 +20,9 @@ import { NivoGraphType } from "../../../common/types/DataVizSchemaTypes"; import { atomDataVizDFDTypes, atomNivoGraphDataVariablesSchema, - atomNivoGraphType + atomNivoGraphType, } from "../../../stores/DataFrameVisualizationStore"; - function PlotTypeSettings() { const [expanded, setExpanded] = React.useState(true); const dtypes = useAtomValue(atomDataVizDFDTypes); @@ -41,7 +40,15 @@ function PlotTypeSettings() { .map(([colName, dtype], colIdx) => { if (colName === "SUBJECT" || (dtype !== permittedType && permittedType !== "Any")) return null; return ( - + {colName} ); @@ -53,7 +60,9 @@ function PlotTypeSettings() { Plot Variables} - subheader={Define which columns should be plotted and the graph type} + subheader={ + Define which columns should be plotted and the graph type + } avatar={ @@ -80,7 +89,10 @@ function PlotTypeSettings() { fullWidth label="Graph Type" value={nivoGraphType} - onChange={e => setNivoGraphType(e.target.value as NivoGraphType)} + onChange={e => { + setNivoGraphType(e.target.value as NivoGraphType); + setNivoGraphDataSchema({ ...nivoGraphDataSchema, XAxisVar: "", YAxisVar: "", GroupingVar: "" }); + }} > {(["Scatterplot", "Swarmplot"] as NivoGraphType[]).map((gType, gTypeIdx) => { return ( @@ -99,7 +111,13 @@ function PlotTypeSettings() { label="X Axis" value={nivoGraphDataSchema.XAxisVar} onChange={e => { - setNivoGraphDataSchema({ ...nivoGraphDataSchema, XAxisVar: e.target.value as string }); + setNivoGraphDataSchema({ + ...nivoGraphDataSchema, + HoverVariables: nivoGraphDataSchema.HoverVariables + ? nivoGraphDataSchema.HoverVariables.filter(l => l !== e.target.value) + : [], + XAxisVar: e.target.value as string, + }); }} > {renderOptions(nivoGraphType === "Swarmplot" ? "Categorical" : "Continuous", "XAxisVar")} @@ -113,7 +131,13 @@ function PlotTypeSettings() { label="Y Axis" value={nivoGraphDataSchema.YAxisVar} onChange={e => { - setNivoGraphDataSchema({ ...nivoGraphDataSchema, YAxisVar: e.target.value as string }); + setNivoGraphDataSchema({ + ...nivoGraphDataSchema, + HoverVariables: nivoGraphDataSchema.HoverVariables + ? nivoGraphDataSchema.HoverVariables.filter(l => l !== e.target.value) + : [], + YAxisVar: e.target.value as string, + }); }} > {renderOptions("Continuous", "YAxisVar")} diff --git a/src/pages/DataVisualization/DataVizSettings/SwarmPlotVisualsSettings.tsx b/src/pages/DataVisualization/DataVizSettings/SwarmPlotVisualsSettings.tsx index bba1f2c..efcc174 100644 --- a/src/pages/DataVisualization/DataVizSettings/SwarmPlotVisualsSettings.tsx +++ b/src/pages/DataVisualization/DataVizSettings/SwarmPlotVisualsSettings.tsx @@ -115,12 +115,13 @@ function SwarmPlotVisualsSettings() { setEASLPlotSettings({ - path: "nodeSize", + path: "interSeriesGap", value: v as number, }) } diff --git a/src/pages/DataVisualization/PlotEASLMainView.tsx b/src/pages/DataVisualization/PlotEASLMainView.tsx index 29a72c2..2067512 100644 --- a/src/pages/DataVisualization/PlotEASLMainView.tsx +++ b/src/pages/DataVisualization/PlotEASLMainView.tsx @@ -1,5 +1,9 @@ +import Avatar from "@mui/material/Avatar"; import Box from "@mui/material/Box"; import Button from "@mui/material/Button"; +import Card from "@mui/material/Card"; +import CardHeader from "@mui/material/CardHeader"; +import Divider from "@mui/material/Divider"; import Grid from "@mui/material/Grid"; import Paper from "@mui/material/Paper"; import Skeleton from "@mui/material/Skeleton"; @@ -16,6 +20,7 @@ import EASLScatterplot from "./DataVizPlots/EASLScatterplot"; import EASLSwarmplot from "./DataVizPlots/EASLSwarmplot"; import PlotSettingsDrawer from "./DataVizSettings/PlotSettingsDrawer"; import MRIMultiView from "./MRIView/MRIMultiView"; +import ScatterPlotIcon from "@mui/icons-material/ScatterPlot"; function PlotEASLMainView() { const graphType = useAtomValue(atomNivoGraphType); @@ -45,7 +50,22 @@ function PlotEASLMainView() { }} > - Data Visualization + + Data Visualization} + subheader={ + + Plot an interactive chart and load CBF volumes from individual subjects/visits/sessions by clicking on data points + + } + avatar={ + + + + } + /> + + - - - } title={Specify Datatypes} subheader={ @@ -58,6 +53,16 @@ function StepClarifyDataTypes() { outright Ignored } + avatar={ + + + + } + action={ + + + + } /> diff --git a/src/pages/DataVisualization/StepDefineDataframeLoc.tsx b/src/pages/DataVisualization/StepDefineDataframeLoc.tsx index e450536..862cb59 100644 --- a/src/pages/DataVisualization/StepDefineDataframeLoc.tsx +++ b/src/pages/DataVisualization/StepDefineDataframeLoc.tsx @@ -1,15 +1,21 @@ +import CircleIcon from "@mui/icons-material/Circle"; +import FolderIcon from "@mui/icons-material/Folder"; +import Avatar from "@mui/material/Avatar"; import Box from "@mui/material/Box"; import Button from "@mui/material/Button"; +import Card from "@mui/material/Card"; +import CardContent from "@mui/material/CardContent"; +import CardHeader from "@mui/material/CardHeader"; +import Divider from "@mui/material/Divider"; import Grid from "@mui/material/Grid"; import List from "@mui/material/List"; import ListItem from "@mui/material/ListItem"; import ListItemIcon from "@mui/material/ListItemIcon"; import ListItemText from "@mui/material/ListItemText"; -import CircleIcon from "@mui/icons-material/Circle"; import Paper from "@mui/material/Paper"; import Typography from "@mui/material/Typography"; import { DataFrame, fromCSV, IDataFrame, ISeries } from "data-forge"; -import { useSetAtom, atom, useAtom } from "jotai"; +import { atom, useAtom, useSetAtom } from "jotai"; import React from "react"; import { SubmitErrorHandler, SubmitHandler, useForm } from "react-hook-form"; import { DataVizLoadDFSchema } from "../../common/schemas/DataVizLoadDFSchema"; @@ -25,6 +31,7 @@ import { YupResolverFactoryBase } from "../../common/utilityFunctions/formFuncti import RHFFilepathTextField from "../../components/FormComponents/RHFFilepathTextfield"; import RHFSelect, { RHFControlledSelectOption } from "../../components/FormComponents/RHFSelect"; import OutlinedGroupBox from "../../components/OutlinedGroupBox"; +import FabDialogWrapper from "../../components/WrapperComponents/FabDialogWrapper"; import { atomCurrentMRIViewSubject, atomDataVizCurrentStep, @@ -35,15 +42,9 @@ import { atomOfAtomMRIData, atomOfAtomsDataVizSubsetOperations, defaultNivoGraphDataVariablesSchema, - loadDataFrameDataVizDefaults, } from "../../stores/DataFrameVisualizationStore"; import { atomDataVizModuleSnackbar } from "../../stores/SnackbarStore"; -import Card from "@mui/material/Card"; -import CardHeader from "@mui/material/CardHeader"; -import Avatar from "@mui/material/Avatar"; -import CardContent from "@mui/material/CardContent"; -import FolderIcon from "@mui/icons-material/Folder"; -import Divider from "@mui/material/Divider"; +import HelpDataViz__StepDefineDataframeLoc from "../Help/HelpDataViz__StepDefineDataframeLoc"; const atlasesOptions: RHFControlledSelectOption[] = [ { label: "WholeBrain Grey Matter", value: "TotalGM" }, @@ -55,6 +56,15 @@ const atlasesOptions: RHFControlledSelectOption @@ -154,8 +164,6 @@ function StepDefineDataframeLoc() { const setCurrentMRIViewSubject = useSetAtom(atomCurrentMRIViewSubject); const setAtomsMRIData = useSetAtom(atomOfAtomMRIData); - // TODO: There needs to be help info on this page - const { control, handleSubmit } = useForm({ // defaultValues: loadDataFrameDataVizDefaults, defaultValues: dataVizLoadSettings, @@ -200,7 +208,8 @@ function StepDefineDataframeLoc() { // Otherwise, merge tmpIDF = innerJoin(tmpIDF, fromCSV(fetchDF.data, { dynamicTyping: true }).after(0), "SUBJECT"); } // End of for loop - EASLDF = new DataFrame(tmpIDF.toArray()); + // Drop Site column; it should be coming from a user's metadata + EASLDF = new DataFrame(tmpIDF.hasSeries("Site") ? tmpIDF.dropSeries("Site").toArray() : tmpIDF.toArray()); console.log("Step 'Define Runtime Envs' -- EASLDF: ", EASLDF.toArray()); @@ -222,7 +231,17 @@ function StepDefineDataframeLoc() { }); return; } - MetadataDF = fromCSV(fetchMetaDF.data, { dynamicTyping: true }); + // Remove columns that may cause issues when merging + const initialMetadataDF = fromCSV(fetchMetaDF.data, { dynamicTyping: true }); + const colsToDrop = []; + for (const colName of EASLColnamesNotPermittedInMetadata) { + if (initialMetadataDF.hasSeries(colName)) { + colsToDrop.push(colName); + } + } + MetadataDF = + colsToDrop.length > 0 ? new DataFrame(initialMetadataDF.dropSeries(colsToDrop).toArray()) : initialMetadataDF; + console.log("Step 'Define Runtime Envs' -- MetadataDF: ", MetadataDF.toArray()); } // End of if statement @@ -231,8 +250,7 @@ function StepDefineDataframeLoc() { */ let mergedDF: DataFrame = null; if (MetadataDF) { - // Sanity Check for merge column - // TODO: It may be worthwhile to any filter out invalid subjects that aren't present in rawdata... + // Sanity Check for main merge column existing if (!MetadataDF.getColumnNames().includes("SUBJECT")) { setDataVizSnackbar({ severity: "error", @@ -245,23 +263,63 @@ function StepDefineDataframeLoc() { }); return; } - mergedDF = new DataFrame(outerLeftJoin(EASLDF, MetadataDF, "SUBJECT").toArray()); + + // Sanity Check for non-null values in merge columns + let nullItems: number; + if (!MetadataDF.hasSeries("session")) { + nullItems = MetadataDF.getSeries("SUBJECT") + .filter(val => val == null) + .count(); + } else { + nullItems = MetadataDF.subset(["SUBJECT", "session"]) + .filter(row => row["SUBJECT"] == null || row["session"] == null) + .count(); + } + if (nullItems > 0) { + setDataVizSnackbar({ + severity: "error", + title: "Empty values found in the SUBJECT and/or session columns of the Metadata dataframe", + message: [ + `A proper merging of the ExploreASL and Metadata dataframes is impossible when there are empty values in the "SUBJECT" and/or "session" columns of the Metadata dataframe.`, + " ", + "Please remove rows with the empty values in this/these column(s) and try again.", + ], + }); + return; + } + + // Perform an outer-left join on the ExploreASL and Metadata dataframes; + // on SUBJECT and session if the latter is present; otherwise just on SUBJECT + mergedDF = new DataFrame( + outerLeftJoin( + EASLDF, + MetadataDF, + MetadataDF.hasSeries("session") ? ["SUBJECT", "session"] : "SUBJECT" + ).toArray() + ); } else { mergedDF = EASLDF; } - console.log("Step 'Define Runtime Envs' -- Merged Dataframe: ", mergedDF.toArray()); + console.log("Step 'Define Runtime Envs' -- Merged Dataframe: ", mergedDF.toString()); - if (mergedDF.count() === 0 && MetadataDF != null) { - setDataVizSnackbar({ - severity: "error", - title: "Invalid Merge with Metadata", - message: [ - "The ExploreASL and Metadata spreadsheet do not have any overlapping subjects.", - "Ensure that the Metadata spreadsheet contains a SUBJECT column with the same subjects as the ExploreASL spreadsheet.", - "Subject names are based on the subject folders located within the rawdata folder.", - ], - }); + if (mergedDF.count() === 0) { + MetadataDF != null + ? setDataVizSnackbar({ + severity: "error", + title: "Invalid Merge with Metadata", + message: [ + "The ExploreASL and Metadata spreadsheet do not have any overlapping subjects/sessions.", + "Ensure that the Metadata spreadsheet contains a SUBJECT column with the same subjects as the ExploreASL spreadsheet.", + `The Metadata spreadsheet can also contain a "session" column.`, + "Subject names are based on the subject folders located within the rawdata folder.", + ], + }) + : setDataVizSnackbar({ + severity: "error", + title: "Error while loading ExploreASL Data", + message: ["Failed to load and/or merge one or more ExploreASL spreadsheets."], + }); return; } @@ -286,7 +344,7 @@ function StepDefineDataframeLoc() { return (
- + Load DataFrame} subheader={ @@ -299,6 +357,11 @@ function StepDefineDataframeLoc() { } + action={ + + + + } /> diff --git a/src/pages/Help/HelpDataViz__StepClarifyDataTypes.tsx b/src/pages/Help/HelpDataViz__StepClarifyDataTypes.tsx new file mode 100644 index 0000000..4e98793 --- /dev/null +++ b/src/pages/Help/HelpDataViz__StepClarifyDataTypes.tsx @@ -0,0 +1,75 @@ +import ExpandMoreIcon from "@mui/icons-material/ExpandMore"; +import Accordion from "@mui/material/Accordion"; +import AccordionDetails from "@mui/material/AccordionDetails"; +import AccordionSummary from "@mui/material/AccordionSummary"; +import DialogContent from "@mui/material/DialogContent"; +import DialogContentText from "@mui/material/DialogContentText"; +import Divider from "@mui/material/Divider"; +import Stack from "@mui/material/Stack"; +import Typography from "@mui/material/Typography"; +import React, { useState } from "react"; +import { DialogTitleH4 } from "../../components/DialogTitle"; + +function HelpDataViz__StepClarifyDataTypes() { + const [expanded, setExpanded] = useState(false); + + const handleAccordionChange = (panel: string) => (event: React.SyntheticEvent, isExpanded: boolean) => { + setExpanded(isExpanded ? panel : false); + }; + + return ( + <> + Help Load Dataframe + + + Select a question that best describes your concern. + +
+ + }> + What do I do in this section? + + + +

+ In general, this section is just to help the program verify whether a given column has been + interpreted correctly. Most of the time, the correct inference should have been made. +

+
+

+ However, there is one common scenario that this program cannot detect: the case where categorical + variables are encoded by numbers (i.e. Female is 0 and Male is 1 for a categorical variable Sex). In + such an event, the user should clarify the data type context. +

+ + +
+ +
+ + }> + Why can't I change the data type of some columns?! + + + +

+ The most common reason is that you're trying to convert a categorical variable into a continuous + one when said categorical variable contains alphabetic characters. It doesn't make sense to try + such a conversion. +

+ + +
+ + + + ); +} + +export default React.memo(HelpDataViz__StepClarifyDataTypes); diff --git a/src/pages/Help/HelpDataViz__StepDefineDataframeLoc.tsx b/src/pages/Help/HelpDataViz__StepDefineDataframeLoc.tsx new file mode 100644 index 0000000..1da2a88 --- /dev/null +++ b/src/pages/Help/HelpDataViz__StepDefineDataframeLoc.tsx @@ -0,0 +1,247 @@ +import React, { useState } from "react"; +import { DialogTitleH4 } from "../../components/DialogTitle"; +import Accordion from "@mui/material/Accordion"; +import AccordionDetails from "@mui/material/AccordionDetails"; +import AccordionSummary from "@mui/material/AccordionSummary"; +import DialogContent from "@mui/material/DialogContent"; +import DialogContentText from "@mui/material/DialogContentText"; +import Divider from "@mui/material/Divider"; +import Stack from "@mui/material/Stack"; +import Typography from "@mui/material/Typography"; +import ExpandMoreIcon from "@mui/icons-material/ExpandMore"; +import { BulletPointList } from "./HelpStyledComponents"; +import Link from "@mui/material/Link"; +import HelperImage__DataVizRerunPopulationModule from "../../assets/img/HelperImages/HelperImage__DataVizRerunPopulationModule.png"; +import Box from "@mui/material/Box"; + +function HelpDataViz__StepDefineDataframeLoc() { + const [expanded, setExpanded] = useState(false); + + const handleAccordionChange = (panel: string) => (event: React.SyntheticEvent, isExpanded: boolean) => { + setExpanded(isExpanded ? panel : false); + }; + + return ( + <> + Help Load Dataframe + + + Select a question that best describes your concern. + +
+ + }> + What does the "Data Visualization" part of the program do? + + + +

The intent of this module is two-fold:

+ +
  • + To allow users to visualize how the cerebral blood flow (CBF) data of analyzed subjects maps + against other variables of interest. This greatly speeds up the identification of quantitative + outliers. +
    • +
  • + To assist users in quality control. If a data point comes off as an outlier, users can just click on + said data point and be presented with the CBF volumes as interactive axial, coronal, and sagittal + slice views. While ExploreASL outputs very helpful summaries, they are restricted to only certain + slices which risks the chance of artifacts being omitted. With an interactive view, this is no + longer an issue. +
    • +     +     +     +
    + +
    + + }> + Can I use this part of the program to visualize only my own spreadsheets? + + + +

    + Unfortunately, at the current time, this option is not available, since the program is specifically + geared towards plotting the output of ExploreASL. You can, however, merge the ExploreASL data with + your own metadata as long as the latter possesses a column called SUBJECT +

    +     +     +
    + +
    + + }> + Can I load Microsoft Excel spreadsheets as my metadata? + + + +

    Yes and no.

    +
    +

    + Only comma-separated values (.csv) or tab-separated values (.tsv) files are accepted for metadata. And + while Excel spreadsheets don't fit that criteria, all excel spreadsheets have the option to be + exported as one of the two in Microsoft Excel. You'll just have to do that extra step. +

    +     +     +
    + +
    + + }> + Why isn't it accepting the atlases I've selected?! + + + +

    + Because it would have been necessary to have those atlases first indicated in the "Define + Parameters" section of this program. +

    + +

    Fortunately, it isn't too much of a hassle to get things to work with these steps:

    + +
      +
    1. + Go back to the "Define Parameters" section of this program and load in your study's + dataPar.json file. +
      2. +
    2. + Go to the "Processing Parameters" tab of that section and indicate the atlases you are + interested in. +
      3. +
    3. Save the new dataPar.json file which will overwrite the old one.
      4. +
    4. + Go to the "Process Studies" --> "Prepare A Re-run" section. Load in the study + which will need to have its Population Module re-run. +
      5. +
    5. + Expand the xASL_module_Population portions and select the step associated with calculating ROI + statistics. Tell the program to delete that step. + +
      6. +
    6. Go to the "Run ExploreASL" tab and run the Population Module for this study.
      7. +
    7. Once completed successfully, you will be permitted to indicate those atlases.
      8. +
    +     +     +
    + +
    + + }> + What are the requirements for the program to accept my metadata? + + + +

    + The metadata spreadsheet must contain a column called SUBJECT in order to perform an{" "} + + outer-left join + {" "} + with the ExploreASL data (the latter being the "left" portion of the join). As this is an + outer-left join, subjects which are present in the metadata but absent in the ExploreASL data will be + removed. +

    +
    +

    + In addition to the SUBJECT column (which must be present regardless) you can also merge with an + additional column called: session. Again, the nature of the join is outer-left. Rows for SUBJECT and + session which are present in the metadata but absent in the ExploreASL data will be removed. +

    +     +     +
    + +
    + + }> + Will some of my metadata columns be removed? + + + +

    + It is possible that some of your metadata columns will be removed. Currently, the following conditions + will cause the removal of a column: +

    +
    + +
  • It is completely empty.
    • +
  • + It contains mixed types of data (i.e. pure numbers in certain cells and alphanumeric characters in + other cells) +
    • +
  • + It contains complex/nested values (i.e. if this: [1, 2, 3] was inside a cell, it is interpreted + as an array of numbers). Arrays and JSON-like content is interpreted as complex. +
    • +     +     +     +
    + +
    + + }> + Do you support custom ROIs (i.e. masks of clusters in a voxelwise analysis?) + + + +

    + At the current time, only defined regions from popular atlases (i.e. Harvard-Oxford Cortical) are + supported.  +

    +
    +

    + In the future, we will consider allowing users to indicate their own regions by + supplying: +

    + + +
  • + A NIFTI file in MNI-space (121 x 145 x 121 voxels) with integers masking the regions of interest. +
    • +
  • + A ".tsv" file with one row with the names of regions. The order of these names will have + to correspond to the ascending order of integers in the NIFTI file. +
    • +     +     +     +
    +     +     + + ); +} + +export default React.memo(HelpDataViz__StepDefineDataframeLoc); diff --git a/src/pages/Help/HelpImport__StepDefineRuntimeEnvs.tsx b/src/pages/Help/HelpImport__StepDefineRuntimeEnvs.tsx index 2010355..9a8da12 100644 --- a/src/pages/Help/HelpImport__StepDefineRuntimeEnvs.tsx +++ b/src/pages/Help/HelpImport__StepDefineRuntimeEnvs.tsx @@ -75,7 +75,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What is ExploreASL Type? @@ -110,7 +113,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What is ExploreASL Path? @@ -150,7 +156,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What is MATLAB Runtime Path @@ -184,7 +193,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What is the Study Root Folder? @@ -279,7 +291,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What does Ignore mean? @@ -301,7 +316,10 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - + }> What if a folder contains two pieces of information? @@ -316,7 +334,6 @@ function HelpImport__StepDefineRuntimeEnvs() {
    - diff --git a/src/pages/ImportModule/StepDefineRuntimeEnvs.tsx b/src/pages/ImportModule/StepDefineRuntimeEnvs.tsx index 143b2cf..1b1c1d6 100644 --- a/src/pages/ImportModule/StepDefineRuntimeEnvs.tsx +++ b/src/pages/ImportModule/StepDefineRuntimeEnvs.tsx @@ -48,19 +48,13 @@ function StepDefineRuntimeEnvs({ } action={ - + } /> - (theme.palette.mode === "dark" ? "#1e1e1e" : "#ffffff")} diff --git a/src/stores/DataFrameVisualizationStore.ts b/src/stores/DataFrameVisualizationStore.ts index 4e7b92e..8a1abf4 100644 --- a/src/stores/DataFrameVisualizationStore.ts +++ b/src/stores/DataFrameVisualizationStore.ts @@ -168,7 +168,7 @@ export const atomEASLSwarmplotSettings = atom({ plotLayout: "vertical", margins: { top: 40, right: 50, bottom: 60, left: 90 }, interSeriesGap: 0, - colorScheme: "nivo", + colorScheme: "set1", nodeSize: 10, nodeBorderWidth: 1, @@ -226,7 +226,7 @@ const EASLScatterPlotLegend: NivoLegendProps = { translateY: 0, itemsSpacing: 5, symbolSize: 12, - itemWidth: 100, + itemWidth: 120, itemHeight: 12, itemDirection: "left-to-right", symbolShape: "square", @@ -242,7 +242,7 @@ const EASLScatterPlotLegend: NivoLegendProps = { export const atomEASLScatterplotSettings = atom({ margins: { top: 40, right: 130, bottom: 60, left: 90 }, - colorScheme: "nivo", + colorScheme: "set1", nodeSize: 10,