Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to restrict floodfill to a bounding box #8267

Merged
merged 29 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8e656ad
avoid high frequency logs of awaited-missing-bucket messages
philippotto Dec 9, 2024
ee6b663
improve comments
philippotto Dec 9, 2024
070b5f5
move useSelector so that it's only active when the context menu is op…
philippotto Dec 9, 2024
41f139f
move floodfill saga code into own module
philippotto Dec 9, 2024
d921995
hardcode floodfill constraint to current bbox
philippotto Dec 9, 2024
f9ff2ae
add toggle to navbar to enable/disable bbox restriction for floodfill
philippotto Dec 9, 2024
4b29cec
fix linting
philippotto Dec 9, 2024
fbcbd6c
debug test
philippotto Dec 10, 2024
daf2c0c
only create bbox after *3d* floodfill terminated early
philippotto Dec 12, 2024
b667220
fix test and creation of bbox in exceeding case
philippotto Dec 12, 2024
f714568
format
philippotto Dec 12, 2024
20ec8c4
remove only modifier again
philippotto Dec 12, 2024
be9df82
fix floodfill saga termination; improve ui
philippotto Dec 13, 2024
8349c0c
fix flickering of undo/redo buttons when clicking them
philippotto Dec 13, 2024
ee03e68
integrate new icon and don't create bbox in 3d case when the bbox res…
philippotto Dec 17, 2024
0e26bb0
avoid frequent vector3 allocation in floodfill
philippotto Dec 17, 2024
a267052
ignore changed md and snap files in yarn test-changed
philippotto Dec 17, 2024
445e4d9
fix bounding box creation outside of DS bbox
philippotto Dec 17, 2024
aabbfbe
even when using isFloodfillRestrictedToBoundingBox, check that the bb…
philippotto Dec 17, 2024
35a4913
Merge branch 'master' of github.com:scalableminds/webknossos into flo…
philippotto Dec 17, 2024
637ce46
update changelog
philippotto Dec 17, 2024
3959e9b
update docs
philippotto Dec 18, 2024
b056759
fix colliding hide instructions of progress callback when doing multi…
philippotto Dec 18, 2024
d13abc2
immediately hide floodfill success toast if the operation has finishe…
philippotto Dec 18, 2024
da9cead
fix messages mock in tests
philippotto Dec 18, 2024
a5c516a
more icon_restricted_floodfill.jpg to subfolder
philippotto Jan 3, 2025
78a1a33
deduplicate success toast logic
philippotto Jan 3, 2025
e747a7a
refactor flood fill saga
philippotto Jan 3, 2025
36ceb5f
Merge branch 'master' into floodfill-in-bbox
philippotto Jan 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions frontend/javascripts/oxalis/default_state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ const defaultState: OxalisState = {
gpuMemoryFactor: Constants.DEFAULT_GPU_MEMORY_FACTOR,
overwriteMode: OverwriteModeEnum.OVERWRITE_ALL,
fillMode: FillModeEnum._2D,
isFloodfillRestrictedToBoundingBox: false,
interpolationMode: InterpolationModeEnum.INTERPOLATE,
useLegacyBindings: false,
quickSelect: {
Expand Down
12 changes: 12 additions & 0 deletions frontend/javascripts/oxalis/model/accessors/tracing_accessor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import type {
import type { ServerTracing, TracingType } from "types/api_flow_types";
import { TracingTypeEnum } from "types/api_flow_types";
import type { SaveQueueType } from "oxalis/model/actions/save_actions";
import BoundingBox from "../bucket_data_handling/bounding_box";
import type { Vector3 } from "oxalis/constants";

export function maybeGetSomeTracing(
tracing: Tracing,
Expand Down Expand Up @@ -86,7 +88,17 @@ export function selectTracing(

return tracing;
}

export const getUserBoundingBoxesFromState = (state: OxalisState): Array<UserBoundingBox> => {
const maybeSomeTracing = maybeGetSomeTracing(state.tracing);
return maybeSomeTracing != null ? maybeSomeTracing.userBoundingBoxes : [];
};

export const getUserBoundingBoxesThatContainPosition = (
state: OxalisState,
position: Vector3,
): Array<UserBoundingBox> => {
const bboxes = getUserBoundingBoxesFromState(state);

return bboxes.filter((el) => new BoundingBox(el.boundingBox).containsPoint(position));
};
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ const warnMergeWithoutPendingOperations = _.throttle(() => {
);
}, WARNING_THROTTLE_THRESHOLD);

const warnAwaitedMissingBucket = _.throttle(() => {
ErrorHandling.notify(new Error("Awaited missing bucket"));
}, WARNING_THROTTLE_THRESHOLD);

export function assertNonNullBucket(bucket: Bucket): asserts bucket is DataBucket {
if (bucket.type === "null") {
throw new Error("Unexpected null bucket.");
Expand Down Expand Up @@ -773,7 +777,7 @@ export class DataBucket {
// In the past, ensureLoaded() never returned if the bucket
// was MISSING. This log might help to discover potential
// bugs which could arise in combination with MISSING buckets.
console.warn("Awaited missing bucket.");
warnAwaitedMissingBucket();
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -487,7 +487,7 @@ class DataCube {
additionalCoordinates: AdditionalCoordinate[] | null,
segmentIdNumber: number,
dimensionIndices: DimensionMap,
floodfillBoundingBox: BoundingBoxType,
_floodfillBoundingBox: BoundingBoxType,
zoomStep: number,
progressCallback: ProgressCallback,
use3D: boolean,
Expand All @@ -498,14 +498,17 @@ class DataCube {
}> {
// This flood-fill algorithm works in two nested levels and uses a list of buckets to flood fill.
// On the inner level a bucket is flood-filled and if the iteration of the buckets data
// reaches an neighbour bucket, this bucket is added to this list of buckets to flood fill.
// reaches a neighbour bucket, this bucket is added to this list of buckets to flood fill.
// The outer level simply iterates over all buckets in the list and triggers the bucket-wise flood fill.
// Additionally a map is created that saves all labeled voxels for each bucket. This map is returned at the end.
//
// Note: It is possible that a bucket is multiple times added to the list of buckets. This is intended
// Note: It is possible that a bucket is added multiple times to the list of buckets. This is intended
// because a border of the "neighbour volume shape" might leave the neighbour bucket and enter it somewhere else.
// If it would not be possible to have the same neighbour bucket in the list multiple times,
// not all of the target area in the neighbour bucket might be filled.

const floodfillBoundingBox = new BoundingBox(_floodfillBoundingBox);

// Helper function to convert between xyz and uvw (both directions)
const transpose = (voxel: Vector3): Vector3 =>
Dimensions.transDimWithIndices(voxel, dimensionIndices);
Expand Down Expand Up @@ -689,16 +692,19 @@ class DataCube {
} else {
// Label the current neighbour and add it to the neighbourVoxelStackUvw to iterate over its neighbours.
const neighbourVoxelIndex = this.getVoxelIndexByVoxelOffset(neighbourVoxelXyz);

if (bucketData[neighbourVoxelIndex] === sourceSegmentId) {
const currentGlobalPosition = V3.add(
currentGlobalBucketPosition,
V3.scale3(adjustedNeighbourVoxelXyz, currentMag),
);

if (
bucketData[neighbourVoxelIndex] === sourceSegmentId &&
floodfillBoundingBox.containsPoint(currentGlobalPosition)
) {
bucketData[neighbourVoxelIndex] = segmentId;
markUvwInSliceAsLabeled(neighbourVoxelUvw);
neighbourVoxelStackUvw.pushVoxel(neighbourVoxelUvw);
labeledVoxelCount++;
const currentGlobalPosition = V3.add(
currentGlobalBucketPosition,
V3.scale3(adjustedNeighbourVoxelXyz, currentMag),
);
coveredBBoxMin = [
Math.min(coveredBBoxMin[0], currentGlobalPosition[0]),
Math.min(coveredBBoxMin[1], currentGlobalPosition[1]),
Expand Down
257 changes: 257 additions & 0 deletions frontend/javascripts/oxalis/model/sagas/volume/floodfill_saga.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
import { message } from "antd";
import { V3 } from "libs/mjs";
import createProgressCallback from "libs/progress_callback";
import Toast from "libs/toast";
import * as Utils from "libs/utils";
import type { BoundingBoxType, LabeledVoxelsMap, OrthoView, Vector3 } from "oxalis/constants";
import Constants, { FillModeEnum, Unicode } from "oxalis/constants";

import { getDatasetBoundingBox, getMagInfo } from "oxalis/model/accessors/dataset_accessor";
import { getActiveMagIndexForLayer } from "oxalis/model/accessors/flycam_accessor";
import { enforceActiveVolumeTracing } from "oxalis/model/accessors/volumetracing_accessor";
import { addUserBoundingBoxAction } from "oxalis/model/actions/annotation_actions";
import { setBusyBlockingInfoAction } from "oxalis/model/actions/ui_actions";
import {
finishAnnotationStrokeAction,
updateSegmentAction,
} from "oxalis/model/actions/volumetracing_actions";
import BoundingBox from "oxalis/model/bucket_data_handling/bounding_box";
import Dimensions from "oxalis/model/dimensions";
import type { Saga } from "oxalis/model/sagas/effect-generators";
import { select, take } from "oxalis/model/sagas/effect-generators";
import { requestBucketModificationInVolumeTracing } from "oxalis/model/sagas/saga_helpers";
import { Model } from "oxalis/singletons";
import { call, put } from "typed-redux-saga";
import { getUserBoundingBoxesThatContainPosition } from "../../accessors/tracing_accessor";
import { applyLabeledVoxelMapToAllMissingMags } from "./helpers";
import _ from "lodash";

function* getBoundingBoxForFloodFill(
position: Vector3,
currentViewport: OrthoView,
): Saga<BoundingBoxType | { failureReason: string }> {
const isRestrictedToBoundingBox = yield* select(
(state) => state.userConfiguration.isFloodfillRestrictedToBoundingBox,
);
if (isRestrictedToBoundingBox) {
philippotto marked this conversation as resolved.
Show resolved Hide resolved
const bboxes = yield* select((state) =>
getUserBoundingBoxesThatContainPosition(state, position),
);
if (bboxes.length > 0) {
const smallestBbox = _.sortBy(bboxes, (bbox) =>
new BoundingBox(bbox.boundingBox).getVolume(),
)[0];
return smallestBbox.boundingBox;
} else {
return {
failureReason:
"No bounding box encloses the clicked position. Either disable the bounding box restriction or ensure a bounding box exists around the clicked position.",
};
}
}
philippotto marked this conversation as resolved.
Show resolved Hide resolved

const fillMode = yield* select((state) => state.userConfiguration.fillMode);
const halfBoundingBoxSizeUVW = V3.scale(Constants.FLOOD_FILL_EXTENTS[fillMode], 0.5);
const currentViewportBounding = {
min: V3.sub(position, halfBoundingBoxSizeUVW),
max: V3.add(position, halfBoundingBoxSizeUVW),
};

if (fillMode === FillModeEnum._2D) {
// Only use current plane
const thirdDimension = Dimensions.thirdDimensionForPlane(currentViewport);
const numberOfSlices = 1;
currentViewportBounding.min[thirdDimension] = position[thirdDimension];
currentViewportBounding.max[thirdDimension] = position[thirdDimension] + numberOfSlices;
}

const datasetBoundingBox = yield* select((state) => getDatasetBoundingBox(state.dataset));
const { min: clippedMin, max: clippedMax } = new BoundingBox(
currentViewportBounding,
).intersectedWith(datasetBoundingBox);
return {
min: clippedMin,
max: clippedMax,
};
philippotto marked this conversation as resolved.
Show resolved Hide resolved
}

const FLOODFILL_PROGRESS_KEY = "FLOODFILL_PROGRESS_KEY";
export function* floodFill(): Saga<void> {
yield* take("INITIALIZE_VOLUMETRACING");
const allowUpdate = yield* select((state) => state.tracing.restrictions.allowUpdate);

while (allowUpdate) {
const floodFillAction = yield* take("FLOOD_FILL");

if (floodFillAction.type !== "FLOOD_FILL") {
throw new Error("Unexpected action. Satisfy typescript.");
}

const { position: positionFloat, planeId } = floodFillAction;
const volumeTracing = yield* select(enforceActiveVolumeTracing);
if (volumeTracing.hasEditableMapping) {
const message = "Volume modification is not allowed when an editable mapping is active.";
Toast.error(message);
console.error(message);
continue;
}
const segmentationLayer = yield* call(
[Model, Model.getSegmentationTracingLayer],
volumeTracing.tracingId,
);
const { cube } = segmentationLayer;
const seedPosition = Dimensions.roundCoordinate(positionFloat);
const activeCellId = volumeTracing.activeCellId;
const dimensionIndices = Dimensions.getIndices(planeId);
const requestedZoomStep = yield* select((state) =>
getActiveMagIndexForLayer(state, segmentationLayer.name),
);
const magInfo = yield* call(getMagInfo, segmentationLayer.mags);
const labeledZoomStep = magInfo.getClosestExistingIndex(requestedZoomStep);
const additionalCoordinates = yield* select((state) => state.flycam.additionalCoordinates);
const oldSegmentIdAtSeed = cube.getDataValue(
seedPosition,
additionalCoordinates,
null,
labeledZoomStep,
);

if (activeCellId === oldSegmentIdAtSeed) {
Toast.warning("The clicked voxel's id is already equal to the active segment id.");
continue;
}

const busyBlockingInfo = yield* select((state) => state.uiInformation.busyBlockingInfo);

if (busyBlockingInfo.isBusy) {
console.warn(`Ignoring floodfill request (reason: ${busyBlockingInfo.reason || "unknown"})`);
continue;
}
// As the flood fill will be applied to the volume layer,
// the potentially existing mapping should be locked to ensure a consistent state.
const isModificationAllowed = yield* call(
requestBucketModificationInVolumeTracing,
volumeTracing,
);
if (!isModificationAllowed) {
continue;
}
yield* put(setBusyBlockingInfoAction(true, "Floodfill is being computed."));
const boundingBoxForFloodFill = yield* call(getBoundingBoxForFloodFill, seedPosition, planeId);
if ("failureReason" in boundingBoxForFloodFill) {
Toast.warning(boundingBoxForFloodFill.failureReason);
return;
}
philippotto marked this conversation as resolved.
Show resolved Hide resolved
const progressCallback = createProgressCallback({
pauseDelay: 200,
successMessageDelay: 2000,
// Since only one floodfill operation can be active at any time,
// a hardcoded key is sufficient.
key: "FLOODFILL_PROGRESS_KEY",
});
yield* call(progressCallback, false, "Performing floodfill...");
console.time("cube.floodFill");
const fillMode = yield* select((state) => state.userConfiguration.fillMode);

const {
bucketsWithLabeledVoxelsMap: labelMasksByBucketAndW,
wasBoundingBoxExceeded,
coveredBoundingBox,
} = yield* call(
{ context: cube, fn: cube.floodFill },
seedPosition,
additionalCoordinates,
activeCellId,
dimensionIndices,
boundingBoxForFloodFill,
labeledZoomStep,
progressCallback,
fillMode === FillModeEnum._3D,
);
console.timeEnd("cube.floodFill");
yield* call(progressCallback, false, "Finalizing floodfill...");
const indexSet: Set<number> = new Set();

for (const labelMaskByIndex of labelMasksByBucketAndW.values()) {
for (const zIndex of labelMaskByIndex.keys()) {
indexSet.add(zIndex);
}
}

console.time("applyLabeledVoxelMapToAllMissingMags");

for (const indexZ of indexSet) {
const labeledVoxelMapFromFloodFill: LabeledVoxelsMap = new Map();

for (const [bucketAddress, labelMaskByIndex] of labelMasksByBucketAndW.entries()) {
const map = labelMaskByIndex.get(indexZ);

if (map != null) {
labeledVoxelMapFromFloodFill.set(bucketAddress, map);
}
}

applyLabeledVoxelMapToAllMissingMags(
labeledVoxelMapFromFloodFill,
labeledZoomStep,
dimensionIndices,
magInfo,
cube,
activeCellId,
indexZ,
true,
);
}

yield* put(finishAnnotationStrokeAction(volumeTracing.tracingId));
yield* put(
updateSegmentAction(
volumeTracing.activeCellId,
{
somePosition: seedPosition,
someAdditionalCoordinates: additionalCoordinates || undefined,
},
volumeTracing.tracingId,
),
);

console.timeEnd("applyLabeledVoxelMapToAllMissingMags");

if (wasBoundingBoxExceeded) {
philippotto marked this conversation as resolved.
Show resolved Hide resolved
yield* call(
progressCallback,
true,
<>
Floodfill is done, but terminated since the labeled volume got too large. A bounding box
<br />
that represents the labeled volume was added.{Unicode.NonBreakingSpace}
<a href="#" onClick={() => message.destroy(FLOODFILL_PROGRESS_KEY)}>
Close
</a>
</>,
{
successMessageDelay: 10000,
},
);
yield* put(
addUserBoundingBoxAction({
boundingBox: coveredBoundingBox,
name: `Limits of flood-fill (source_id=${oldSegmentIdAtSeed}, target_id=${activeCellId}, seed=${seedPosition.join(
",",
)}, timestamp=${new Date().getTime()})`,
color: Utils.getRandomColor(),
isVisible: true,
}),
);
} else {
yield* call(progressCallback, true, "Floodfill done.");
}

cube.triggerPushQueue();
yield* put(setBusyBlockingInfoAction(false));

if (floodFillAction.callback != null) {
floodFillAction.callback();
}
}
}
Loading