Skip to content

Latest commit

 

History

History
56 lines (33 loc) · 3.18 KB

README.md

File metadata and controls

56 lines (33 loc) · 3.18 KB
Raw

Lesionize - Synthetic Stroke Lesion Generation for MRI Data

Deploy with Vercel

Lesionize

How it works

Lesionize utilizes a latent diffusion model to generate synthetic stroke lesions that mimic those found in ischemic stroke patient MRI scans. This functionality is specifically designed to assist stroke detection and segmentation projects by providing augmented data for training and improving stroke detection models.

The application allows users to upload an image of an MRI scan. The uploaded images are then processed using the latent diffusion model through a Next.js API route, and the generated image with synthetic stroke lesions is returned to the user. The latent diffusion model is hosted on Replicate, and Upload is used for image storage.

Datasets procured

For the purposes of this application, MRI DWI Images of the axial plane were gathered from two main datasets

  1. ISLES22 Grand Challenge Dataset -- This dataset contains MRI DWI images of the brains of stroke patients. The lesions within the brain have been annotated on the MRI images.
  2. OpenNeuro NIMH Healthy Research Volunteer Dataset -- This dataset contains MRI DWI images of the brains of healthy patients.

The data was processed and cleaned to allow Hugging Face to curate our custom dataset using ImageFolder.

Training Approach

We utilized the InstructPix2Pix training approach and pipeline for our use case.

This pipeline takes in an input image and prompt, and outputs the input image edited according to the prompt's specifications.

Thus, each training data object consisted of the following data:

  • input image: the MRI DWI scan of the brain of a healthy patient
  • prompt: a prompt describing the type of lesions to insert into the MRI scan of the healthy brain
  • edited image: the MRI DWI scan of the brain of a patient with stroke lesions

Thus, we allow the model to pick up "instructions" from the prompt, to amend the input image.

The training script can be found within the diffusers module, or by clicking here

Running Locally

Installing the dependencies.

npm install

Running the application.

Then, run the application in the command line, and it will be available at http://localhost:3000.

npm run dev

One-Click Deploy

Deploy the example using Vercel:

Deploy with Vercel