Skip to content

Latest commit

 

History

History
247 lines (174 loc) · 13.7 KB

README_en.md

File metadata and controls

247 lines (174 loc) · 13.7 KB
Raw

English | 简体中文

A easy-to-use, efficient, smart 3D medical image annotation platform

License python version support os

Brief Introduciton

3D medical data annotation is an important part of training 3D image segmentation models and promotes disease diagnosis and treatment prediction, but 3D medical data annotation relies on time-consuming and laborious manual annotation by professionals. The low labeling efficiency leads to the lack of large-scale labeling data, which seriously hinders the development of medical AI. To solve this problem, we launched EISeg-Med3D, an intelligent annotation platform for 3D medical images based on interactive segmentation.

EISeg-Med3D is a 3D slicer extension for performing Efficient Interactive Segmentation on Medical image in 3D medical images. Users will guide a deep learning model to perform segmentation by providing positive and negative points. It is simple to install, easy to use and accurate, which can achieve ten times efficiency lift compares to manual labelling. At present, our medical annotation provides the try-on experience on the specified MRI vertebral data. If there is a need for 3D annotation on other data, you can make a contact.

Feature

  • Efficient：Each category only needs a few clicks to generate 3d segmentation results, ten times efficient compares to time-consuming and laborious manual annotation.

  • Accurate：The mIOU can reach 0.85 with only 3 clicks. with the segmentation editor equipped with machine learning algorithm and manual annotation, 100% accuracy is right on your hand.

  • Convenient：Install our plugin within three steps; labeling results and progress are automatically saved; the transparency of labeling results can be adjusted to improve labeling accuracy; user-friendly interface interaction makes labeling worry-free and hassle-free。

Contents

  1. News
  2. EISeg-Med3D Model Introduction
  3. User Guide
  4. TODO
  5. License
  6. Thanks

最新消息

  • [2022-09] EISeg-Med3D is officially released, a user-friendly, efficient and intelligent 3D medical image annotation platform including high-precision models on specified vertebral data.

EISeg-Med3D Model

The EISeg-Med 3D model structure is shown in the figure below. We innovatively introduce the 3D model into the medical interactive segmentation, and modify the point sampler module and the click feature extrator of RITM to be compatible with 3D data, so as to directly label 3D medical images. Compared with 2D interactive annotation on 3D images, our method is more acurate and more efficient.

The overall model includes two parts: click generation module, click feature generation module, click feature and input image fusion and segmentation model:

  • Click generation module 3D click sampler: generate the positive and negative click through sampling on the 3D labelled data directly, where the positive point is a random point at the center of the 3D target segment, and the negative point is a random point at the edge of the 3D target segment.
  • Click feature generation module 3D feature extractor: After the click is generated, a sphere with radius R is generated on the basis of the original click in the form of disk to expand the feature coverage.
  • Fusion of click feature and input image: The input image and the generated click feature are remapped and fused through the convolution block, so that the network obtains information from both the image and the clicks, and performs 3D segmentation on a the assigned area to obtain the annotation result.
  • Segmentation model: The segmentation model is the 3D segmentation model Vnet, generates the prediction results shown in the figure, and approximates the real results under the constraints of Dice loss and CE loss. Thus, in the prediction stage, input images and specified clicks generate the desired annotation results.

EISeg-Med3D Model

User Guide

The overall process of using EISeg-Med3D is shown in the figure below. We will introduce in the following three steps including environment installation, model and data download and user guide. The steps to use our platform can also be found in the video in the introduction.

The overall process

The process of AI-assisted labelling

环境安装

  1. Download and install 3D slicer：Slicer website

  2. Download code of EISeg-Med3D：

git clone https://github.com/PaddlePaddle/PaddleSeg.git
  1. Find the slicer python executor in python interpreter of slicer. Then install PaddlePaddle refer to install doc in windows cmd.
import sys
import os
sys.executable # "D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe"

In CMD, If you install on Windows with CUDA 11.1 GPU, follow the command here：：

"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip install paddlepaddle-gpu==2.3.1.post111 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html
Common FAQ
  1. FileNotFoundError when install PaddlePaddle：

Solution：Find the subprocess.py that raise the error, change the attribute of Popen shell=True。
  1. ERROR: No .egg-info directory found in xxx:
Solution：Please refer to PaddlePaddle#2718. Execute the following code will solve the error. 
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip uninstall setuptools
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe"  -m pip install paddleseg simpleitk
"D:/xxxx/Slicer 5.0.3/bin/PythonSlicer.exe" -m pip install setuptools
  1. When load module, the poped window says "One or more requested modules and/or depandencoes may not have been loaded".

Solution：You may forget to install some library we need to import, eg: paddle, paddleseg, simpleitk and etc. Please refer to step 1 and 2 to install and restart Slicer to import.
  1. Fail to open extention: xxx/PaddleSeg/EISeg/med3d/EISefMed3D

Solution：The path you need to choose is "xxx/PaddleSeg/EISeg/med3d/" but not "xxx/PaddleSeg/EISeg/med3d/EISefMed3D"

Model and Data Downloading

Currently we provide trial experience on the following models and data:

Data Model Links
MRI-spine Interactive Vnet pdiparams-pw: 6ok7 | pdmodel-pw: sg80 | Spine Data

User Guide

1. Load the Extension

  • Locate Extension wizard:

  • Click on "Select Extension"，and choose PaddleSeg/EISeg/med3d directory, and click to load corresponding module.

* After loading, switch to EISegMed3D。

2. Load Model

  • Load the downloaded model in Model Settings: click on the ... button of Model Path, choose local file of .pdodel suffix and load .pdiparams file in Param Path in the same way.
  • Click on Load Static Model button. And Sucessfully loaded model to gpu! window will be prompt is the model is loaded successfully.

3. Load Medical Data

  • Click on the button behind Data Folder, choose the folder that you saved your downloaded data. And all of the data under that folder will be loaded and you can see the labelling status of loaded data in Progress.

4. Switch Between Images.

  • Click on the Prev Scan button to see the previous image.
  • Click on the Next Scan button to see the next image.

5. Start to label

  • Click Add/Remove in Segment Editor to add or remove the label. You can change the name of added label by double click the label item.
  • Choose the label you want to label and click on the Positive Point or Negative Point to enter interactive label mode。
  • Click on Finish Segment button to finish annotation of current segment, you can further edit the annotatioin using tools in segment editor or you can repeat previous step to label next category. If you finished the annotation on this case, you can click on the Finish Scan button.

6. Check Label Progress

  • In Annotation Progress of Progress, you can checkout the labelling progress of loaded images.
  • Doule click on one of the image in the chart below Annotation Progress will jump to the corresponding image.

TODO

In the future, we want to continue to develop EISeg-Med3D in these aspects, welcome to join our developer team.

  • Work on larger vertebrae datasets and improve generality of our spine model.
  • Develop models trained on multiple organs to obtain models that generalize to multiple organs.

License

EISeg-Med3D is released under the Apache 2.0 license.

Attribution

Thanks to Idea icons created by Vectors Market - Flaticon for facsinating icons.