Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed file structure. Updated to Python3/Keras2 #23

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 27 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,27 +48,30 @@ This tutorial depends on the following libraries:

Also, this code should be compatible with Python versions 2.7-3.5.

You can also set up a virtual enviroment and run your code from there:
* Create virtual enviroment folder: ```virtualenv -p python3 venv```
* Activate your virtual enviroment: ```source vev/bin/activate```
* Install packages needed in enviroment: ```pip3 install -r requirements.txt```
* Check installed packages: ```pip3 freeze```

### Prepare the data

First transfer 3D volume tiff to 30 512*512 images.

To feed the unet, data augmentation is necessary.

An [image deformation](http://faculty.cs.tamu.edu/schaefer/research/mls.pdf) method is used, the code is
An [image deformation](http://faculty.cs.tamu.edu/schaefer/research/mls.pdf) method is used, the code is

availabel in this [repository](https://github.com/cxcxcxcx/imgwarp-opencv).




### Define the model

* Check out ```get_unet()``` in ```unet.py``` to modify the model, optimizer and loss function.

### Train the model and generate masks for test images

* Run ```python unet.py``` to train the model.

* Run ```python3 unet.py``` to train the model.

After this script finishes, in ```imgs_mask_test.npy``` masks for corresponding images in ```imgs_test.npy```
should be generated. I suggest you examine these masks for getting further insight of your model's performance.
Expand All @@ -81,6 +84,25 @@ Use the trained model to do segmentation on test images, the result is statisfac

![img/0label.png](img/0label.png)

## Folder structure

* ```./data```: Contains sample data which can be used to test unet.
* ```./deform```: Contains 3 folders which are used when running your program. Images in ```./deform/train``` and ```./deform/label``` are used to train your network. After building your model images from ```./deform/test``` are used as inputs to produce results.
* ```./npydata```: Contains the ```.npy``` files generated from ```.tif``` images from our deform folder.
* ```./results```: Outputs from our unet are saved in this folder.

## From clone to your first results

* After cloning this project the first step is loading up the deform folder with the data you want to use to build and test your model. There is a sample dataset in the ```./data``` folder.

* ```python3 data.py``` to generate ```.npy``` files from your input data. This should produce 3 files in your ```./npydata``` folder.

* ```python3 unet.py``` to build your model. This should create a ```unet.hdf5``` file which is your model and can be loaded in your future programs. It also generates the results from the test dataset.

* Additionally you can use ```python3 test_predict.py``` to generate results for different test sets.




## About Keras

Expand Down
16 changes: 8 additions & 8 deletions data.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
import numpy as np
import numpy as np
import os
import glob
#import cv2
#from libtiff import TIFF

class myAugmentation(object):

"""
A class used to augmentate image
Firstly, read train image and label seperately, and then merge them together for the next process
Expand All @@ -15,7 +15,7 @@ class myAugmentation(object):
"""

def __init__(self, train_path="train", label_path="label", merge_path="merge", aug_merge_path="aug_merge", aug_train_path="aug_train", aug_label_path="aug_label", img_type="tif"):

"""
Using glob to get all .img_type form path
"""
Expand Down Expand Up @@ -52,7 +52,7 @@ def Augmentation(self):
imgtype = self.img_type
path_aug_merge = self.aug_merge_path
if len(trains) != len(labels) or len(trains) == 0 or len(trains) == 0:
print "trains can't match labels"
print("trains can't match labels")
return 0
for i in range(len(trains)):
img_t = load_img(path_train+"/"+str(i)+"."+imgtype)
Expand Down Expand Up @@ -135,10 +135,10 @@ def splitTransform(self):

class dataProcess(object):

def __init__(self, out_rows, out_cols, data_path = "../deform/train", label_path = "../deform/label", test_path = "../test", npy_path = "../npydata", img_type = "tif"):
def __init__(self, out_rows, out_cols, data_path = "./deform/train", label_path = "./deform/label", test_path = "./deform/test", npy_path = "./npydata", img_type = "tif"):

"""

"""

self.out_rows = out_rows
Expand Down Expand Up @@ -208,7 +208,7 @@ def load_train_data(self):
imgs_mask_train = imgs_mask_train.astype('float32')
imgs_train /= 255
#mean = imgs_train.mean(axis = 0)
#imgs_train -= mean
#imgs_train -= mean
imgs_mask_train /= 255
imgs_mask_train[imgs_mask_train > 0.5] = 1
imgs_mask_train[imgs_mask_train <= 0.5] = 0
Expand All @@ -222,7 +222,7 @@ def load_test_data(self):
imgs_test = imgs_test.astype('float32')
imgs_test /= 255
#mean = imgs_test.mean(axis = 0)
#imgs_test -= mean
#imgs_test -= mean
return imgs_test

if __name__ == "__main__":
Expand Down
Binary file added deform/.DS_Store
Binary file not shown.
Binary file added deform/label/.DS_Store
Binary file not shown.
Binary file added deform/label/0.tif
Binary file not shown.
Binary file added deform/label/1.tif
Binary file not shown.
Binary file added deform/label/10.tif
Binary file not shown.
Binary file added deform/label/11.tif
Binary file not shown.
Binary file added deform/label/12.tif
Binary file not shown.
Binary file added deform/label/13.tif
Binary file not shown.
Binary file added deform/label/14.tif
Binary file not shown.
Binary file added deform/label/15.tif
Binary file not shown.
Binary file added deform/label/16.tif
Binary file not shown.
Binary file added deform/label/17.tif
Binary file not shown.
Binary file added deform/label/18.tif
Binary file not shown.
Binary file added deform/label/19.tif
Binary file not shown.
Binary file added deform/label/2.tif
Binary file not shown.
Binary file added deform/label/20.tif
Binary file not shown.
Binary file added deform/label/21.tif
Binary file not shown.
Binary file added deform/label/22.tif
Binary file not shown.
Binary file added deform/label/23.tif
Binary file not shown.
Binary file added deform/label/24.tif
Binary file not shown.
Binary file added deform/label/25.tif
Binary file not shown.
Binary file added deform/label/26.tif
Binary file not shown.
Binary file added deform/label/27.tif
Binary file not shown.
Binary file added deform/label/28.tif
Binary file not shown.
Binary file added deform/label/29.tif
Binary file not shown.
Binary file added deform/label/3.tif
Binary file not shown.
Binary file added deform/label/4.tif
Binary file not shown.
Binary file added deform/label/5.tif
Binary file not shown.
Binary file added deform/label/6.tif
Binary file not shown.
Binary file added deform/label/7.tif
Binary file not shown.
Binary file added deform/label/8.tif
Binary file not shown.
Binary file added deform/label/9.tif
Binary file not shown.
Binary file added deform/test/0.tif
Binary file not shown.
Binary file added deform/test/1.tif
Binary file not shown.
Binary file added deform/test/10.tif
Binary file not shown.
Binary file added deform/test/11.tif
Binary file not shown.
Binary file added deform/test/12.tif
Binary file not shown.
Binary file added deform/test/13.tif
Binary file not shown.
Binary file added deform/test/14.tif
Binary file not shown.
Binary file added deform/test/15.tif
Binary file not shown.
Binary file added deform/test/16.tif
Binary file not shown.
Binary file added deform/test/17.tif
Binary file not shown.
Binary file added deform/test/18.tif
Binary file not shown.
Binary file added deform/test/19.tif
Binary file not shown.
Binary file added deform/test/2.tif
Binary file not shown.
Binary file added deform/test/20.tif
Binary file not shown.
Binary file added deform/test/21.tif
Binary file not shown.
Binary file added deform/test/22.tif
Binary file not shown.
Binary file added deform/test/23.tif
Binary file not shown.
Binary file added deform/test/24.tif
Binary file not shown.
Binary file added deform/test/25.tif
Binary file not shown.
Binary file added deform/test/26.tif
Binary file not shown.
Binary file added deform/test/27.tif
Binary file not shown.
Binary file added deform/test/28.tif
Binary file not shown.
Binary file added deform/test/29.tif
Binary file not shown.
Binary file added deform/test/3.tif
Binary file not shown.
Binary file added deform/test/4.tif
Binary file not shown.
Binary file added deform/test/5.tif
Binary file not shown.
Binary file added deform/test/6.tif
Binary file not shown.
Binary file added deform/test/7.tif
Binary file not shown.
Binary file added deform/test/8.tif
Binary file not shown.
Binary file added deform/test/9.tif
Binary file not shown.
Binary file added deform/train/0.tif
Binary file not shown.
Binary file added deform/train/1.tif
Binary file not shown.
Binary file added deform/train/10.tif
Binary file not shown.
Binary file added deform/train/11.tif
Binary file not shown.
Binary file added deform/train/12.tif
Binary file not shown.
Binary file added deform/train/13.tif
Binary file not shown.
Binary file added deform/train/14.tif
Binary file not shown.
Binary file added deform/train/15.tif
Binary file not shown.
Binary file added deform/train/16.tif
Binary file not shown.
Binary file added deform/train/17.tif
Binary file not shown.
Binary file added deform/train/18.tif
Binary file not shown.
Binary file added deform/train/19.tif
Binary file not shown.
Binary file added deform/train/2.tif
Binary file not shown.
Binary file added deform/train/20.tif
Binary file not shown.
Binary file added deform/train/21.tif
Binary file not shown.
Binary file added deform/train/22.tif
Binary file not shown.
Binary file added deform/train/23.tif
Binary file not shown.
Binary file added deform/train/24.tif
Binary file not shown.
Binary file added deform/train/25.tif
Binary file not shown.
Binary file added deform/train/26.tif
Binary file not shown.
Binary file added deform/train/27.tif
Binary file not shown.
Binary file added deform/train/28.tif
Binary file not shown.
Binary file added deform/train/29.tif
Binary file not shown.
Binary file added deform/train/3.tif
Binary file not shown.
Binary file added deform/train/4.tif
Binary file not shown.
Binary file added deform/train/5.tif
Binary file not shown.
Binary file added deform/train/6.tif
Binary file not shown.
Binary file added deform/train/7.tif
Binary file not shown.
Binary file added deform/train/8.tif
Binary file not shown.
Binary file added deform/train/9.tif
Binary file not shown.
Binary file added npydata/imgs_mask_train.npy
Binary file not shown.
Binary file added npydata/imgs_test.npy
Binary file not shown.
Binary file added npydata/imgs_train.npy
Binary file not shown.
21 changes: 21 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
bleach==1.5.0
cycler==0.10.0
enum34==1.1.6
h5py==2.7.1
html5lib==0.9999999
Keras==2.1.2
Markdown==2.6.10
matplotlib==2.1.0
numpy==1.13.3
olefile==0.44
Pillow==4.3.0
protobuf==3.5.0.post1
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2017.3
PyYAML==3.12
scipy==1.0.0
six==1.11.0
tensorflow==1.4.1
tensorflow-tensorboard==0.4.0rc3
Werkzeug==0.13
Binary file added results/0.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/10.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/11.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/12.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/13.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/14.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/15.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/16.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/17.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/18.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/19.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/20.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/21.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/22.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/23.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/24.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/25.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/26.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/27.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/28.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/29.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/3.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/4.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/5.jpg
Binary file added results/6.jpg
Binary file added results/7.jpg
Binary file added results/8.jpg
Binary file added results/9.jpg
Binary file added results/imgs_mask_test.npy
Binary file not shown.
2 changes: 1 addition & 1 deletion test_predict.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@

imgs_mask_test = model.predict(imgs_test, verbose=1)

np.save('imgs_mask_test.npy', imgs_mask_test)
np.save('./results/imgs_mask_test.npy', imgs_mask_test)
Loading