DMENet: Deep Defocus Map Estimation Network

Checkout for the demo!
The demo server might occasionally be offline

This repository contains the official TensorFlow implementation of the following paper:

Deep Defocus Map Estimation using Domain Adaptation
Junyong Lee, Sungkil Lee, Sunghyun Cho and Seungyong Lee, CVPR 2019

Getting Started

Prerequisites

Tested environment

1. Setup environment

   • Option 1. docker

     • Download the docker image and run container: docker pull codeslake/ubuntu18.04-conda:DME.
     • Inside container, activate DMENet environment (i.e., conda activate DMENet).

   • Option 2. manual installation (conda should be installed)

     $ conda create --name DMENet python=3.6
     $ conda activate DMENet
     $ conda install cudatoolkit=10.0
     $ conda install cudnn=7.6
     $ pip install tensorflow-gpu==1.15
     $ pip install tensorlayer==1.11.1

2. Install requirement: pip install -r requirements.txt

3. Download and unzip datasets under [DATASET_ROOT].

   ├── [DATASET_ROOT]
   │   ├── train
   │   │   ├── SYNDOF
   │   │   ├── CUHK
   │   │   ├── Flickr
   │   ├── test
   │   │   ├── CUHK
   │   │   ├── RTF
   │   │   ├── SYNDOF
   

   Note:

   • [DATASET_ROOT] is currently set to ./datasets/. It can be specified by modifying config.data_offset in ./config.py.

4. Download pretrained weights of DMENet and unzip it as in [LOG_ROOT]/DMENet_BDCS/checkpoint/DMENet_BDCS.npz ([LOG_ROOT] is currently set to ./logs/).

5. Download pretrained VGG19 weights and unzip as in pretrained/vgg19.npy (for training only).

Logs

• Training and testing logs will be saved under [LOG_ROOT]/[mode]/:

  ├── [LOG_ROOT]
  │   ├── [mode]
  │   │   ├── checkpoint      # model checkpoint
  │   │   ├── log             # scalar/image log for tensorboard
  │   │   ├── sample          # sample images of training 
  │   │   ├── result          # resulting images of evaluation
  

  Note:

  • [LOG_ROOT] is currently set to ./logs/. It can be specified by modifying config.root_offset in ./config.py.

Testing final model of CVPR 2019

Please note that due to the server issue, the checkpoint used for the paper is lost.
The provided checkpoint is the new checkpoint that shows the closest evaluation results as in the paper.

Checkout updated performance with the new checkpoint.

• Test the final model by:

  python main.py --mode DMENet_BDCS --test_set CUHK

  Note:

  • Testing results will be saved in [LOG_ROOT]/[mode]/result/[test_set]/:

    ...
    ├── [test_set]
    │   ├── image                     # input defocused images
    │   ├── defocus_map               # defocus images (network's direct output in range [0, 1])
    │   ├── defocus_map_min_max_norm  # min-max normalized defocus images in range [0, 1] for visualization
    │   ├── sigma_map_7_norm          # sigma maps containing normalized standard deviations (in range [0, 1]) for a Gaussian kernel. For the actual standard deviation value, one should multiply 7 to this map.
    

  • Quantitative results are computed from matlab. (e.g., evaluation on the RTF dataset).

  • Options
    • --mode: The name of a model to test. The logging folder named with the [mode] will be created as [LOG_ROOT]/[mode]/. Default: DMENet_BDCS
    • --test_set: The name of a dataset to evaluate. CUHK | RTF0 | RTF1 | RTF1_6 | random. Default: CUHK
      • The folder structure can be modified in the function get_eval_path(..) in ./config.py.
      • random is for testing models with any images, which should be placed as [DATASET_ROOT]/random/*.[jpg|png].

• Checkout the evaluation code for the RTF dataset, and the deconvolution code.

Training & testing the network

• Train the network by:

  python main.py --is_train --mode [mode]

  Note:

  • If you train DMENet with newly generated SYNDOF dataset from this repo, comment this line and uncomment this line before the training.

• Test the network by:

  python main.py --mode [mode] --test_set [test_set]

  • arguments
    • --mode: The name of a model to train. The logging folder named with the [mode] will be created as [LOG_ROOT]/[mode]/. Default: DMENet_BDCS
    • --is_pretrain: Pretrain the network with the MSE loss (True | False). Default: False
    • --delete_log: Deletes [LOG_ROOT]/[mode]/* before training begins (True | False). Default: False

Citation

If you find this code useful, please consider citing:

@InProceedings{Lee_2019_CVPR,
    author = {Lee, Junyong and Lee, Sungkil and Cho, Sunghyun and Lee, Seungyong},
    title = {Deep Defocus Map Estimation Using Domain Adaptation},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2019}
}

Contact

Open an issue for any inquiries. You may also have contact with [email protected]

Resources

All material related to our paper is available via the following links:

Link
Paper PDF
Supplementary Files
Checkpoint Files
Datasets
SYNDOF Generation Repo

License

This software is being made available under the terms in the LICENSE file.

Any exemptions to these terms require a license from the Pohang University of Science and Technology.

About Coupe Project

Project ‘COUPE’ aims to develop software that evaluates and improves the quality of images and videos based on big visual data. To achieve the goal, we extract sharpness, color, composition features from images and develop technologies for restoring and improving by using them. In addition, personalization technology through user reference analysis is under study.

Please checkout other Coupe repositories in our Posgraph github organization.

Useful Links

• Coupe Library
• POSTECH CG Lab.