Skip to content

RE-LFDE: A Resource-Efficient Hardware Accelerator for Low-bit Light Field Image Depth Estimation

Notifications You must be signed in to change notification settings

sansi-zhang/RE-LFDE

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

84 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RE-LFDE

RE-LFDE: A Resource-Efficient Hardware Accelerator for Low-bit Light Field Image Depth Estimation

RE-LFDE Network

Software Preparation

Requirement

  • PyTorch 1.3.0, torchvision 0.4.1. The code is tested with python=3.8, cuda=11.0.
  • A GPU with enough memory

Datasets

  • We used the HCI 4D LF benchmark for training and evaluation. Please refer to the benchmark website for details.

Path structure

  • ./dataset

    • training
      Location of the training data.
    • validation
      Verify where the data is stored.
  • ./Figure

    • paper_picture
      Images from the paper.
    • hardware_picture
      Hardware design picture.
  • ./Hardware
    A file containing a series of hardware for the RE-LFDE and ablation experimental groups.

    • RE-LFDE
      It contains the bit files and the hwh files for hardware, and the project code for PYNQ implementation.
    • Net_Lp
      Contains the bit files and the hwh files for hardware.
    • Net_Op
      Contains the bit files and the hwh files for hardware.
    • Net_w2bit
      Contains the bit files and the hwh files for hardware.
    • Net_w8bit
      Contains the bit files and the hwh files for hardware.
  • ./implement
    RE-LFDE implementation files and data preprocessing file on Pytorch.

  • ./jupyter
    Network execution scripts, as well as some algorithm implementation scripts.

  • ./log
    Log files that record the accuracy of each verification scenario during verification.

  • ./loss
    loss Drop image, recording the loss for each validation.

  • ./model
    Network and regular functions to call.

  • ./param
    The checkpoint of the networks is stored here.

  • ./Results
    Store network test results, pfm files and converted png files.

    • our network
      • Net_Full
      • Net_Quant
    • Necessity analysis
      • Net_None
      • Net_77
      • Net_DPP
    • Performance improvement analysis
      • Net_Lp
      • Net_w2bit
      • Net_w8bit
      • Net_Op

Train

  • Set the hyper-parameters in parse_args() if needed. We have provided our default settings in the realeased codes.

  • You can train the network by calling implement.py and giving the mode attribute to train.
    python ./implement/implement.py --net Net_Full --n_epochs 3000 --mode train --device cuda:0

  • Checkpoint will be saved to ./param/'NetName'.

Valition and Test

  • After loading the weight file used by your domain, you can call implement.py and giving the mode attribute to valid or test.
  • The result files (i.e., scene_name.pfm) will be saved to ./Results/'NetName'.

Results

Contrast with the state-of-the-art work

Hardware Preparation

Hardware Requirement

  • ZCU104 platform
  • A memory card with PYNQ installed.
    For details on the initialization of PYNQ on ZCU104, please refer to the Chinese version of the blog "PYNQ".
  • Vivado Tool Kit (vivado, HLS, etc.)
  • An Ubuntu with more than 16GB of memory (the Vivado tool is faster when used in Ubuntu)

Hardware overall

Citiation

If you find this work helpful, please consider citing:
Our paper is currently under submission

Contact

Welcome to raise issues or email to Chuanlun Zhang([email protected] or [email protected]) for any question regarding this work

About

RE-LFDE: A Resource-Efficient Hardware Accelerator for Low-bit Light Field Image Depth Estimation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages