aud2lips

Introduction

GAN based Lip Movement synthesis [https://github.com/ayushanand18/aud2lips]

• TensorFlow Keras based conditional Generative Adversarial Network model to synthesize lip movement. The model takes input as a facial image and spectrogram of the speech and generates a facial image with corresponding lip movement.
• Currently under development.

Usage

• Make sure you have Jupyter, TensorFlow Keras, matplotlib, opencv, numpy, tqdm and other requirement libraries installed.
• Run the 2-train-gan-relook.ipynb Notebook.

Author

Ayush Anand

References

@inproceedings{RekikICIAR14,
author    = {Ahmed Rekik and Achraf {Ben-Hamadou} and Walid Mahdi},
title     = {A New Visual Speech Recognition Approach for {RGB-D} Cameras},   
booktitle = {Image Analysis and Recognition - 11th International Conference, {ICIAR}  2014, Vilamoura, Portugal, October 22-24, 2014}
year      = {2014},   pages     = {21--28} }

@misc{https://doi.org/10.48550/arxiv.1611.07004,
  doi = {10.48550/ARXIV.1611.07004},
  url = {https://arxiv.org/abs/1611.07004},
  author = {Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A.},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Image-to-Image Translation with Conditional Adversarial Networks},
  publisher = {arXiv},
  year = {2016},
  copyright = {arXiv.org perpetual, non-exclusive license}
}