Face Recognition Using One Shot Learning (Siamese network) and Model based (PCA) with FaceNet_Pytorch

Refer to the report for results!

https://aravindchandradoss.github.io/facenet-pytorch/

This repo is build on top of facenet-pytorch and tensorflow-facenet

Quick start

you can directly use the embedded data (embedname.npy : label and embedimg.npy : embedded image features) by running python run.py (step 4) and check the results, or else you can setup FaceNet and use your own data.

1. Either install using pip:

   pip install facenet-pytorch

   or clone this repo, removing the '-' to allow python imports:

   git clone https://github.com/AravindChandradoss/facenet-pytorch.git facenet_pytorch

2. If you are cloning, make sure to have the ROOT of the clone repo in you PYTHONPATH

   export PYTHONPATH=/path/to/root/of/facenet_pytorch

3. Run preprocess.py and emb.py to preprocess and embed the images

   cd CV_P4
   python preprocess.py
   python emb.py

   This will create embedname.npy (label) and embedimg.npy (embedded image features)
4. Now, run run.py

   cd CV_P4
   python run.py

Setting up FaceNet-Pytorch

1. Either install using pip:

   pip install facenet-pytorch

   or clone this repo, removing the '-' to allow python imports:

   git clone https://github.com/timesler/facenet-pytorch.git facenet_pytorch

   or use a docker container (see timesler/jupyter-dl-gpu):

   docker run -it --rm timesler/jupyter-dl-gpu pip install facenet-pytorch && ipython

2. In python, import the module:

   from facenet_pytorch import MTCNN, InceptionResnetV1

3. If required, create a face detection pipeline using MTCNN:

   mtcnn = MTCNN(image_size=<image_size>, margin=<margin>)

4. Create an inception resnet (in eval mode):

   resnet = InceptionResnetV1(pretrained='vggface2').eval()

5. Process an image:

   from PIL import Image
   
   img = Image.open(<image path>)
   
   # Get cropped and prewhitened image tensor
   img_cropped = mtcnn(img, save_path=<optional save path>)
   
   # Calculate embedding (unsqueeze to add batch dimension)
   img_embedding = resnet(img_cropped.unsqueeze(0))
   
   # Or, if using for VGGFace2 classification
   resnet.classify = True
   img_probs = resnet(img_cropped.unsqueeze(0))

See help(MTCNN) and help(InceptionResnetV1) for usage and implementation details.

Pretrained models

See: models/inception_resnet_v1.py

The following models have been ported to pytorch (with links to download pytorch state_dict's):

Model name	LFW accuracy (as listed here)	Training dataset
20180408-102900 (111MB)	0.9905	CASIA-Webface
20180402-114759 (107MB)	0.9965	VGGFace2

There is no need to manually download the pretrained state_dict's; they are downloaded automatically on model instantiation and cached for future use in the torch cache. To use an Inception Resnet (V1) model for facial recognition/identification in pytorch, use:

from facenet_pytorch import InceptionResnetV1

# For a model pretrained on VGGFace2
model = InceptionResnetV1(pretrained='vggface2').eval()

# For a model pretrained on CASIA-Webface
model = InceptionResnetV1(pretrained='casia-webface').eval()

# For an untrained model with 100 classes
model = InceptionResnetV1(num_classes=100).eval()

# For an untrained 1001-class classifier
model = InceptionResnetV1(classify=True, num_classes=1001).eval()

Both pretrained models were trained on 160x160 px images, so will perform best if applied to images resized to this shape. For best results, images should also be cropped to the face using MTCNN (see below).

By default, the above models will return 512-dimensional embeddings of images. To enable classification instead, either pass classify=True to the model constructor, or you can set the object attribute afterwards with model.classify = True. For VGGFace2, the pretrained model will output logit vectors of length 8631, and for CASIA-Webface logit vectors of length 10575.

References

1. David Sandberg's facenet repo: https://github.com/davidsandberg/facenet

2. F. Schroff, D. Kalenichenko, J. Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering, arXiv:1503.03832, 2015. PDF

3. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman. VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018. PDF

4. D. Yi, Z. Lei, S. Liao and S. Z. Li. CASIAWebface: Learning Face Representation from Scratch, arXiv:1411.7923, 2014. PDF

5. K. Zhang, Z. Zhang, Z. Li and Y. Qiao. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks, IEEE Signal Processing Letters, 2016. PDF