Initial Commit

Initial Commit

UI Developed

Bug Fix in Utils.py

Update README.md

Update README.md

.gitignore

@@ -0,0 +1,7 @@
+logs/
+venv/
+data/
+features/
+__pycache__/
+.ipynb_checkpoints/
+*.zip

README.md

@@ -1 +1,94 @@
-# face-recognition
+# Face Recognition Software
+
+I have explained the details about the model structure, dataset, and other implementational details in the blog post here. Here, I will provide the steps for using this repository.
+
+#  Installations
+
+The following steps are tested for ubuntu 20.04 with a python version of 3.8
+
+```
+sudo apt update
+sudo apt upgrade
+sudo snap install cmake
+```
+Clone the project to any directory and open the terminal in that directory.
+We will have to create some directories that will be essential for our storing images and features
+```
+mkdir data
+mkdir features
+```
+
+I generally like to create a virtual environment for each of my projects so the next step is optional.
+
+```
+virtualenv -p /usr/bin/python3.8 venv
+source venv/bin/activate
+```
+Now we will install all the pip packages required for running this applications. 
+```
+pip install -r requirement.txt
+```
+If you want to just try out the Browser-Based UI tool or run the notebook, then you can download the pre-trained model weights from [here](https://drive.google.com/file/d/1MegWliwXx2J-xHYX6iETl7hXUtLRk2sC/view?usp=sharing). After extracting the files, your directory should look like this.
+## Check you setup
+```
+.
+├── app.py
+├── config.py
+├── data
+├── features
+├── logs
+│   ├── func
+│   ├── model
+│   └── scalars
+├── notebooks
+│   ├── DatagGeneration.ipynb
+│   ├── Real-time-prediction.ipynb
+│   └── SiameseNetwork-TripletLoss.ipynb
+├── README.md
+├── reqirement.txt
+├── siameseNetwork.py
+├── static
+│   ├── css
+│   └── images
+├── templates
+│   ├── index.html
+│   └── results.html
+├── utils.py
+```
+
+## Running the browser-based tool
+
+If you have the same set of files and folders in the directory then you can run the following command
+```
+python app.py
+```
+The flask app will start and you will be able to collect training data for any new person and generate features for that person and check the real-time recognition. You can add as many people as you want to. The images collected from this tool will be added to the data folder and the corresponding features generated will be stored in the features folder.
+
+_**Note :** If you delete any person's images from the data folder, you need to delete the .pkl file inside the features folder as well. Also, the pickle file will be generated only when you hit submit images in the browser tool._
+
+## Running the Notebooks
+
+You can start the jupyter-lab or jupyter notebook server and check the notebooks folder. 
+Notebook [Real-time-prediction.ipynb](https://github.com/dedhiaparth98/face-recognition/blob/master/notebooks/Real-time-prediction.ipynb) can be used for evaluating the model. It's the notebook version of the browser-based tool. However, the prediction in real-time webcam frame is much faster here as the browser sends API calls to the backend and each image frame of the video is send whereas here, it's not necessary.
+
+Other instructions for running this notebook are provided in the notebook itself. However, the data directory is shared between this notebook and the browser-based tool.
+
+## Training from scratch
+
+If you wish to train your model and get your own weights, then you can use [SiameseNetwork-TripletLoss.ipynb](https://github.com/dedhiaparth98/face-recognition/blob/master/notebooks/SiameseNetwork-TripletLoss.ipynb). I had trained the same on colab and kept the lines of code for mounting the drive and other TensorFlow logging. Please refer to the blog post link above for learning more about the training details.
+
+## For Web Developers - I have a question
+
+I have tried using socket connection as well as ajax calls for sending data to the backend while running prediction calls on the images. It was counter-intuitive to know that the socket connection was giving me a slower frame rate than the ajax calls.
+
+The current implementation is with Ajax call but the commented code for the socket is kept in both frontend and backend. **So if you know why is socket slow than ajax ?** then please drop me a message on [twitter](https://twitter.com/Parth_dedhia98) or [linkedin](https://www.linkedin.com/in/parth-dedhia). Thanks in advance !!
+
+## References
+
+1. O. M. Parkhi, A. Vedaldi, A. Zisserman, Deep Face Recognition, British Machine Vision Conference, 2015.
+1. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman, VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018.
+3. F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR, 2015.
+4. G. Koch, R. Zemel, R. Salakhutdinov, Siamese Neural Networks for One-shot Image Recognition, ICML deep learning workshop. Vol. 2. 2015.
+5. [https://github.com/rcmalli/keras-vggface](https://github.com/rcmalli/keras-vggface)
+6. [https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly](https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly)
+7. [https://medium.com/datadriveninvestor/speed-up-your-image-training-on-google-colab-dc95ea1491cf](https://medium.com/datadriveninvestor/speed-up-your-image-training-on-google-colab-dc95ea1491cf)

app.py

@@ -0,0 +1,89 @@
+import os
+import cv2
+import json
+import base64
+import config
+import numpy as np
+from utils import generate_dataset_festures, predict_people
+from flask import Flask, request, render_template, jsonify
+# from flask_socketio import SocketIO, emit
+
+app = Flask(__name__)
+# app.config['SECRET_KEY'] = 'secret!'
+# socketio = SocketIO(app)
+
+generate_dataset_festures()
+
+@app.route('/')
+def index():
+    return render_template('index.html')
+
+@app.route('/submit-name', methods=['POST'])
+def submit_name():
+    if request.method == 'POST':
+        name = request.form['name'].lower()
+        if name not in os.listdir(config.data_dir):
+            return "SUCCESS"
+        return "FAILURE"
+
+@app.route('/submit-photos', methods=['POST'])
+def submit_photos():
+    if request.method == 'POST':
+        name = request.form['name'].lower()
+        images = json.loads(request.form['images'])
+
+        os.mkdir(os.path.join(config.data_dir, str(name)))
+
+        person_directory = os.path.join(config.data_dir, name)
+        for i, image in enumerate(images):
+            image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8)
+            image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR)
+            cv2.imwrite(os.path.join(person_directory, str(i) + '.png'), image)
+
+        generate_dataset_festures()
+
+        return "results"
+
+@app.route("/results")
+def results():
+    return render_template("results.html")
+
+@app.route("/predict-frame", methods=['POST'])
+def predict_frame():
+    if request.method == 'POST':
+        image = request.form['image']
+        image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8)
+        image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR)
+
+        image = predict_people(image)
+
+        retval, buffer = cv2.imencode('.png', image)
+        img_as_text = base64.b64encode(buffer)
+
+        return img_as_text
+
+# I have tried out the prediction code with socket library as well as with AJAX calls,
+# and for my system locally the ajax calls work much faster than the socket connection. 
+# Even I fell it's counter intuitive but still, its fast.
+
+# @socketio.on('connect', namespace='/socket-connection')
+# def socket_connection():
+#     emit('connection-response', {'data': 'Connected'})
+
+# @socketio.on('prediction', namespace='/socket-connection')
+# def prediction_image(message):
+#     image = message['data']
+#     image_numpy = np.fromstring(base64.b64decode(image.split(",")[1]), np.uint8)
+#     image = cv2.imdecode(image_numpy, cv2.IMREAD_COLOR)
+
+#     image = predict_people(image)
+
+#     retval, buffer = cv2.imencode('.png', image)
+#     img_as_text = base64.b64encode(buffer).decode('utf-8')
+
+#     emit('prediction-response', {'img': img_as_text})
+
+
+if __name__ == "__main__":
+    # socketio.run(app, debug=False)
+    app.run(debug=False)

config.py

@@ -0,0 +1,6 @@
+data_dir = './data'
+feature_dir = './features'
+people = None
+features = None
+model = None
+face_detector = None

notebooks/DatagGeneration.ipynb

@@ -0,0 +1,100 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import cv2\n",
+    "import dlib\n",
+    "import shutil\n",
+    "import numpy as np\n",
+    "from tqdm.notebook import tqdm\n",
+    "from imutils import face_utils"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "face_detector = dlib.get_frontal_face_detector()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "if os.path.isdir('./dataset'):\n",
+    "    shutil.rmtree('./dataset')\n",
+    "os.mkdir('./dataset')\n",
+    "os.mkdir('./dataset/images')\n",
+    "    \n",
+    "dataset_path = './dataset/images'\n",
+    "path = './vggface2_test/test'\n",
+    "list_of_images = []\n",
+    "for dirname in tqdm(os.listdir(path)):\n",
+    "    image_folder_path = os.path.join(path, dirname)\n",
+    "    os.mkdir(os.path.join(dataset_path, dirname))\n",
+    "    for image in tqdm(os.listdir(image_folder_path), leave=True, position=1):\n",
+    "        image_path = os.path.join(image_folder_path, image)\n",
+    "        img = cv2.imread(image_path)\n",
+    "        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
+    "        faces = face_detector(gray, 0)\n",
+    "        if len(faces) == 1:\n",
+    "            for face in faces:\n",
+    "                face_bounding_box = face_utils.rect_to_bb(face)\n",
+    "                if all(i >= 0 for i in face_bounding_box):\n",
+    "                    [x, y, w, h] = face_bounding_box\n",
+    "                    frame = img[y:y + h, x:x + w]\n",
+    "                    save_image = os.path.join(os.path.join(dataset_path, dirname), image)\n",
+    "                    cv2.imwrite(save_image, frame)\n",
+    "                    list_of_images.append(dirname + '/' + image)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with open('./dataset/list.txt', 'w') as f:\n",
+    "    for item in list_of_images:\n",
+    "        f.write(\"%s\\n\" % item)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}