Signly is a desktop application written in Java language that can recognise and translate static American Sign Language gestures into English text and audio output. It captures the gestures using a small device Leap Motion Controller. The application aims to help hearing impaired people to communicate with the rest of the public by providing software that acts as an automated computer interpreter.
The application has been developed using the following tools:
-
Java Enterprise Edition 8.0 (Java EE)
-
NetBeans IDE 8.2
-
Neuroph Artificial Neural Network Framework
-
Leap Motion Controller & SDK (Leap Motion Orion 4.0.0)
-
FreeTTS 1.2.3
Before cloning the repo, make sure you have the following software installed on your computer:
Leap Motion Controller SDK (Leap Motion Orion 4.0.0)
- Plug in Leap Motion Controller to your computer
2. git clone https://github.com/MonikaCat/Signly
3. cd Signly/src/signly
- Run SignlyMain.java
The system achieved high accuracy of 84.80 %. Several letters including A, D, E, K, N, R accomplished lower value than anticipated due to high similarity between them and controller inability to correctly recognise gestures that involve placing finger on top of another.
- Monika Pusz - www.puszmonika.com
- The application integrates the hand visualizer developed by Github user RuZman (https://github.com/RuZman/LeapFX). The developer had been contacted and his approval obtained to integrate it with this project.
- For the purpose of this project the dynamic gestures J and Z were replaced with the temporary static representation.