ML2: Object Oriented Hand Classifier Recognition with LEAP and Unity

We are investigating sign recognition interfaces to use interactivity in learning. LEAP Motion is a motion sensor system that tracks the hands and implements those movements in 3D environments.

Our goal is to use LEAP Motion to identify parameters of different ASL handshapes that are object-specific classifiers. This project aims to create an immersive game experience where young deaf learners will learn the correct handshapes and movements to guide the object.

Through this project, we want to understand the integration of LEAP Motion and Unity (game engine) to better define sign/movement recognition that can contribute to language learning programs.