LIP READING AND PREDICTION SYSTEM BASED ON DEEP LEARNING

Abstract

Speech perception is characterized as a
multimodal process, which means it elicits several
meanings. Understanding a message can be aided by,
and in some cases even made necessary by, lip reading,
which overlays visual cues on top of auditory signals.
Lip-reading is a crucial field with many uses, including
biometrics, speech recognition in noisy environments,
silent dictation, and enhanced hearing aids. It is a
challenging research project in the area of computer
vision, whose major goal is to watch the movement of
human lips in a video and recognize the textual content
that goes with it. Yet, due to the constraints of lip
changes and the depth of linguistic information, the
complexity of lip identification has increased, which has
slowed the growth of study themes in lip language.
Nowadays, deep learning has advanced in several
sectors, giving us the confidence to perform the task of
lip recognition. Lip learning based on deep learning
often entails extracting features and comprehending
images using a network model, as opposed to classical
lip recognition that recognizes lip characteristics. The
design of the network framework for data gathering,
processing, and data recognition for lip reading is the
main topic of this discussion. In this research, we
created a reliable and accurate method for lip reading.
We first isolate the mouth region and segment it, after
which we extract various aspects from the lip image,
such as the Hog, Surf, and Haar features. Lastly, we use
Gated Recurrent Units to train our deep learning model
(GRU).