Translation of Continuous Signs in Ethiopian Sign Language to Amharic Text
No Thumbnail Available
Date
2017-11-05
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Addis Ababa University
Abstract
A Sign Language is a visual language that uses a system of manual, facial and body movements as the means of communication. Sign language is not an universal language, and different sign languages are used in different countries, like the many spoken languages all over the world. Sign languages that exist around the world are usually identified by the country where they are used such as Ethiopian Sign Language (EthSL). Sign language is the basic alternative communication method between hearing impaired people and several dictionaries of words have been defined to make this communication possible. Even if it is widely used in the hearing impaired community, they struggle to communicate with hearing people due to the language barrier. Due to this communication gap hearing impaired people encounter so many problems in their daily life since they are living with the people who communicate with spoken languages. Unfortunately, few people have knowledge of sign language in our daily life. In general, interpreters can help us to communicate with these challengers, but they only can be found in Government Agencies. Moreover, it is expensive to employ interpreter on personal behalf and inconvenient when privacy is required. Consequently, it is very important to develop a system which fills the communication gap between the hearing impaired and hearing people. Many researches are conducted on the recognition of EthSL but mostly they are limited to recognition of isolated words and highly affected by lighting and complex background. The goal of this study is to recognize Continuous Signs in EthSL.
In this research, we use Three Dimensional (3D) depth information from hand motions and body joints generated from Microsoft Kinect. After extracting manual and non-manual sign language features, the system stores them in a gesture dictionary. A Random Forest algorithm is used to match gestures with those stored in the dictionary and later converted to Amharic text. we proposed language model providing simple solution with inverted indexing concept to reorder topic-comment pattern and the subject topic pattern of EthSL sentence to subject-object-verb sentence pattern of spoken Amahric language. We also address challenges such as Movement Epenthesis (ME) and sign segmentation which appears in continuous sign language recognition by analyzing hand movement pauses. The performance of the system was measured in two categories at: word, and sentence level and we got system accuracy of 84.4% at word level and 60% at sentence level.
Description
Keywords
Sign Language, Random Forest, Recognition, Continuous Translation