Speech Translation into Pakistan Sign Language

Abstract: ABSTRACT Context: Communication is a primary human need and language is the medium for this. Most people have the ability to listen and speak and they use different languages like Swedish, Urdu and English etc. to communicate. Hearing impaired people use signs to communicate. Pakistan Sign Language (PSL) is the preferred language of the deaf in Pakistan. Currently, human PSL interpreters are required to facilitate communication between the deaf and hearing; they are not always available, which means that communication among the deaf and other people may be impaired or nonexistent. In this situation, a system with voice recognition as an input and PSL as an output will be highly helpful. Objectives: As part of this thesis, we explore challenges faced by deaf people in everyday life while interacting with unimpaired. We investigate state of art work done in this area. This study explores speech recognition and Machine translation techniques to devise a generic and automated system that converts English speech to PSL. A prototype of the proposed solution is developed and validated. Methods: Three step investigation is done as part of thesis work. First, to understand problem itself, interviews were conducted with the domain experts. Secondly, from literature review, it is investigated whether any similar or related work has already been done, state of the art technologies like Machine translation, speech recognition engines and Natural language processing etc. have been analyzed. Thirdly, prototype is developed whose validation data is obtained from domain experts and is validated by ourselves as well as from domain experts. Results: It is found that there is a big communication gap between deaf and unimpaired in Pakistan. This is mainly due to the lack of an automated system that can convert Audio speech to PSL and vice versa. After investigating state of the art work including solutions in other countries specific to their languages, it is found that no system exists that is generic and automated. We found that there is already work started for PSL to English Speech conversion but not the other way around. As part of this thesis, we discovered that a generic and automated system can be devised using speech recognition and Machine translation techniques. Conclusion: Deaf people in Pakistan lack a lot of opportunities mainly due to communication gap between deaf and unimpaired. We establish that there should be a generic and automated system that can convert English speech to PSL and vice versa. As part of this, we worked for such a system that can convert English speech to PSL. Moreover, Speech recognition, Machine translation and Natural language processing techniques can be core ingredients for such a generic and automated system. Using user centric approach, the prototype of the system is validated iteratively from domain experts.