R & D

22

Sign language summaries of competition information and results increase the enjoyment of sports programs and sports news for persons with impaired hearing, and STRL is moving forward with research on presenting sports com-petition information as computer-generated sign language. Here, we describe a translation assistant system for the ef-ficient production of sign language CG (computer graphic) from Japanese text. Motion capture is used to prepare sign language finger motion and facial expression data. Japanese text that is input to the system is automatically translated to sign language word strings by machine translation. How-ever, the context cannot be completely analyzed for some input sentences, so the order in which the sign language is presented may be incorrect. To address this problem, we developed a user interface that facilitates the rearrangement and insertion of sign language words.

In the example presented in the figure, a nod action that represents a semantic separator is inserted in the wrong

place. The interface we developed displays the word string and makes it possible to change the order or insert sign language words intuitively by a mouse operation to make corrections while checking the generated sign language CG. There is also a function that records corrections for use as training data and a function for later reuse of the correc-tion history as templates. These functions can be used to improve the translation accuracy.

The system requires that some work be performed by persons who understand sign language, but the amount of training data increases with the continued use of the system so production efficiency should increase over time. In fu-ture work, we will conduct trials on sports news programs and extend the application of computer-generated sign language to general news programs.

Shuichi Umeda

Translation Assistant System for Sign Language CG from Japanese Text for Sports News

Computer-generated sign language

AI support

Training data

Machine translation Motion synthesis

・Correction of word string・Correction of computer graphic movement

Recording of corrections

‘Tokyo’ , ‘location’ , ‘nod’ , ‘Olympic’ , ‘nod’ , ‘be held’ , ‘nod’

Japanese text: “The Olympics are held in Tokyo”

Sign language word string:

Sign language word string

User interface for the production support system

Tokyo location nod Olympic nod be held nod

nodRearrangement of machine translation results

Sign language generation process in the translation assistant system