AI boosts communication for deaf athletesーNHK WORLD-JAPAN NEWS

Key Concepts

• Deaf Olympics (Deaflympics)
• AI-powered communication system
• Sign language recognition
• Real-time translation (sign language to text)
• Japanese Sign Language (JSL)
• International Sign Language (ISL)
• American Sign Language (ASL)
• Data collection for AI training
• Gesture recognition accuracy

Deaf Olympics Communication Hub: AI-Powered Sign Language Translation

Tokyo recently hosted its inaugural Deaf Olympics, an event that brought together approximately 3,000 athletes with hearing impairments who utilize various sign languages. To facilitate interaction among these athletes, a novel AI-powered system was developed. This system, referred to as the "Deaf Olympics communication hub," is designed to convert sign language into text in real-time and can translate into multiple languages.

Development and Collaboration

The system is a product of a collaborative effort between the Tokyo Metropolitan Government and a major telecommunications company. It is capable of recognizing and processing Japanese Sign Language (JSL), International Sign Language (ISL), and American Sign Language (ASL). The primary benefit highlighted is the elimination of the need for human interpreters, as the system directly converts sign language into text, making it potentially easier to popularize.

Technical Challenges and Data Collection

Engineer Kumatsu, who has a hearing impairment, led the research into sign language recognition systems. He shared his personal experience of feeling isolated and left behind in conversations due to his hearing impairment. A significant hurdle in developing the system was the collection of diverse sign language data. While ample data for JSL was available, acquiring sufficient data for ISL presented a challenge. The AI needed to be trained to recognize the specific movements and idiosyncrasies inherent in different sign languages. Kumatsu emphasized, "Unless we gather vast amounts of this data, we cannot improve sign language recognition accuracy."

System Capabilities and Limitations

The system was operational for the opening ceremony, capable of translating sign language into 11 languages. However, the transcript notes that the system still encounters difficulties with certain gestures, illustrating this with the example: "if you sign I want to eat hamburger steak, it produces a different translation." Despite these limitations, the development team views the Deaf Olympics as a crucial opportunity to gather more data on sign language movements and idiosyncrasies to further train the AI and enhance its recognition accuracy and overall quality.

Future Applications

Tokyo and the collaborating telecommunications company express aspirations to deploy this system in other public spaces, such as hotels and tourist facilities, to improve accessibility and communication for individuals who use sign language.

Conclusion

The AI-powered communication hub developed for the Deaf Olympics represents a significant advancement in assistive technology for the deaf community. While challenges remain in achieving perfect gesture recognition across all sign languages, the system's ability to provide real-time translation from sign language to text offers a tangible solution for enhanced communication and inclusion. The ongoing efforts to collect diverse sign language data are critical for refining the AI's accuracy and expanding its applicability beyond the sporting event.