Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal Assistants

Overview

• This study evaluated the usability of different input methods for intelligent personal assistants (IPAs) among deaf participants in a smart home environment.
• The researchers used a Wizard-of-Oz methodology to compare American Sign Language (ASL), Tap to Alexa, and smart home apps.
• The results showed a slight preference for ASL over the other input methods.
• The linguistic analysis of the participants' signing revealed a diverse range of expressions and vocabulary.

Plain English Explanation

In this study, the researchers looked at different ways for deaf people to interact with intelligent personal assistants (IPAs) like Alexa or Google Assistant in a smart home setting. They used a special method called Wizard-of-Oz, which means the researchers pretended the IPA was working, even though it was actually a person controlling the responses.

The researchers tested three different ways for the deaf participants to communicate with the IPA:

1. American Sign Language (ASL)
2. Tapping on the Alexa device
3. Using a smart home app

The results showed that the deaf participants slightly preferred using ASL over the other two options. When the researchers analyzed the signs the participants used, they found a wide variety of vocabulary and expressions. On average, each participant used about 47 different signs and 10 different fingerspelled words.

The researchers think this information can help design IPA systems that work better for deaf users, starting with limited-vocabulary apps and eventually working towards understanding full ASL. This could make it easier for deaf people to use smart home technology and voice assistants in the future.

Technical Explanation

The researchers used a Wizard-of-Oz methodology to evaluate and compare the usability of three input methods for interacting with IPAs in a limited-domain smart home environment:

1. American Sign Language (ASL)
2. Tap to Alexa
3. Smart home apps

The study involved 23 deaf participants who completed a series of tasks using each input method. The researchers observed the participants and collected data on their performance, preferences, and linguistic expressions.

The results showed a slight usability preference for ASL over the other two input methods. Linguistic analysis of the participants' signing revealed a diverse range of vocabulary and expressions, with an average of 47 ± 17 signs and 10 ± 7 fingerspelled words per participant, for a total of 246 different signs and 93 different fingerspelled words across all participants.

The researchers discuss the implications of these findings for the design of limited-vocabulary applications as a stepping-stone toward general-purpose ASL recognition in the future. This could help make AI-based automated speech therapy tools and voice-based interfaces more accessible for the deaf community.

Critical Analysis

The study provides valuable insights into the usability of different input methods for deaf users interacting with IPAs in a smart home environment. However, the researchers acknowledge that the limited-domain nature of the tasks may have influenced the participants' preferences and linguistic expressions.

Additionally, the small sample size of 23 participants may not be representative of the broader deaf population, and the Wizard-of-Oz methodology may not accurately reflect the performance of a fully functional IPA system. Further research is needed to validate the findings in more realistic settings and with larger, more diverse participant groups.

The researchers also note the need for continued development of sign language recognition technology to support more natural and expressive interactions between deaf users and IPAs. Integrating advanced natural language processing and computer vision capabilities will be crucial for realizing the full potential of these systems.

Conclusion

This study provides valuable insights into the usability of different input methods for deaf users interacting with intelligent personal assistants in a smart home environment. The findings suggest a slight preference for American Sign Language, and the linguistic analysis reveals the diverse range of expressions and vocabulary used by the participants.

These insights can inform the design of limited-vocabulary applications as a stepping-stone towards more advanced, general-purpose sign language recognition capabilities. This could ultimately lead to the development of IPA systems that are more accessible and inclusive for the deaf community, empowering them to fully participate in the smart home revolution.