Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen Environments

Overview

• Researchers evaluated the usability of interactive personal assistants (IPAs) like Amazon Alexa for deaf and hard-of-hearing users in a simulated smart home kitchen environment
• Users performed activities that got their hands dirty, then tried two input methods for IPAs: American Sign Language (ASL) and touchscreen apps
• Participants significantly preferred using ASL over touch-based apps when their hands were dirty, although the preference was not significantly greater than when their hands were clean
• Participants expressed strong enthusiasm for ASL-based IPA interaction
• Preliminary observations suggest that having dirty hands may affect the way people sign, which could pose challenges for building IPAs that natively support sign language

Plain English Explanation

The researchers looked at how easy it is for deaf and hard-of-hearing people to use interactive personal assistants (IPAs) like Alexa in a simulated smart home kitchen. They had participants do tasks that got their hands dirty, then tested two ways the participants could give commands to the IPAs: American Sign Language (ASL) and touchscreen apps.

The participants significantly preferred using ASL over the touchscreen apps when their hands were dirty, although this preference wasn't much stronger than when their hands were clean. The participants also expressed a lot of enthusiasm for using ASL to interact with the IPAs.

Interestingly, the researchers noticed that having dirty hands seemed to affect how the participants signed, which could make it challenging to build IPAs that can natively understand sign language inputs. Overall, the study suggests that sign language could be a more natural and preferred way for deaf and hard-of-hearing people to control smart home devices, even when their hands are busy.

Technical Explanation

The researchers designed an experiment to assess the usability of IPAs for deaf and hard-of-hearing users in a simulated smart home kitchen environment. Participants were asked to perform activities that dirtied their hands, and then tried two different input methods for controlling the IPAs: ASL in a Wizard-of-Oz setup, and touchscreen-based smart home apps.

Usability ratings showed that participants significantly preferred the ASL-based interaction over the touch-based apps when their hands were dirty, though this preference was not significantly greater than in a previous study where their hands were clean. Participants also expressed strong enthusiasm for the ASL-based IPA interaction, as measured by Netpromoter scores and direct questions about their preferences.

Preliminary observations further suggested that having dirty hands may affect the way people sign, which could pose challenges for building IPAs that can natively support sign language input. This aligns with prior work on the impact of physical context on sign language and the need for real-time AI systems that can adapt to sign language in diverse settings.

Critical Analysis

The study provides valuable insights into the usability of IPAs for deaf and hard-of-hearing users, particularly in the context of hands-busy scenarios like a smart home kitchen. The preference for ASL-based interaction, even with dirty hands, suggests that sign language could be a more natural and intuitive control method for this user group.

However, the researchers note that the increase in preference for ASL over touch-based apps when hands were dirty was not as large as they had expected, based on prior work. This may indicate that other factors, such as familiarity with touchscreen interfaces, play a role in user preferences. Further research is needed to fully understand the tradeoffs and optimal design strategies for IPA interfaces for deaf and hard-of-hearing users.

Additionally, the preliminary observations about how dirty hands may affect signing patterns highlights an important technical challenge that needs to be addressed for building IPAs that can seamlessly support sign language input. More research is needed to understand the extent of this issue and develop robust solutions.

Overall, this study provides a valuable contribution to the understanding of accessibility and usability considerations for IPA technologies, particularly for users with hearing impairments. Continued exploration of these issues will be crucial for ensuring that smart home and voice-based AI systems are truly inclusive and accessible to all.

Conclusion

This study examined the usability of interactive personal assistants (IPAs) for deaf and hard-of-hearing users in a simulated smart home kitchen environment. The researchers found that participants significantly preferred using American Sign Language (ASL) over touchscreen-based apps when their hands were dirty, although this preference was not significantly greater than when their hands were clean.

Participants also expressed strong enthusiasm for the ASL-based IPA interaction, suggesting that sign language could be a more natural and intuitive control method for this user group. However, preliminary observations indicate that having dirty hands may affect the way people sign, which could pose challenges for building IPAs that can natively support sign language input.

Overall, this study highlights the importance of considering accessibility and usability for diverse user groups when designing voice-based AI systems and smart home technologies. Continued research in this area will be crucial for ensuring these technologies are truly inclusive and meet the needs of all users.