Gallaudet experts drive accessibility of speech tech for deaf voices 

Gallaudet has received a subaward to literally include Deaf voices in the Speech Accessibility Project at the University of Illinois at Urbana-Champaign, which works to improve the accessibility of speech recognition technology. 

Some people use their voices to control tech, from cell phones and remote controls to home appliances and in transportation. Voice command capabilities are made possible through training AI and machine learning. The Speech Accessibility Project is creating datasets of more diverse speech patterns, which will in turn be used to train speech recognition technologies to work more accurately and correctly, regardless of how one speaks. 

As a bilingual university proud to be a home to deaf and hard of hearing people who communicate in a variety of ways, Gallaudet’s contribution to the Speech Accessibility Project can have profound impacts on the future of voice recognition accessibility. 

“Deaf and hard-of-hearing (DHH) individuals who use spoken language may have unique speech characteristics due to reduced auditory feedback. When speech systems recognize DHH users accurately, it affirms their communication preferences and reduces stigma around ‘non-standard’ speech,” says Dr. Raja Kushalnagar, Principal Investigator of Gallaudet’s subaward. 

Dr. Kushalnagar is a Professor in STAMP, Director of the B.S. in Information Technology, and Co-Director of the M.S. in Accessible Human-Centered Computing. He is joined on this project by Co-Principal Investigator Dr. Kara Hawthorne, who is an Associate Professor and Director of the Ph.D. Program in Hearing, Speech, and Language Sciences.

Led by the University of Illinois’ Beckman Institute for Advanced Science and Technology, the project brings researchers together with corporate entities Amazon, Apple, Google, Meta, and Microsoft. The project’s first two phases involved collecting voice samples from people with Amyotrophic Lateral Sclerosis (ALS), Parkinson’s disease, cerebral palsy, Down syndrome, and those who have had a stroke, or cerebrovascular accident (CVA). The Gallaudet team is involved in the next phase of the research, which focuses on DHH people and those who stutter. 

Some DHH people use speech in daily life, often alongside the use of American Sign Language, and the Gallaudet team says the fruits of this project will benefit them. “It can help them participate fully in mixed-hearing group conversations without needing an intermediary. It can respect DHH communication preferences (e.g., recognizing code-switching between ASL and English) and validate Deaf cultural and linguistic identities,” Kushalnagar says.

Mark Hasegawa-Johnson, a professor of electrical and computer engineering at the University of Illinois Urbana-Champaign and leader of the Speech Accessibility Project, said he’s delighted to be working with Gallaudet.

“For decades, I have been following and admiring the technology research being done by Dr. Kushalnagar and colleagues at Gallaudet,” Hasegawa-Johnson said. “They have demonstrated that innovative technology arises from new ways of thinking about human interaction. Gallaudet University has always encouraged that type of originality, and I am very pleased to have a chance to work with them.”

The Gallaudet team aims to recruit over 400 DHH participants. In practical terms, paid volunteers record speech samples by reading out loud hundreds of scripted sentences and responding to a number of narrative prompts (for instance, responding to the question “What are your hobbies?”). The research is phased in over time and shared with corporate partners to make iterative improvements in products that use speech recognition. 

Kushalnagar says, “These customized speech recognition systems can work alongside sign language recognition systems, creating more inclusive multimodal platforms for communication, education, and content creation.” The Gallaudet team is excited about future applications of the research: by including DHH people in the development of speech tools, the project reinforces accessible communication for all–Deaf, Hard of Hearing, and Hearing people.