Does Alexa understand your child? Researchers at Western University are looking into this

Olivia Daub’s two-year-old son is obsessed with “doodies.” He talks about them and yells for them at 5 a.m. every day.

Daub said most people have no idea what her son is saying, but she knows to bring him the small, dark blue fruit he really wants: blueberries.

“We’ve all been children and we’ve all had the experience of not being understood by adults. Conversely, we (adults) have had a really hard time understanding children because they produce speech and language in ways that are different from adults.”

It’s even more difficult for artificial intelligence (AI) to understand children’s conversations, said Daube, an assistant professor in the School of Communication Sciences and Disorders at Western University in London, Ontario. That’s why she’s leading new research into how AI can better understand the way children talk.

Dub said that automated speech recognition software, such as automatic closed captions on Zoom meetings and Amazon’s Alexa virtual assistant, has become good at recognizing adult speech, but it still struggles to accurately understand what young children are saying.

“I think we’ve all seen that YouTube clip of a kid asking Alexa to play a song, and doing something completely different and really inappropriate,” she said. “This study is trying to understand how we can leverage AI and machine-learning principles to improve recognition in toddlers and preschoolers.”

To do this, she is working with Western electrical and computer engineering assistant professor Sudeh Nikan to train an AI model on children’s common speech patterns and shortcuts.

“Most of the speech models we have are trained with adult speech, which is why most of these models are not very successful at recognizing children’s speech, especially the mistakes they make,” Nikan said.

“You have to provide examples for the AI ​​to be able to understand and differentiate common mistakes and speech disorder problems.”

How will the study work?

Daub plans to bring 30 children to play, tell stories and talk to research assistants. Each session will be recorded and transcribed by humans, who will also collect data on the children’s speaking patterns.

A common pattern, Daub said, is that many English-speaking children struggle to pronounce the “r” sound and use the “w” sound instead.

Such data will be handed over to Nikan, which will feed the information into a private AI model to train it.

“We can fine-tune these models using data annotated specifically for this purpose,” Nikan said, adding that the AI ​​models will also be trained with some of the data already available from OpenAI Online.

Daub and his team have met nine children so far and are still looking for more study participants.

Clinical and everyday uses for AI models

While the research is in its early stages, Dub and Nikan said their goal is to train an AI model that can be used in a clinical setting, to help speech language pathologists analyze and transcribe what children are saying.

He said, “I don’t think we’ll ever be 100 percent accurate unless we’re following kids 24 hours a day… but I think we can get much closer than where we are.”

Down the line, Daub said, if AI can better understand preschoolers, it could improve tools like closed captioning and voice-activated accessibility software, as well as give kids more room to play with technology.

“We can really think creatively about what these little people can contribute to society. They’re not just consumers of the world around them. Giving them access to technology is also an important consideration,” Dub said.