Avatar, robotics, and visual language experts from around the world collaborate at Gallaudet

“Robotics and Visual language learning: What Does the Future Hold for Young Children?”

More than 40 avatar, robotics, and visual language scientists gathered at Gallaudet University on November 15 and 16 for a National Science Foundation (NSF) Signing Creatures Workshop. Sharing discoveries from their different scientific disciplines, workshop participants worked together to advance ideas about the creation of innovative avatar and robotic learning, language, and reading tools aimed at enriching the lives of young children.  

The Workshop began with a fast-paced event that was open to the public called “The Knowledge Explosion” in which each of the 41 participants presented in just two minutes the overarching questions in their science that propelled them, identified the major scientific discoveries that they have made spanning their career, and ended with their field’s next-step burning questions. Among the attendees included experts from Gallaudet University, Yale University, Rutgers University, PBS KIDS Interactive, Disney Research, Stockholm University, Massachusetts Institute of Technology, University of Hamburg (Germany), University of Southern California, Drexel University, Georgia Institute of Technology, and more.

The Workshop promoted the NSF’s priority to advance groundbreaking interdisciplinary communication across visual language, robotics, and avatar sciences, and the Science of Learning. “Our main Workshop objectives were successfully achieved. From multiple scientific points of view, participants assessed the potential to create innovative tools and funding revenues for signing avatars and robots that would provide early language experience and interaction with young children. Of course, these creations are only to be tools supporting social interaction and communication, and never to replace vital human social interaction,” said Dr. Laura-Ann Petitto, workshop chair, and co-principal investigator and science director of the NSF Science of Learning Center, Visual Language and Visual Learning (VL2) at Gallaudet University, as well as the scientific director of the Petitto Brain and Language Laboratory (BL2) for neuroimaging.

“Gallaudet University has a unique obligation to contribute knowledge and scholarship likely to benefit the deaf and hard of hearing community,” said President T. Alan Hurwitz. “Events like the Signing Creatures Workshop help us accomplish this mission and also to reach Goal E of our Strategic Plan which is to establish Gallaudet as the epicenter of research, development and outreach leading to advancements in knowledge and practice for deaf and hard of hearing people and all humanity.”

As a key Workshop outcome, participants formed new interdisciplinary research networks to address the main issue of signing creatures and socially assistive robotics that will facilitate acquisition and fluency in children’s language, reading and literacy, especially the young visual learner. “The opportunities ahead of us are fantastic. Our colleagues now see Gallaudet University as a must-have partner in engaging new research questions and product ideas. The ideas we came up with — young visual learners are about to get some amazing new toys in the next decade!” said Adam Stone, workshop manager and Ph.D. student in Gallaudet’s new Educational Neuroscience program.

The workshop resulted from a NSF grant written by Petitto (PI, IIS-1343948). Dr. David Traum, the grant’s co-principal investigator and workshop co-chair, is a principal scientist at the Institute for Creative Technologies at the University of Southern California. Both VL2 and BL2 hosted the workshop.

The capacity for avatar 3-D visual image rendering, in combination with advancements in the discipline of robotics and its representation of more fluid human movement, together make possible a vast array of potential teaching and learning educational translational products, some of which can – for the first time- be ideally suited for the educational needs of the young deaf visual learner.  Artificial avatar creatures, including animated signing virtual humans and robots, can provide visual inputs to deaf children built within communicative toys and games, similar to learning products readily available to hearing children.

Petitto provided pertinent research evidence from language acquisition and neural plasticity demonstrating that all children must receive early and systematic language exposure in order to develop language and reading mastery. “Despite the mounting scientific evidence regarding the healthy brain and developmental benefits afforded to young deaf children with early exposure to a signed language, this is often not readily available to deaf children who are born to non-signing parents,” Petitto noted.

Dr. Tom Allen, co-principal investigator of VL2, observed: “The workshop provided a thrilling context for innovation. Working as a team, Gallaudet scientists and translation experts joined with avatar and robotic scientists to take revolutionary steps towards the creation of learning tools for all society. The workshop closed with especially exciting ideas for generations of young visual learners to come.”

Gallaudet University, federally chartered in 1864, is a bilingual, diverse, multicultural institution of higher education that ensures the intellectual and professional advancement of deaf and hard of hearing individuals through American Sign Language and English. Gallaudet maintains a proud tradition of research and scholarly activity and prepares its graduates for career opportunities in a highly competitive, technological, and rapidly changing world.

Visual Language and Visual Learning (VL2) is a Science of Learning Center in the United States, funded by the National Science Foundation, and is based at Gallaudet University in Washington, D.C. VL2 is a collaborative effort with more than 15 labs nationwide, all interested in the visual learning process. VL2 seeks to understand more about how learning through visual processes, visual language, and visually based social experience contributes to the development of language, reading, and literacy, and in ways that provide fascinating cognitive and linguistic advantages to the young visual learner. This knowledge will not only benefit deaf and hard of hearing learners, but all humans.