NASA Tech Tuesday: Seeing Is Communicating

Communicating when a traumatic brain injury, stroke, or disease has made speech impossible can be daunting. Specialized eye-tracking technology uses eye movement to enable people living with disabilities to connect one-on-one over the phone or via the internet.

Eye-tracking systems for computers pinpoint a person’s gaze – where the eye is looking at a screen – by reflecting infrared light off the cornea and capturing it with a camera, using image-processing software to determine the eye’s orientation. The technology isn’t new, but it has become much more widely accessible, thanks in part to a collaboration between NASA’s Jet Propulsion Laboratory in Southern California and a Fairfax, Virginia-based company called Eyegaze Inc.

When the company built the first model in 1988, its computers were bulky, requiring three shipping boxes for equipment and a company staff member to set up the system. That cost limited access, and the learning process could be intimidating.

In 1998, NASA and Eyegaze entered a public-private partnership via Congressional funding to make the hardware smaller, more portable, and affordable without compromising efficiency. It also reduced the weight of the original system by six times and its volume by nearly the same factor. Other advancements served as a springboard for two more decades of development. By collaborating with JPL, the two entities were able to miniaturize and improve the company’s Eyegaze Edge system and lower costs, eliminating barriers to ownership of this communications technology.

“Working with NASA, we were able to make the device less bulky,” said Preethi Vaidyanathan, an engineer with Eyegaze. “Since then, we integrated the external components into a small camera.” It mounts above or below a standard computer screen and requires less than 15 seconds to calibrate to an individual’s gaze.

Visual Surfing

As personal electronic devices and internet access became commonplace, Eyegaze customers wanted to do more than type. The company’s work with NASA and other government partners put it in a position to meet that demand.

Eyeworld integrates with computers, mobile phones, and tablets, allowing the Eyegaze camera to function as an external mouse and keyboard. By enabling almost any computer function, it lets users chat online, post to social media, send emails, text, or make phone calls. It’s also possible to change room lighting, adjust thermostat settings, and operate other environmental controls using Amazon Echo and Google Home via pages of specialized buttons with one-glance button controls.

During the COVID-19 pandemic, Eyegaze integrated communications programs such as Zoom and WhatsApp to allow people to connect with their family and friends. To help combat a sense of isolation, the company added meditation apps and Window Swap, which stream the views from different windows worldwide. Virtual museums take the viewer on guided tours. A music streaming service syncs to favorite music apps and offers the International Radio Garden app to sample music from other cultures. Access to online gaming communities provides another way to connect.

“We are constantly thinking about what the customers want. One thing is freedom of movement, so controlling a wheelchair gives them that to some extent,” said Vaidyanathan. Eyegaze Edge integrates with the Ability Drive application used with motorized mobility devices. Looking at a specific button enables hands-free control of a wheelchair’s motion.

The Right to Speak

Numerous medical conditions can lead to the kind of physical paralysis making it impossible to speak. Well-publicized brain and spinal cord injuries in athletes have led to broader awareness. But there is a host of other causes, such as cerebral palsy, ALS, multiple sclerosis, muscular dystrophy, and spinal muscular atrophy. These conditions can result in locked-in syndrome, which paralyzes all voluntary muscles except those that control eye movement. Eye-tracking software is a lifeline for patients, using what may be their only remaining voluntary movement, but even that can be impeded.

Eyegaze Edge measures several eye features, making it possible for the tracker to work even when less of the pupil is visible. The company employs clinical specialists who can troubleshoot issues that may prevent customers from using the system, eye conditions like cataracts, dry eyes, and eyelids that droop, partly covering the pupil.

The company now serves individuals in 44 countries, including nations in Southeast Asia and Africa. Adults, children as young as 18 months old, military veterans, and others can communicate using only their eyes. If only vertical eye movement is possible, Eyegaze Edge offers a special vertical keyboard screen. Pages of commonly used phrases communicate a statement with a single glance. Large, easy-to-use buttons allow patients to communicate directly with healthcare providers and caregivers for everyday needs.

Clinicians use Eyegaze Edge and teach their patients how to communicate using the technology. In some cases, Vaidyanathan said, this technology even allows people to share their final thoughts with family and friends.

“NASA helped us get our technology to the size of a laptop – small and sleek. But we continue to investigate and design our solutions to meet ongoing needs. Communication is a key right, so we must accommodate these different challenges,” she said.

NASA has a long history of transferring technology to the private sector. The agency’s Spinoff publication profiles NASA technologies that have transformed into commercial products and services, demonstrating the broader benefits of America’s investment in its space program. Spinoff is a publication of the Technology Transfer program in NASA’s Space Technology Mission Directorate (STMD).

For more information on how NASA brings space technology down to Earth, visit:

spinoff.nasa.gov

Written by Margo Pierce
NASA’s Spinoff Publication