Technology

Vocal Joystick

People with motor disabilities often find it frustrating or impossible to use a computer mouse. Now, engineers in the US have developed a program that provides an alternative, by using one of the oldest and versatile modes of communication: the human voice. “There are many people who have perfect us of their voice, but don't have use of their hands and arms,” said Jeffrey Bilmes, associate professor of electrical engineering at the University of Washington, who led the team that developed the ‘Vocal Joystick’ software.

The technology already helps the disabled in activities such as browsing the Web, drawing on a screen, and playing a video game. Vocal Joystick – an inexpensive technology requires only a microphone, a computer with a standard sound card and a user who can produce vocal sounds.

Bilmes and his team assigned vowel sounds to correspond to mouse movements: Up is ‘a’ as in ‘cat’ right is ‘aw’ an in ‘Aw’, shucks. There are sounds for more complex movements as well: ‘Down and to the right’ is ‘oh’ as in bold.

The program also gives users control over the mouse pointer's speed. A very quiet ‘ee’ as in ‘beet’ will move the cursor slowly to the right; a loud ‘oo’ as in boot’ sends it plunging downward.

Once users are accustomed to the commands, they can do sophisticated on–screen work, such as illustrations or digital photo restoration. Early tests suggest that an experienced user of Vocal Joystick would have as much control as someone using a handheld device. In the laboratory, student jonathan Malkin, who helped develop the tool, uses Vocal Joystick to play a game called Fish Tale. It takes only two minutes to train the program for his voice.

The newest development uses the program to control a robotic arm. The pitch of the tone moves the arm up and down; other commands are unchanged. This is the first time that vocal commands have been used to control a three–dimensional object, Bilmes said.

Voice painter Philip Chavez, who has been using tools like speech recognition to create art on his computer for 15 years, is already praising the new painting application, VoiceDraw.

“Doing it by voice really allows me to work much longer,” he said. Although the researchers initially wondered if people would feel silly making the sounds, their first test reassured them.

“It's an exciting idea. I think its has tremendous potential,” said Kurt Johnson, a professor of rehabilitation medicine who is helping with the tests. Bilmes said he hopes people will become more adept at using the system over time. Future research will incorporate advanced controls that use more aspects of the human voice, such as vibrato, degree of nasality and trills.

A version also exists for operating a robotic arm, and Bilmes believes the technology could be used to control an electronic wheelchair.

Source: A moving speech, Mumbai Mirror, Mumbai, 9 November 2007.