Twitter Facebook Google RSS
Inside Science and Technology 

Body Language Takes on New Meaning in PC World 


By Grace V. Jean 

With the advent of touch-screen technologies, navigating the digital world is no longer limited to typing on a keyboard and clicking on a mouse. Today, we drag our fingers across handheld devices and tabletop displays to interact with electronic data. In the near future, we may command such systems without touching them at all.

The same researchers who developed the touch capability for Microsoft Surface are exploring ways to interface with computers and other digital gadgets using hand gestures.

“We’re tracking specific kinds of motions,” says Hrvoje Benko, a researcher on the Redmond, Wash.-based Microsoft Research team, during a recent road show.

Inside a small makeshift planetarium, he stands behind an omni-directional projector that is producing a 360-degree image of the nighttime sky. He places his hands into the light beam so that they cast a shadow on the dome above — the better to demonstrate his motions in the darkened space.

“We can literally reach into the image and move the sky,” he says as he touches his index fingers to his thumbs in a pinching motion and moves his hands apart to “pull” the edges of the sky outward. The overhead image expands, zooming in on a cluster of distant stars. He reverses the motion, pushing his hands towards each other. This time, the sky contracts and zooms out beyond the Milky Way galaxy.

The secret lies inside the projector where a wide-angle lens focuses the data onto the 360-degree environment. The team has added an infrared camera that shares the projector’s lens to detect a person’s gestures in mid-air. Anywhere the images are projected, the camera can also “see.” Algorithms allow the system to correlate the holes made by a person’s hands to computer functions.

Touching the thumbs and index fingers together to form a large teardrop-shaped hole signals to the computer to listen for a vocal command. “Crab Nebula,” Benko says into a headset microphone. The planetarium fills with the requested telescope imagery. Using the same method, he switches to a replica of his team’s office building, pulls up real-time video footage of the planetarium’s exterior and immerses viewers in a snow globe simulation.

“More and more, we’re seeing the emergence of other kinds of data that might be more suitable for other form factors and displays,” he says.

There could be complex, three-dimensional battlefield awareness data that a ground commander might want to examine. “You don’t have to gesture crudely at a flat display and not understand what we’re talking about in 3-D,“ says Andy Wilson, the senior researcher on the team. “We can actually walk into the dome and I can take you there. By putting yourself into the data, you can get a better sense of what’s happening.”

Researchers say that a corporate conference room could be another outlet for their omni-directional projector. “You can think about having something like this on the ceiling, and all of a sudden, every surface in the room is potentially [a display],” says Benko. “You can put content anywhere you want.”

But the brightness and resolution of current projector technologies are limiting factors. To convey intentions through gestures, one must stand close to the projector. “What we’d like to do is relax that a little bit and start interacting more in the full volume of the space,” says Wilson. Ultimately, researchers want conference attendees to be able to interact with the visual information no matter where they are sitting in relation to the projector. They could walk up to an image projected on a wall, for example, and manipulate it by touching it and moving it around with their hands. “The spatial dimension comes into play,” says Benko.

The team is not wedded to projectors and it points out that researchers are looking at interactions with displays embedded in other form factors. The growing popularity and affordability of high definition displays means that American households in the future likely will contain an ecosystem of multiple and disparate display technologies. How those might be tied together and how we will interface with them also are being investigated.

“We’re saying there will be a variety of different devices. You’ll still have a keyboard, probably, to deal with Excel spreadsheets,” says Wilson. But gestures or other control mechanisms could allow people to interact more collaboratively with other types of information, such as the latest vacation photos on a giant flatscreen display along a hallway, he adds. “The opportunities are there to make it a highly interactive, shared experience,” says Benko.
Submit Your Reader's Comment Below
The content of this field is kept private and will not be shown publicly.
Please enter the text displayed in the image.
The picture contains 6 characters.
*Legal Notice

NDIA is not responsible for screening, policing, editing, or monitoring your or another user's postings and encourages all of its users to use reasonable discretion and caution in evaluating or reviewing any posting. Moreover, and except as provided below with respect to NDIA's right and ability to delete or remove a posting (or any part thereof), NDIA does not endorse, oppose, or edit any opinion or information provided by you or another user and does not make any representation with respect to, nor does it endorse the accuracy, completeness, timeliness, or reliability of any advice, opinion, statement, or other material displayed, uploaded, or distributed by you or any other user. Nevertheless, NDIA reserves the right to delete or take other action with respect to postings (or parts thereof) that NDIA believes in good faith violate this Legal Notice and/or are potentially harmful or unlawful. If you violate this Legal Notice, NDIA may, in its sole discretion, delete the unacceptable content from your posting, remove or delete the posting in its entirety, issue you a warning, and/or terminate your use of the NDIA site. Moreover, it is a policy of NDIA to take appropriate actions under the Digital Millennium Copyright Act and other applicable intellectual property laws. If you become aware of postings that violate these rules regarding acceptable behavior or content, you may contact NDIA at 703.522.1820.

  Bookmark and Share