Air Force Creating Realistic Avatars with 3-D Scanning Technology

To create an object with a 3-D printer, the user first must obtain either computer-aided design data or a 3-D scanner that can capture the object’s appearance and create a digital model.

But for the Air Force, the use of 3-D scanners extends far beyond additive manufacturing. The Air Force Research Laboratory’s 711 Human Performance Wing is employing such technology to render high-fidelity avatars for simulation and video game training, officials told National Defense.

John Camp, leader of the biofidelic avatar modeling team, said there are three parts to creating a new avatar: the skeletal structure that gives the individual its unique proportions, the animation and what developers call the “mesh” — the visual representation of a human in the virtual world.

To create the mesh, the team uses a whole body scanner comprising several cameras arranged in a ring around the subject, said Dustin Bruening, biosignature discovery team lead. Each camera takes a photo, which produces a digital, 3-D cloud of points on the body. Connecting those points with lines results in the creation of a mesh.

Animation and skeletal structure is captured with motion capture technology similar to the process used to create Gollum in the Lord of The Rings film trilogy, he said.

“We put a lot of reflective stickers or dots on a person and have them move around in a motion capture area, and motion capture cameras take the reflections of those dots,” Bruening said. “The end result of that is a stick figure or a skeletal model of a person moving around.”

Then developers fuse the imagery from the 3-D scanner and motion capture to create an avatar, he said.

“So far, what we’ve done is being able to replicate motion, and we’re now looking into how to create new models or new meshes to represent individuals,” Camp said. “Down the road, what we’d like to do is look at multi-modal representations of humans” — meaning imagery of humans as seen through infrared and radar.

Using 3-D scanners imparts avatars with a level of biofidelity that would allow programmers to create scenarios where avatars look and react in a diverse way that a population of humans would, he said. “Commercial-off-the-shelf software has a single skeletal model for every single model in that world. So male, female, child, the skeletal proportions are the same. They just kind of stretch the whole thing or squish the whole thing.”

Isiah Davenport, creative director of multimedia for Infoscitex, an engineering research-and-development company working with the Air Force on the project, said one application would be to use the realistic avatars to train analysts how to spot at a distance if a person is carrying a concealed weapon or an improvised explosive device.

“We are able to actually model that kind of fidelity, not only in our motion capture,” but also in the way that the mesh of the simulated clothing folds and moves along the skeletal structure of an avatar. However, he said, one problem is that most current commercial gaming systems and engines do not support that level of realism.

Topics: Science and Engineering Technology, Simulation Modeling Wargaming and Training, Videogames