A lot of virtual training happens in video game-like environments, where soldiers see combat through the eyes of a superhero character.
But if the Army is going to train its troops through gaming, officials say the characters in the virtual world should perform more like actual soldiers.
That is one part of the reasoning behind a new idea the Army has to create avatars for every soldier. These digital representations would accompany service members throughout their training and allow them to see, through simulation, how their skills, or lack thereof, would play in life and death situations.
The influence of video games on military training has been substantial, and the military’s interest in avatars — for soldiers and other actors in simulations — is growing. It was evident in the many products on display at the world’s largest military training and simulation conference in Orlando — in the graphics, the props and the apparent ease with which younger soldiers adapt to a virtual setting. And at the entrance to the showroom floor, greeting attendees to the Interservice/Industry Training, Simulation and Education Conference, was an avatar.
A pixilated character named “Information Jason” bantered with service members and industry executives. It engaged in small talk and told them jokes. The avatar was performing the motions and speaking the words of a man behind a curtain several yards away. It was the creation of Organic Motion, a company that also supplied technology for a Lockheed Martin Corp. system demonstrated at the conference.
The Avatar Target Insertion System had onlookers gathering around to watch a service member talking to a suspicious computerized character in a simulated Afghanistan village. The avatar on the screen was able to hold a conversation in real-time. It responded to specific questions and commands. It was being controlled by an actor in New York City.
“Some things we have the capability to do very easily,” said Chester Kennedy, vice president of engineering, global training and logistics at Lockheed. “Some things we’re not quite there with. The step in between is what I’m calling a manned avatar. There is a person driving the characteristics of that avatar, they just don’t have to be in the same physical space.”
Avatars could be controlled by people in theater to imbue training with the most up-to-date information and scenarios on the ground. War is not static, Kennedy said. Threats are constantly changing. A trainee does not need to physically be in Afghanistan to benefit from a role-playing experience with someone who is, he said.
“With avatar technology, you can take somebody today who experienced a new threat and have him role-play for those going into theater in real time,” Kennedy said. “At some point in the future, we should be able to model those human behaviors and really create what the Army is trying to do. It is helping us to push that technology along. If you look at the training continuum, how many things can be satisfied by an [artificially intelligent] avatar today as opposed to two years ago or a year ago? We’re continually dramatically improving.”
The military eventually could have individualized systems available 24 hours a day, seven days a week. Service members would be able to log on and interact with avatars in subjects specific to their training whenever they have time, Kennedy said.
Soldiers, themselves, would have avatar doubles under the Army’s nascent plan for virtual training.
“You design an avatar that has the individual facial features of a soldier,” said James Blake, the Army’s program executive officer for simulation, training and instrumentation. “Then you add more of what he looks like, physical attributes. If he’s a tall person, the avatar would be tall. If he’s a short person, the avatar would be short. When you’re in your game environment you’d like to have the physical and mental attributes of that individual reflected in that virtual world.”
The idea is to encode the soldier’s DNA, so to speak, within a digital representation. This means the computer character would run as fast or jump as high as a soldier did during a physical training test. The avatar’s marksmanship also would be tied to how effective a soldier has been in weapon drills.
“It’s very early in development. There is a lot of homework to be done here,” Blake said. “But you might think of it as actually being on a smartcard one day. You go to your post, you plug it in and we know everything about the soldier. And when a soldier behaves in a virtual environment with his squad members that representation is in there. So if the soldier is not a good marksman it will reflect in his contribution to the performance of that squad.”
The Army needs to determine how difficult it will be to accomplish this for every soldier. It may be that only three or four attributes really make a difference in virtual training, and avatars do not need to be so precise in their relationship to their human counterparts. Whatever the case, the Army wants to avoid the common pitfall of avatars in video games — the urge of the participant to create a superhuman character that in no way represents himself.
“In most games, if you want to have a character, you want to be a superhero,” Blake said. “Most people don’t come into the [gaming] environment as themselves. They come into it to become someone else, because the goal is to become immortal, stronger, faster. What the Army is suggesting is maybe we need to develop a character that is representative of the individual so when we put that soldier’s character in simulation it performs the same way the individual would perform.”
Army training officials began mentioning their avatar concept at conferences this past year. It has been attracting interest and gaining support from industry, which along with the Army, has been putting more emphasis on developing training systems for individual soldiers.
“Where things have really failed have been with individual soldiers,” said Andrew Tschesnok, CEO of Organic Motion.
A flight simulator offers the pilot replicas of the tools he would use to fly the actual aircraft. A vehicle simulator does the same. “But with individuals, there are no buttons to push,” Tschesnok said.
So far, the solution has been to create tools for ground troops. There is a mish-mash of “unnatural” gadgets available, such as fake guns with joysticks on them, that in no way resemble what troops would use or encounter in real life, he said. The goal of Organic Motion’s technology is to allow a simulation to see the ground soldier the same way a flight system sees an airplane.
The most common way to digitize a person’s specific movements is to dress him in a black suit that is covered with white sensor balls. This method has been used for years to achieve accurate portrayals of professional athletes in video games. But Organic Motion has found a way to replicate the movements of humans without attaching any devices to the body. This computer vision system consists of a suite of cameras that can see exactly how a body moves.
“They can just jump into the simulator, jump back out and go into after action review,” Tschesnok said.
This allows a single actor to play multiple roles in a fluid simulation, which offers one way to shave costs. Some systems cost millions of dollars just to keep a dozen or so actors on stand-by for a year. Organic Motion has made it so a person in Afghanistan, for example, could play multiple characters in different rooms during a simulation, Tschesnok said.
The Army will have to be careful with its idea to create training avatars for soldiers, he said. If the service were just to create a “second life” digital representation that troops still predominately operated through the use of keyboards and game controllers, “how is that teaching anybody about the real world?” he said.
Officials say that the Army’s avatar concept would enhance the training experience for soldiers by gearing it toward individual needs. This will require better collection of data, said Keith Catanzano, vice president at Booz Allen Hamilton.
“We’re starting to see performance records be captured in more detail,” he said. “And if there’s an avatar on the front of a database with the name and ID on the front, that would kind of be the next step. But first you have to make sure you have all the data that you’re going to need in a much richer way than we have collected now. I think that that’s feasible. I think that’s going to go in incremental steps — launch and learn, launch again and learn. But it really comes back down to the data. You have all the data collected, now let’s use it.”
Catanzano sees two different sets of data — the circumstances surrounding a soldier’s learning such as location and instructor, and specific information related to actual performance of a military operation. Collecting details inside each of these categories will allow the Army to constantly update soldier avatars.
The concept is catching on throughout the service, including with aviators and operators of unmanned aircraft systems.
It would be a big leap forward if “every soldier came in … and the bottom of their ground control station on their screen had an avatar and that avatar was them,” said Col. Robert Sova, who recently announced his retirement from his position as the Army Training and Doctrine Command capabilities manager for UAS.
Each time a soldier develops a skill, the avatar could receive some sort of badge and become more knowledgeable too, he suggested.
“[In gaming] they want to make that avatar better than they are, so they keep doing things in training and make them better,” Sova said. “So every soldier makes that avatar better than them and therefore makes the organization that they are a part of better and makes our fighting force better.”