Virtual, Augmented Reality Tech Transforming Training

A subject matter expert at the Air Force Research Laboratory flies a virtual F-16 during the Secure Live Virtual Constructive Advanced Training Environment Phase III capstone demonstration at Nellis Air Force Base, Nevada.

Air Force photo by William Graver

Augmented and virtual reality technology as well as artificial intelligence are disrupting military training as the landscape of warfare shifts.

The United States is facing advanced adversaries which requires the Pentagon to better prepare for a high-end fight. To do so, the Defense Department is leveraging game-based training systems, augmented/mixed reality and other cutting-edge tools.

A number of new transformative technologies have been introduced over the last few decades.

Although virtual reality technology has existed for more than 30 years, it is an invaluable tool for bringing better training opportunities to the services, Marlo Brooke, CEO and founder of AVATAR Partners, a software company that provides augmented and mixed reality solutions, said during a panel discussion at the Virtual Interservice/Industry Training, Simulation and Education Conference in December. I/ITSEC is hosted by the National Training and Simulation Association, an affiliate of the National Defense Industrial Association.

Virtual reality, or VR, immerses users in a computer-generated environment.

But the biggest game-changer has been augmented and mixed-reality capabilities, Brooke said. Augmented reality, or AR, transposes data or other digitally created images on top of a real-world field of view. Mixed reality also merges real and virtual worlds to produce new environments where physical and digital entities co-exist.

The ability to affordably produce mixed-reality training systems is a necessity, Brooke noted.

“This all needs to be sustainable and cost effective. And by that, I mean it has to have scalability,” she said. “It needs to scale-up, support multi-platforms [and] be cost effective from the start.”

Daniel Serfaty, CEO and founder of Aptima Inc., a manufacturer of military training technology, said he couldn’t recall a technology that has taken off as quickly as mixed reality systems have.

“We’ve all witnessed over the past three [to] four years … that mixed reality went from a nice thing to put on the show floor for people to play with, to a real medium for training,” he said.

One training system that enables mixed reality and is currently being used by the Pentagon is Raydon Corp.’s Mission Reconfigurable, or MR, system.

The platform is a tool that can support Army ground vehicle or rotary-wing aircraft training.

The portable capability comprises vehicle kits added to a standard set of base seats which consist of an instructor/operation station, two mobile computer racks, and three to four base seats/stands. The kits contain the components necessary to train drivers, commanders, gunners, loaders, squad leaders or any other vehicle-specific position. The capability can replicate 55 different ground vehicles with 16 kits and 13 aircraft with six kits.

“It is an open platform, we’ve designed it where the platform itself is designed to run anybody’s software,” said Don Ariel, who recently served as CEO of Raydon. “What I think this is going to do is it’s going to accelerate [the] great capabilities of great innovators,” and aid the transition of their innovation into the hands of people evaluating it for use in a number of training infrastructures.

Mixed reality is part of the Army Synthetic Training Environment’s Reconfigurable Virtual Collective Trainer, or RVCT, Ariel said.

The Synthetic Training Environment, or STE, is a training and mission rehearsal tool that brings together live, virtual, constructive and gaming environments to improve soldier and unit readiness. The STE falls under Army Futures Command’s soldier lethality modernization portfolio.

Virtual and constructive training aids utilize simulators and computer-generated entities to simulate real-world scenarios in the system. The Army intends to pair them with live training and commercial gaming technologies to create a mixed reality environment that prepares troops for battle in a cost-effective way.

The RVCT includes aviation platforms, ground platforms, dismounted infantry collective maneuver training, gunnery training and mission rehearsal capabilities, according to the Army. Cole Engineering Services Inc., a Florida-based IT company, is also contracted with the Army to work on the system.

Mike Knowles, president at Cubic Mission and Performance Solutions, said live, virtual and constructive training is an important future pathway for boosting the readiness of servicemembers.

In near-peer fights that take place in multiple domains such as air, ground, maritime, cyber and space simultaneously, the Pentagon needs new ways to train troops, he said.

The combination of the three methods give the Defense Department an edge to operate in any domain, he said.

They can provide training at more convenient times and in more locations across the globe, Knowles said.

“We actually can provide complementary training through many different modes all along the way so that it can actually be ready and present at all times, rather than at [prescribed] training times,” he said.

For example, Cubic has developed an immersive game-based learning tool used by the Navy to train sailors on littoral combat ships. The Immersive Virtual Shipboard Environment immerses trainees in a 3D virtual environment to teach them tasks in digitally replicated shipboard settings, according to the company.

“Early on in the game-based training world we came on board with the littoral combat ship operations. And we’ve seen a … report to Congress, two years ago, that indicated some of these new technologies and disruptions had taken from six months down to one-month time, to qualify a maintainer for a ship on the LCS,” he said. “We can see how innovative technology can really make a change in bringing people to bear in capability proficiency and readiness.”

These types of technologies have also resulted in improved retention for sailors, he said.

Additionally, Cubic has partnered with both the Air Force and Navy to conduct large force exercises utilizing a live-virtual-constructive system for air combat training at Nellis Air Force Base, Nevada, he noted.

The Secure LVC Advanced Training Environment, Advanced Technology Demonstration was conducted at Nellis in 2018, with Cubic serving as the industry integrator. The event included Air Force F-15E and Navy F/A-18F aircraft, along with F-16 and F/A-18 simulators. “We were able to do a LVC instrumentation inside of two years. We were able to get to a large force exercise, [technology readiness level] 7,” he said.

“Because technology is moving so fast, we took what would have normally been a five-, six-, seven-year development and did it inside of 24 months.”

Now an air-based LVC capability exists that notional timelines suggested would be developed in the 2027 to 2030 timeframe, he added.
Although mixed reality and LVC solutions have begun to streamline training for warfighters, there are still a number of major hurdles that industry has to overcome, Brooke said.

“One is not knowing where to start. The second is, if you’ve started the challenge of realistically scaling from prototype to production,” Brooke said. “Mixed reality is interesting in that it requires a very interesting blend of both software engineering and mechanical engineering know-how. And in software development, there are seven ways at a minimum to build a single solution.”

Testing mixed reality environments can be an unforgiving process, Brooke said.

A software “could work for a highly controlled prototype, but absolutely not in the real world,” she said. “We need to test on the actual equipment and in the actual real world, harsh conditions where the warfighter is doing their work and where you have network issues, you have issues with light and darkness and all different kinds of challenges.”

This issue has made AVATAR Partners an “anti-prototype company” in some ways, Brooke said.

“Generally speaking, prototypes are so theoretical that they don’t work in the real world,” she said. “You can take the same resources and take a real-world training need — one that’s highly critical, very difficult to perform and frequently perform — and build a production system from the start that gives you that precision training.”

Beyond technical challenges, Knowles sees the Pentagon’s acquisition system as a major obstacle for future training systems.

“Realistically, it’s about finding avenues to continue to get that technology procured and into the hands of the war­fighters as quickly as we can,” he said. “We know that has been a long battle through acquisition processes and all to get there, but I think we’re seeing some positive movements where we can build upon those to continue to move it forward.

“We’re seeing use of OTAs, we’re seeing use of other contract vehicles that allow us to speed things up, allow us to move from prototypes to production, much quicker,” he noted, referring to other transaction authority agreements.

Topics: Training and Simulation