Inside modern aircraft cockpits, computers project navigational information onto a translucent screen that sits in front of the windshield. This digital “heads-up” display is meant to help pilots fly without having to look down at the instrument panel. But it limits the pilot’s movements and his field of view because he must gaze directly through the screen to look at the readouts.
Scientists are developing technologies to give aviators better in-flight information on wearable displays that untether them from those stationary screens. Just as forthcoming F-35 fighter pilots will see avionics information projected onto their helmet displays, commercial airline pilots in the future may simply look through their sunglasses to see traffic and weather information superimposed over lenses.
Head-worn displays are part of the ongoing effort to modernize the air transportation system, which currently is based on radar and radio technologies. The next generation system will switch to satellite and computer-based technologies to help aircraft fly to their destinations via safer and more direct routes. Not only will that save time and money, but transportation officials also hope that the automation will ease airport congestion by guiding pilots to their runways and gates with little to no human interference.
To help attain that vision, some companies are developing see-through monocle displays that swing into position in front of the eye. Such systems aim to liberate pilots from heads-up displays by giving them the freedom to move their heads without losing sight of the digital data overlays.
Tracking technologies are needed to help synchronize the pilot’s movements with the information that is projected onto his head-worn display.
“You don’t want symbology to be swimming or lagging behind as the pilot moves his head left or right,” says Mike Donfrancesco, vice president of sales at InterSense Inc. The Massachusetts-based firm is one of several companies that produce trackers for use in the defense, medical and entertainment industries. Until recently, few had tackled the challenge of adapting the technology for commercial aviation use.
Traditional tracking systems involve a series of cameras and sensors placed around the object being monitored. But because that “outside-in” model translates into expensive hardware and space-consuming infrastructure, InterSense has pursued an “inside-out” approach that places the tracker on the object itself. It has produced a system that is used by military helicopter and jet fighter pilots for training in flight simulators.
The core technology is based on a combination of cameras, gyroscopes and accelerometers that are worn to gauge head movements inside the cockpit. But the current technology is too cumbersome for commercial aviation use, says Donfrancesco.
The company in January received a small business innovation research contract from NASA to continue developing a miniature tracking technology that will fit onto a pair of sunglasses. Aided by the small cameras and sensors created for use in cell phones and other portable devices, engineers have been downsizing the system.
Their goal is to place a small inertial sensor on one side of a pair of glasses and a battery on the other, and to embed cameras along the rim of the eyeglass frame so that they have a clear line of sight to the instrument panel. This hybrid inertial-optical system will use reference points within the cockpit to track where the pilot is looking.
“The challenging part is not so much the accuracy, but maintaining that accuracy,” says Donfrancesco. Because the sensors are so small, the precision of the tracker tends to drift over time. To correct for the deviation, engineers are relying on the optical sensors embedded on the sunglasses to help triangulate the location of the tracker. Specially designed algorithms will help translate that location and correlate it to data for a specific vantage point.
“We’re trying to have the tracker on the pilot with a very low amount of infrastructure or reference points needed in the cockpit,” says Donfrancesco. The prototype initially may require the installation of light emitting diodes in the cockpit as reference points for the cameras to maintain tracking accuracy. But the goal is to use the cockpit’s natural features, such as instrument displays, as reference points.
NASA has been evaluating the prototype in flight simulators. It plans to test the technology in a series of live flight tests at the end of the year.
“We’re still a few years away from seeing this in an actual plane,” says Donfrancesco. “When the newer display technology comes out, we’ll hopefully produce some more prototypes and get more commercial partners interested in this.”