The National Defense Industrial Association in March hosted the 27th National Test and Evaluation conference in Tampa, Fla. The theme of this year’s event, “Test & Evaluation: Serving the War Fighter,” was particularly appropriate given budget belt tightening and operational commitments around the world.
Keynote speakers, beginning with Michael Gilmore, director of the Defense Department’s office of operational test and evaluation, provided participants with a frank and open assessment of the acquisition environment within the department and laid out both the challenges to, and the imperatives for, test and evaluation.
Among the conclusions was that testers continue to play a critical role as an objective check on the effectiveness, suitability, survivability and safety of systems before they are fielded.
However, in the current political and economic environment, there are some serious challenges.
Gilmore said: “It is our job to ensure leadership knows what they are getting and more importantly, what they are not getting so that they know what they can depend on with regards to these systems and their ability to help them fight and win.”
Some key leaders within the Defense Department believe that testing is too expensive, takes too long and slows down the fielding of urgently needed capabilities. In fact, experience has shown testing actually saves money over the lifecycle of a program. Nonetheless, pressure is rising to control costs and eliminate unnecessary or duplicative structure and processes.
At the same time, the services are understandably eager to speed the delivery of improved capabilities to deployed forces around the world. These challenges are not going away and have powerful implications for test and evaluation. We must continue to push for robust, vigorous and efficient testing at a program’s outset, and continuing throughout its development as a means to reduce acquisition risk and overall costs of new systems. The customers are the men and women in combat and they depend on testers to serve as a consumer advocate. However, the community must also find ways to eliminate unnecessary testing and test infrastructure and make the processes more efficient.
Finally, the community must find effective ways to accelerate testing of urgently needed rapid acquisition items and identify and articulate T&E’s return on investment to the program executive officer and program manager community and senior leadership within the military.
Here are some of the recommendations that came out of the conference:
- Continue to pursue greater efficiencies through integration of traditional test activities into one comprehensive test plan using design of experiments as a scientific method to sequence and build on the results of previous testing.
- Testers should engage early in order to ensure the development of testable, mission-oriented requirements.
- Develop and apply accelerated testing approaches to ensure that critically needed capabilities acquired through the rapid acquisition process are not delayed because of traditional test practices.
- Grow reliability at the front end of system development in order to reduce the likelihood of discovering problems late in the acquisition and testing process and produce savings over the lifecycle of the system.
Most of the conference presentations proposed potential solutions that could be adopted by test agencies and policymakers today. Many of these approaches mirror or directly support the initiatives described above.
Meanwhile, the lack of technical experience and skills in the T&E work force is also seen as a significant roadblock to successful development of new weapon systems. Solutions include better academic training, more effective practitioner training and technical certifications. The Defense Department has formalized certification training for acquisition work force members such as contracting, program management, systems engineering, test and evaluation, logistics and information technology.
The conference generated valuable discussions on the challenges facing defense acquisition and supporting T&E along with significant insights and recommended solutions. The NDIA Test & Evaluation division executive committee urges the community to recognize and accept the challenge to improve by making testing more efficient and effective while pressing to ensure early, rigorous, and continuous evaluations as a means to reduce acquisition risk, system performance and overall lifecycle costs of new and upgraded systems.
The full conference agenda is available along with copies of the tutorials and presentations on the NDIA website or directly
here.Jim O’Bryon is chair of NDIA’s Test and Evaluation division.