Here’s the latest chapter of the saga: F-35 pilot counters Director Operational Test & Evaluation (DOT&E) leaked memo.
Three weeks ago, a memo dated Aug. 9 (one week after the Air Force declared the IOC – Initial Operational Capability – of the F-35A) by Michael Gilmore, the Defense Department’s director of operational testing, obtained by Bloomberg News, highlighted several deficiencies.
“The program is actually not on a path toward success but instead on a path toward failing to deliver the full Block 3F capabilities for which the Department is paying almost $400 billion by the scheduled end of System Development and Demonstration (SDD) in 2018.”
According to chief of the Pentagon’s top testing office, at least 15 capabilities in the F-35’s current software version, known as Block 3i, are either still in need of a fix or aren’t ready for testing.
“Unresolved Block 3i deficiencies in fusion, electronic warfare, and weapons employment continue to result in ambiguous threat displays, limited ability to effectively respond to threats, and, in some cases, a requirement for off-board sources to provide accurate coordinates for precision attack. Although the program recently addressed some of the Block 3i deficiencies, many significant deficiencies remain and more are being identified by operational test and fielded units, many of which must be corrected if the program is going to provide the expected “full warfighting capability” described in the Operational
Requirements Document (ORD).”
The memo provides details about all the hundred deficiencies in Block 3i.
“Because Block 3i is an interim capability based on Block 2B, it has numerous inherent limitations that will reduce operational effectiveness and require workarounds if the F-35A in the Block 3i configuration is used in combat.”
There are limitations in the capability to perform Close Air Support missions (in a permissive or low-threat environment); limited weapon load; no gun capability; limited night vision capability; greater reliance on tankers due to limited on-station time; unacceptable sensor fusion; etc. You can read them all here.
A subsequent POGO article provided an in-depth analysis of the above mentioned memo with the following conclusion: “This DOT&E memo clearly exposes the Air Force’s F-35 IOC announcement as nothing more than a publicity stunt.”
On Sept. 16, a new story written by Major Morten “Dolby” Hanche, the famous Royal Norwegian Air Force F-35 pilot who provided first-hand accounts of what dogfighting in the controversial F-35 looks like to a pilot with a significant experience with the F-16, has been published by Kampflybloggen (The Combat Aircraft Blog), the official blog of the Norwegian F-35 Program Office within the Norwegian Ministry of Defence.
In the new post (reposted below under permission) Maj. Hanche, a U.S. Navy Test Pilot School graduate with more than 2,200 hours in the F-16, currently flying as Assistant Weapons Officer with the U.S. Air Force’s 62nd Fighter Squadron at Luke Air Force Base in Arizona, provides his take on the DOT&E memo.
Once again: “Dolby” is an F-35 instructor pilot from the Royal Norwegian Air Force, one of the Joint Strike Fighter customers. Needless to say, he may have a bias for his plane. Still, he’s a respected test pilot, making public claims and providing tons of interesting details about the aircraft that will help you making your own opinion on such a hotly debated topic.
Lack of perfection does not mean disaster – how I read test reports as a pilot
by Morten Hanche
Yet again, information from the «Director Operational Test & Evaluation» (DOT&E) has stirred critics into a frenzy over the F-35. The fact that the information was leaked seems to have agitated people even more. (We have our hands on classified documents! Now we know it all!) Yet again, the leaked memo described aspects of the F-35 which need improvement. Yet again, the report resulted in press articles which painted a pretty sinister picture of the F-35. The article featured in POGO («F-35 May Never Be Ready for Combat») serves as one such example.
I finished up writing this article before getting ready to fly another sortie in the F-35. Based on my own experiences flying the F-35A, I feel that the media´s interpretation of the previous DOT&E report is influenced heavily by unrealistic expectations – something which seems to be a trend. I don´t see the point in countering every claim that´s being brought up. First off, it´d make for a very long article. Secondly, I would not be dealing with the bigger problem, which in my mind is a lack of understanding.
I fully expect the F-35’s most hardened critics to discount this article, regardless of what I write. However, some may choose to believe my story, based on the fact that I know the airplane and its capabilities as a pilot. I don’t make my claims based on bits and pieces of information, derived from potentially unreliable sources. They are based on experience actually flying and training with the jet for nearly a year
My goal is to shed some light on airplane development and testing; why we test, what we discover in testing and what a test report may result in. I write this based on my own experience, both through education at the US Naval Test Pilot School, but more importantly through working with the F-16 and the F-35, both operationally and in test settings.
What smartphones tell us about technology development
I´ll start with smartphones, as another example of technology development. Admittedly, phones are somewhat different from a fighter airplane, but there are similarities. A smartphone is a complex system of systems – just like a fighter jet. The phones keep evolving with both new hard- and software. It is not unheard of therefore that the manufacturers issue updates. Updates which provide new capabilities, but which also aim to correct previous errors.
According to Wikipedia, Apple released its iOS 9.0 operating system to their iPhones and iPads on 16 September 2015. The 9.0.1 update was issued already on 23 September, followed closely by the 9.0.2 update on 30 September. Then 9.1 on 21 October and 9.2 on 8 December 2015.
Such a frequent update rate might indicate that not everything worked perfectly from the start. Still, wouldn´t it be a bit harsh to claim that the phones didn´t work with the first four software versions? Might the truth be a little more nuanced? Can a smartphone be a good product, even if it doesn´t work 100% from day one? Does a smartphone ever work 100%? I have experienced various strange occurences with my phones over the years. Still, for me, having a phone with all its peculiarities has been more useful than the alternative – not having a phone.
This isn’t an article about phones. The point I´m trying to make is that technology development and testing is a series of compromises; compromises in reliability, in performance and in quality. Only rarely is the world black or white. A machine may work well, even if it doesn´t fulfill all specifications. I´ll go on with a brief intro to how we typically test.
…technology development and testing is a series of compromises; compromise in reliability, in performance and in quality. Only rarely is the world black or white. A machine may work well, even if it doesn´t fulfill all specifications.
How we test a fighter jet
Testing of combat aircraft typically sees a disctinction between Developmental Test (DT) and Operational Test (OT). In short we can say that DT seeks to answer whether the machine works according to the design specifications, whether the machine is safe to operate and what its safe operating limits end up being. OT on the other hand seeks to find out whether the machine can solve a particular task, like: «Is the X-YZ able to provide effective Close Air Support, in the presence of threat A, B and C?»
The test program for a machine like the F-35 is an enormous undertaking. The contours of the F-35´s test program are described top-level in the Test and Evaluation Master Plan (TEMP), totaling 1400 pages. Each sub-test in the TEMP results in a detailed test plan for that event. Especially in DT, a test flight is literally planned down to the minute, in order to accomplish as many test points as quickly and safely as possible. Flight testing is an expensive undertaking.
A test program should discover most important errors and flaws. However, time and resources available make it unrealistic to uncover every single issue. Risk is mitigated by testing the most critical components, like the engine in a single-engined fighter, to stricter tolerances. The amount of testing is a statistically driven decision. We know that there are things we don´t know, even at the completion of testing. We also know that there are likely few gross or dangerous errors which haven´t been found.
Each error we find during testing is documented and characterized. The language and format used is to the point. The test engineer and test pilot type up their findings and typically describe the situation «in a vacuum» – without regard for how costly or difficult it might be to address the issue. Each issue is then related to the mission – how will this quality or problem affect the given task?
Such a test report might read something like: «The SuperToaster 3000 was evaluated for uniform heat distribution and time to crispy toast, at the National Toast Center of Excellence, with room temperatures varying between 65 and 75 deg F. The toasting temperature was selected by turning a dial on the front of the toaster. Even with full crispyness selected, the toaster´s maximum temperature was low, and toasting of even the thinnest slices of white bread took more than 10 minutes. During early morning breakfasts, the time consuming toasting process will result in cranky parents, the kids being dropped off late for school and correspondingly negative effects on their grades and later career opportunities.»
This mission relation was probably a little over-the-top – a little like how some media articles relate its tidbits of information to an imagined F-35 mission. In isolation, a system may not work as advertised, but could there be a workaround? (In the toaster-case, maybe cereal for breakfast?)
Anyway, after the issue is documented, the errors are then catalogued, debated over and prioritized. Test engineers, test pilots, design engineers and customer representatives are often involved in the dialogue that follows when something undesirable is discovered. Together, these will have to agree on a path forward. Completely understanding the issue is crucial. Alternatives could be a re-design, accepting the flaw, mitigating the flaw procedurally or compensating by documenting the issue better. The team will have to compromise when prioritizing. Even when developing a new fighter jet, there are limits to what can be fixed, based on cost, time available, test resources available and also the complexity of the problem. Altogether, development and testing is an iterative process, where adjustments may have to take place during DT, OT or after the system is put into operational service.
Where are we with the F-35?
What is then the current state of the F-35? Is it really as bad as the commentaries to the DOT&E report and DOT&E memo might indicate?
Personally, I am impressed by the the F-35. I was relieved to experience just how well the F-35 performs with regard to speed, ceiling, range and maneuverability. It would have been very problematic if the airplane´s performance didn´t hold up in these areas – there´s just no software update which is going to compensate a draggy airframe or a weak engine. (Read more about such a case in the Government Accountability Office, then the General Accounting Office´s report on the Super Hornet).
When asked about my first flight in the F-35, I compared it to flying a Hornet (F/A-18), but with a turbo charged engine. I now can quote a USMC F/A-18 Weapons School Graduate after his first flight in the F-35: «It was like flying a Hornet with four engines!» (His point being that the F-35 can afford to operate at high Angle-of-Attack and low airspeed, but that it will regain the airspeed quickly when needed). Another unintended, but illustrating example on performance came a few weeks back, when a student pilot failed to recognize that he had climbed through our temporary altitude restriction at 40,000´. The F-35 will happily climb past that altitude.
Another critical aspect of the F-35 is its minimal radar signature. Just as with the aerodynamic performance, the «stealthiness» of the F-35 is an inherent quality of the airframe itself. There would be no quick-fix to a disappointing signature. So far, my impression is that the F-35 is very difficult to find. We see this every day when training with the F-35; we detect the F-16s flying in the local airspace at vast ranges, compared to when we detect another F-35.
Sensor stability, and specifically radar stability, has been an issue. I´m not trying to downplay that the radar´s stability needs to improve, but I am not worried. What would have worried me was if the radar had poor detection range, or if the stability issues were caused by «external» factors like limited electrical power supply or limited cooling available. Fortunately, our biggest issues are related to software, and not performance. I think it´s realistic to expect software issues like this to be resolved (just like iOS 9 eventually ended up working well).
Remember that we´re not trying to re-create another «Fourth Gen» fighter in the F-35. If we had set our aim lower, we´d likely have had an easier job of developing the airplane – it would have been easier to build the F-16 again today. But is that what we need? The F-35´s specifications are ambitious, and reflect a machine which will outperform the previous generation of fighters. Having or not having that kind of military advantage eventually becomes a political question. For now, our leaders think we need that military edge.
In this context, I would like to bring up another point. The F-35 is in its infancy as a weapons system. Yet, it is being compared to mature systems like the F-16. The F-16 has been developed and improved for more than 40 years. Correspondingly, certain aspects of the F-16 are more mature than the F-35 at this time. Having said that, I will caution readers against believing that other and «mature» fighters are without their issues. There has been an unprecedented openness about the F-35´s development. The DOT&E report is one example on how media has gained insight into the F-35 Program. I still ask; do those who write critical articles about the program really have a realistic baseline, from which they can reasonably assess the F-35? Next, I´ll give some examples which have influenced at least my own baseline.
The sometimes messy world of fighter development
Credit: RoNAF
Many will agree that the F-16 has been a successful fighter design. The fact that it has been continuously produced since the 1970s should speak for itself. The fighter has come a long way from where it originally started; as a day-only «dogfighter», equipped with heat-seeking missiles. (How would that mission set compare to a post System Development & Demonstration Block 3F F-35 and its mission sets?) Modifications to the «fully developed» F-16 started right away One early and visible modification was the replacement of the horizontal stabilizers with larger «stabs», in order to reduce the F-16´s susceptibility to go out of control during aggressive maneuvering at high Angles-of-Attack (AOA). Going out of control is a bad thing, and could lead to loss of both the jet and its pilot. Since then, the F-16 has kept evolving through many different programs, aimed at improving both structural life and combat capabilities.Other fighters also bear visible marks of error correction. The Hornet-family provides some good examples of aerodynamic «band aids». An example from the F/A-18 «Baby Hornet» is the vertical «fences» mounted on each side of the machine, just aft of the cockpit. These were eventually added to mitigate stress on the vertical tails, which caused their supporting structure to fail.
Credit: U.S. Navy
Another example from the Baby Hornet is how the stabs and rudders are driven to full deflection before takeoff. This modification was necessary to enable the Hornet to lift its nose during takeoff roll. The «band aid» added drag during the takeoff roll. Thus, the takeoff roll increased in distance, but no more than what was considered acceptable. The «band aid» was an easy workaround to what could have been a very costly re-design of the airplane – compromises…
The more modern Super Hornet has a porous fairing where the wing-fold mechanism is located. This was fitted in an attempt to alleviate a problem termed «wing drop». The wing drop in the Super Hornet was described as an abrupt and uncommanded roll, which hampered air combat maneuvering. The «band aid» partially fixed the wing drop issue, but at the same time introduced other problems related to reduced range and increased buffet levels. These were still deemed acceptable trade-offs – compromises…
Even today, our modern-day F-16s live with many issues; errors which were discovered in DT, OT or operational use, but which haven´t been corrected. Either because of prohibitive cost, complexity or because no one understands the failure mechanism – what is causing the problem. I´m not just talking about cosmetic or minor issues. One example is that The Norwegian Armed Forces for a period of about 10 years could not operate its F-16s in single ship formations, in bad weather or at night. The restriction was put in place because the Main Mission Computer (MMC) broke down relatively often. The resulting operational limitations hampered both training and operations. It took more than 10 years to diagnose and correct the issue, mainly because the failure mechanism was illusive.
The most outspoken critics of the F-35 couldn´t have known about our issues with the MMC in the F-16 at the time. If they did, and read that deficiency report, would they have concluded that our F-16s were non-operational, and incapable of fulfilling its mission? I´m tempted to think so, based on how isolated pieces of information about the F-35 often are misinterpreted and taken out of context. Would they have been right in their conclusion? I don´t think anyone could have made that conclusion, based on just the fact that «the MMC sometimes crashes». The reality I know, working with fighters all my life, is not black or white. There are nuances. We work around and overcome problems.
Our F-16s still have issues today which will never be corrected. This is not dramatic or unexpected. The normal state of affairs for a fighter is that we operate in spite of issues with structure, sensors, software and logistics. We´re normally able to work around the major problems while we devise long-term solutions. Some issues are temporary. Some end up being permanent. Compromises… (I personally wouldn´t believe the salesperson claiming to offer a fighter jet which had zero issues).
I said I wouldn´t quibble over individual factual errors which the F-35´s critics present as truth. To me, a compelling argument for how well the F-35 works is evident by what we´re able to do in training. Three weeks back I was part of a four-ship of F-35s. Our mission was to overcome an advanced airborne threat, while locating and destroying an equally advanced surface based air defense system. After neutralizing these threats, we were able to destroy four additional targets. All this prior to receiving the Block 3F capabilities. Suffice to say that this mission would have been close to suicide with a four-ship of F-16s alone!
