Question

Armoring Military Planes During the Second World War, the U.S. military collected data on bullet holes found in B-24 bombers that returned from flight missions. The data showed that most bullet holes were found in the wings and tail of the aircraft. Therefore, the military reasoned that more armor should be added to these regions, as they are more likely to be shot. Abraham Wold, a famous statistician of the era, is reported to have argued against this reasoning. In fact, he argued that based on these data more armor should be added to the center of the plane, and NOT the wings and tail. What was Wald's argument?

Short answer

Abraham Wald argued that the hole-free areas on returning planes like the center of the plane indicated the parts that were vital to the plane's operation and its ability to return from missions. When these areas were hit, the plane was likely to be lost. This is why he suggested armoring these parts more - not the wings and tail. This reason is due to a cognitive bias known as survivorship bias.

Step-by-step solution

Understanding Survivorship Bias

This bias occurs when we focus only on the subjects that 'survive' a certain process and overlook those that didn't because they lack visibility. In this exercise, the airplanes that returned from the missions are the 'survivors' and those that didn't make it back are not considered kind of disappear from the observations.

Applying the Concept to the Exercise

When analyzing only the planes that returned, the military noticed that the bullet holes were mostly on the wings and tail. It would seem logical to conclude that those areas are targeted more and should be better armored. However, they were ignoring the planes that were shot and didn't return.

Identifying the Correct Areas to Armor

The fact that a plane can sustain damage to its wings or tail and return means those areas do not need additional armor. Conversely, the areas without bullet holes, such as the center of the plane, indicates that these places are likely to be fatal when hit. This is why these areas need more armor.

Key concepts

Statistics

Statistics play a critical role in making informed decisions by helping identify patterns and infer conclusions from data. In the scenario of armoring military planes during World War II, statistics provide insights beyond the obvious data points—bullet hole locations on returning B-24 bombers. Analyzing where the bullet holes are not found can be as important as looking where they are. This inverse analysis leads to evidence-based decisions, such as reinforcing less-hit areas that may actually be more critical to the plane's survivability if struck.

Without statistical reasoning, one might easily assume the most frequently hit areas are the most vulnerable and thereby make a potentially fatal error in judgment. Forensic statistics is about piecing together the whole picture, considering both the data you have and the data you lack, due to non-survivors, for instance. In decision-making, especially for war efforts, statistics can mean the difference between life and death and can guide strategy on both small scales, like armor placement, and large scales, such as troop deployment and resource allocation.

Biased Analysis

Biased analysis occurs when the data reviewed is pre-filtered by an event, in this case, the event being the bombers' return. Survivorship bias is a classic example of where decisions are made based on an incomplete dataset—only considering the cases that 'survived' a certain condition. In the historical context of armoring the bombers, focusing on the bullet holes in the returning planes led to the potentially erroneous conclusion that the most hit areas were also the most in need of reinforcement.

Abraham Wald's argument illustrates the importance of recognizing and correcting for biased analyses. The absence of bullet holes in the center fuselage in the returning planes might imply that hits to this area were more likely to result in losses, and therefore, weren’t represented in the analyzed data. When conducting any analysis, it's crucial to question if the data is representative of the whole picture or if it is skewed by the perspective of 'survivors.' This keen awareness is necessary beyond wartime decision-making; in fields such as finance, healthcare, and technology, avoiding biased analysis can lead to better informed and fairer outcomes.

Decision Making in War

During times of war, decision-making is pressured by the need for efficiency and the high stakes involved. Margins for error are significantly reduced due to the tangible consequences—often a matter of life or death. The situation faced by the U.S. military in World War II demonstrates a specific challenge in wartime decision-making: how best to protect aircraft based on damage assessments from returning bombers.

The military's initial inkling to reinforce the wings and tail almost disregarded the essence of survivability and vulnerability. Consideration of survivorship bias, brought forth by a statistician like Abraham Wald, reveals a layer of strategic thinking vital in conflict scenarios—understanding that what is not seen (the planes that did not return) can inform decisions as much as what is seen (the bullet-riddled planes that did return). Leaders in such circumstances must weigh data against the backdrop of what information may be missing and anticipate unseen risks. Ultimately, decisions made in wartime can be a testament to the power of sound statistical analysis and the peril of biased interpretations.