What is the Survivorship Bias?
Survivorship bias is a cognitive shortcut that occurs when a visible successful subgroup is mistaken as an entire group, due to the failure subgroup not being visible. The bias’ name comes from the error an individual makes when a data set only considers the “surviving” observations, without considering those which didn’t survive.
Where this bias occurs
Examples of survivorship bias are noticeable in a wide range of fields, particularly in the business world. Students in business school can recall how unicorn start-ups were commonly applauded within the classroom, serving as an example of what students should strive for — an archetypal symbol of success. Even though Forbes reported that 90% of start-ups fail, entire degrees are dedicated to entrepreneurship, with dozens of students claiming that they will one day found a start-up and become successful.
By looking at successful start-up founders, like Steve Jobs, Bill Gates, and Mark Zuckerberg, an individual could conclude that to reach their level of success, they must simply have an idea, drop out of school, and dedicate their time to their start-up.
In the Scientific American, professor Michael Shermer and Larry Smith from the University of Waterloo describe how advice about commercial successes distorts individual perceptions, as individuals ignore college dropouts or businesses that have failed.
Simply put, what many forget to consider is that these unicorn start-ups are just that: unicorns. Of the thousands of people who attempt to follow the same paths as these business-tycoons, most fail. Still, their stories of failure aren’t shared as avidly as success stories, giving others an inflated idea of our own capabilities and potential achievements. That is not to say that hard work and talent will not achieve success, but rather that as a society, we tend to ignore common failures and hold onto success stories as proof of what is possible. Instead, in this hypothetical, we must also consider that things like luck, timing, connections, and socio-economic background have all added to well-known founders’ achievements.
Individual effects
The Survivorship Bias is harmful due to how commonly it occurs, and how profoundly the bias can influence our choices. The Survivorship Bias has been commonly referenced with financial decision-making, entrepreneurship, gambling, and medical research. When making decisions in these sectors while missing information that did not “survive,” survivorship bias can profoundly impact our perceptions and judgments. Without having all the data needed to make rational decisions, individuals will not be able to make the best possible decisions for themselves.
Systemic effects
Survivorship bias is everywhere we look, as it is a common bias that affects how we interpret data and information when making decisions. Survivorship bias also affects high-level decision making, which then results in systemic challenges across multiple disciplines.
Epidemiology
The Survivorship Bias has been found in instances of disease diagnoses, specifically concerning survival rates post-diagnosis. For example, patients with the best prognosis are similarly those with the lowest risk due to their age, previous health history, and fitness level. The more patients with these positive precursors, the better their survival rates for these healthier individuals. Because patients with a worse health history, in turn, do not survive, they are not included in survival rate calculations; thus, patients are disproportionately represented by healthier individuals. What should also be taken into account is individuals who die shortly after being diagnosed or those who die prior to being diagnosed. By not being included in survival rate calculations, survival outcome is inflated.
Even globally, as we attempt to portray the effects of COVID-19 accurately, many epidemiologists and doctors warn that survival rate calculations do not provide a full picture. Patients who die without being tested for COVID cannot be considered part of the virus’s death count, potentially skewing survival rates. In many countries globally, nations and their health care systems cannot keep up with testing, resulting in potential survivorship bias when looking at data generated from the disease.
Why it happens
The Survivorship Bias is a prevalent cognitive bias, which can be attributed to a fundamental misunderstanding of cause and effect, specifically concerning the concept of correlations versus causation. Though correlation and causation can both exist, correlation does not imply causation.
With the Survivorship Bias, simply because individuals observe a pattern from a dataset, such as the previously mentioned example of successful entrepreneurs and dropping out of school, does not mean that all successful entrepreneurs drop out of school, or that all those who drop out of school will be successful. Causation refers to cases where action A causes B’s outcome, whereas correlation is simply a relationship. The coincidence that many entrepreneurs dropped out of university is a correlation, as the event of dropping out of school, did not necessarily cause their success. The Survivorship Bias, however, causes individuals to believe that the correlation is causation. To summarize, the Survivorship Bias leads to a misunderstanding of cause and effect.
Why it is important
Being aware of survivorship bias and understanding how it can impact your judgment and decision-making is critical to ensure an individual is practicing critical thinking and making the best possible decisions for themselves. Survivorship bias can impact individuals across domains; thus, awareness of survivorship bias can ensure better product decisions, team-decision making, or scientific conclusions. Developing biases is an unavoidable human trait, but taking our time to change them is necessary to ensure that we make the best decision we possibly can.
How to avoid it
Once they are aware of survivorship bias, individuals can practice avoiding it in several ways.
Ask yourself what you don’t see
When making a decision, begin by considering what’s missing. Consider what data didn’t “survive,” from an event, or dataset you are using. By asking questions about what is missing, and taking the time to research these missing data points, you can develop a better understanding before your decision-making moment. Being fully informed, and taking the time to pause, reflect, and research will help ensure the consideration of survivorship bias in your decision-making.
Vet your data sources
Another method to prevent survivorship bias, specifically in your work and research, is to be selective of the data sources used. By ensuring data sources are crafted to ensure accuracy and do not omit critical observations that would change analysis results or decision-making, individuals can reduce the risk of survivorship bias.
How it all started
The term survivorship bias was first coined by Abraham Wald, a famous statistician known for studying World War II aircrafts. When Wald’s research group attempted to determine how war-airplanes could be better protected, the group’s initial approach was to assess which parts of aircrafts had incurred the most damage. Once identifying areas that were in the worst condition, they would then reinforce the aircrafts with more protection in those locations. However, Abraham Wald noted that the aircrafts that were most heavily damaged were the ones that had not returned from battle. Those same airplanes would also provide the most relevant information regarding which parts of the aircraft would need to be reinforced.
Had this research group been unable to identify this critical fact, the aircraft reinforcements they would have suggested would have ignored entirely a subset of planes which arguably had the most valuable data points regarding the research project. The research study results provided an example of how Abraham Wald and his research group at Columbia overcame survivorship bias, saving hundreds of lives.
Example 1 – Financial systems
Survivorship bias also impacts our financial systems. A typical example of survivorship bias can be seen in mutual fund performance. Specifically, survivorship bias describes the tendency for companies or mutual funds to be excluded from performance analysis studies, as they no longer exist. The results from these studies assessing financial markets are then skewed in a more positive light, as only companies that were successful and “survived,” were included in the study.
Survivorship bias can be examined more specifically in the case of mutual funds. A mutual fund is a financial vehicle that pools money collected from investors and is managed by a professional money manager, who then invests in things like stocks, bonds, and other assets.10 When looking at a mutual funds’ investments, those investments will only include investments that are currently successful. Funds that were previously opened and lost money would be either closed or merged with other funds, to hide past poor performances.
Survivorship bias occurs when analysts calculate performance results of groups of investments, such as mutual funds, using only the surviving data at the end of the period, and exclude those funds or companies that no longer exist at the end of a study. For example, in a financial universe where 1,000 funds exist, imagine that 10% of these funds stop existing by years-end due to poor performance. If an analyst is conducting a performance review of these funds but only begins the study at the end of the year, the analyst would fall privy to survivorship bias and omit the failed funds from their final results. By not including funds that failed due to worse behavior, the performance data would indicate a more favorable final result for the theoretical fund universe.
In 1996, researchers Elton, Gruber, and Blake analyzed the relationship between fund sizes and survivorship bias. They found that survivorship bias was more significant in the small-fund sector than in more significant mutual funds. Smaller funds have a higher probability of folding than larger, more established funds, which is why they attributed this to be true for the small-fund sector. The researchers estimated the size of survivorship bias across the United States mutual fund industry as 0.9% annum. Additionally, they defined and measured survivorship bias as the following:
“Bias is defined as average α for surviving funds minus average α for all funds” (Where α is the risk-adjusted return over the S&P 500. This is the standard measure of mutual fund out-performance).
Example 2 – Medical research
Another example of survivorship bias can be seen in the medical field and medical research. In 2010 at the Harvard Medical School and Beth Israel Deaconess Medical Center (BIDMC), a trial was conducted to improve patient survival following trauma. The main issue, which occurs after trauma, is that a patient then develops abnormal blood clotting, meaning any bleeding the individual has increases their chances of bleeding to death.
The Harvard study investigated whether giving trauma patients factors, which are naturally occurring proteins in our bodies that act to encourage blood clotting, would improve survival rates in these trauma patients. The study targeted patients who had received 4-8 blood transfusions within 12 hours of their initial injury. The trial hoped to recruit 1502 patients, but only recruited 573, and thus was later abandoned.
This study’s failure was due to survivorship bias, as the trial only included patients who had survived their initial injury, and who had then received care in the Energy Department before receiving 4-8 blood transfusions. Patients who died from their initial injury were not included in the study, making it challenging to find suitable patients for the trial.
Summary
What it is
Survivorship bias is a type of sample selection bias that occurs when an individual mistakes a visible successful subgroup as the entire group. In other words, survivorship bias occurs when an individual only considers the surviving observation without considering those data points that didn’t survive in the event.
Why survivorship bias happens
Survivorship bias occurs in many disciplines, professions, and fields of research. Survivorship bias can be attributed to the fundamental misunderstanding of cause and effect, and a misunderstanding of correlation versus causation.
Example 1 – Financial systems
The Survivorship Bias occurs in our financial systems, when individuals calculate performance results of groups of investments, such as mutual funds, using only the surviving data at the end of the period, excluding those funds or companies that no longer exist. Typically, mutual funds no longer exist due to poor performance, so omitting them from performance studies usually skews data in an overly positive light.
Example 2 – Medical research
The Survivorship Bias can also be observed in its impact on medical research. In 2010, the Harvard Medical School and Beth Israel Deaconess Medical Center (BIDMC), attempted to conduct a trial on trauma patients and to better their survival outcomes from different types of medical procedures. Due to the specificity of the trial outcomes and the assumption that more patients would survive than was accurate, the trial was only able to recruit 573 patients, of their 1,502 total patients. This study’s failure was due to survivorship bias, as the trial only included patients who had survived their initial injury, and who had then received care in the Energy Department before then receiving 4-8 blood transfusions.
How to avoid it
Once individuals learn about the Survivorship Bias, they can avoid the bias by considering what data points may be missing from a dataset and using accurate data sources that do not omit key observations.