I am in my fourth year as a faculty member and experiencing the typical “how do I get these grants funded” struggles that many, if not all, of us face. Over the past year, I have been given a truly staggering amount of conflicting advice from mentors. This has led me to thinking about survivorship bias in the advice we receive from well-meaning mentors, Twitter, blogs (ahem), and books. The irony is not lost on me: after writing advice/experience blogs on and off for almost three years, I am finally writing one on survivorship bias. This time, n= all of us.

Survivorship bias: Survivorship bias is bias that occurs when only survivors are examined. A classic example of this you will see come across Science Twitter is a graphic of an airplane with red dots, which is a reference to airplanes returning from combat during World War II with bullet holes on the wings. History goes, that in an attempt to minimize airplane loss, military intelligence suggested that they reinforce the areas that were targeted. However, Abraham Wald pointed out that it was the bombers that did not return that represented lethal strikes to the airplanes, and airplanes that had returned likely represented non-lethal strikes. Thus, it was more appropriate to reinforce the areas that never returned with bullet holes, like the pilot’s cabin and the engines. In medical research, this amounts to performing a clinical trial and only analyzing the patients who survived the intervention.

Consequences in an academic research career: There are two major points to consider relevant to survivorship bias in an academic research career. The first is perhaps the most important: we obsess over figuring out “what worked” for successful applicants (survivors). For example, to characterize what makes a successful faculty application, we examine postdocs who successfully transitioned into faculty appointments and characterize what their CVs looked like. Every single postdoc association is likely to have the local new PI come in and talk about how to get a tenure-track academic job. Is there value in these types of seminars? Of course, but we all should be disclosing that this is what worked for us, it may not work for others, and others have found success other ways. The other side of survivorship bias in academic research careers is we become surprisingly superstitious and convinced of the importance of certain things, like the number of aims in a grant or the font in which they are written or how the text was justified (or not). Again, each of us is strongly influenced by what has worked or not, no matter how tenuous the correlation.

Everyone in this career path has survivorship bias: Everyone in this career path, from graduate students through tenured faculty, is biased. Many of us will say, “I did X and it resulted in Y,” or “I do not do X because it did Y.” Advice on grant writing is particularly rife with these types of comments, which is unsurprising considering the paylines at some institutes. I have been told to submit something every grant cycle (or only when the grant is ready or more than one grant per cycle). I have been told to only focus on big grants (or only pilot grants or a mix of grants or only non-NIH grants). I have been told to get more papers out before submitting (or only include unpublished preliminary data or submit as soon as possible so paper expectations are low). Whose advice is the right advice?

How to synthesize less-biased advice: My best advice (ha) to even out the biased advice we get is to solicit input from many individuals on broad topics (like how to run a lab, how to organize time, etc.) and get advice from qualified individuals on specific topics (like how to make this grant on Y competitive at study section X). On broad topics, collecting advice is a good approach. For example, there are many ways to run a research program, and some may not be appealing to you. If you do not sample broadly, you might not discover what works for PIs with your management style. Conversely, some advice should come from specific individuals. Advice on grant content from a standing member of a study section from your institution is priceless. Advice on grant content from your fellow new PI with a grant from said study section a little less so, although it is useful to see the expectations for early stage investigators (ESI). Advice from a legend in the field who writes always-funded renewals, who could turn in a piece of paper with “I need $1.25 million for five years” in the spirit of Otto Warburg and still be funded, should probably be used for high-level conceptual advice.

There is one shining piece of good news when it comes to survivorship bias: most of the mistakes your mentors tell you they have made are likely to be survivable. So go forth, take a chance, and make some mistakes. But when you succeed and tell tales of your success, admit that you advice is entirely biased. Stay tuned for more tales!

Did I get it wrong? Need to vent? Feel free to send some electrons my way in the comments, via Twitter @PipetteProtag, or through traditional electronic mail pipette.protagonist@gmail.com

About the Author

Screen Name

Pipette Protagonist, PhD

About Me

I tell tales of career development awards, job searches, the tenure track, and life in a clinical department as a PhD.