Survivor Bias – What does “random” survival look like?

Imagine a sales team of 1,000 salespeople working for Acme Widgets. The management philosophy at Acme Widgets for the last 20+ years has been that they let the bottom 20% of performers go every year and replace them with 200 new salespeople to keep the sales force at 1,000. In this way, they keep the best people around and give newcomers an opportunity to join the group of winners.

Under this (contrived?) scenario, what is the composition of the sales force?  Can I identify a veteran salesperson who survives many years based on their sales skills? Does the sales force look different if the salespeople survive based on skill or due to random attrition of 20% per year? 

To try to understand where my intuition might lead me astray, I ran a simple model of random attrition of 20% of the sales force every year for 20 years.

Below is a table showing surviving sales people from each year back to 1982.  These are the sales people who are still working for Acme Widgets.  For example, 14 of the 200 new recruits in 1995 are still with Acme Widgets, while only 2 of the 200 newbies in 1987 have survived.

Now imagine the conversations between the new recruits and the old timers… 

First, all the old timers will have noticed that new recruits “just aren’t made like they used to be.”  That is, they drop out at an alarming rate. Notice the large gaps between the numbers of survivors in the first few years.  “You can’t count on these guys being around long.” Thankfully, the old timers don’t disappear nearly as fast.

Second, stores old timers tell of their sales exploits are likely to be of “skills needed for survival”—otherwise why would they have survived so long when nearly all of their contemporaries are gone? (And there is no one around to argue otherwise—“Survivor Bias”.) 

Further, everyone except the first year recruits has survived their entire time at Acme Widgets. So anyone you talk with will report that they are among the successful. The shorter-timers may be skeptical of their long-term prospects, but the longer they survive the more likely they are to identify as a survivor.

Third, more than half of the sales force has been there for 3 years or longer. This looks like stability. The high attrition rate would horrify the survivors if it left empty desks (or bodies).  But Acme keeps the floors clean and the total sales force at 1,000 persons. On any given day, the desks are full and the sales team is hard at it selling the Widgets.

Does a sales force that identifies and keeps skilled sales people look any different? If so, how? 

In our random example, 10% of the group at any given time has survived 10 years or more. These could be the “old horses” (had some big years early on so Acme keeps them around) or the superstars (had many big years). Does our perception of the proportion of the population that are superstars merely fall out from laws of random attrition?

These numbers come from random selection of the bottom 20%.  Without models of individual attributes, this I what the population looks like. 

I don’t want to argue against skill nor against promotion on merit. But I find the fit of the World to this randomness explanation troubling…

Here is the table for 40% attrition per year.  This might be appropriate for something more risky like forecasting markets (or anything really).