Survivor Bias Silliness

Survivor bias is a perennial problem in systems testing. Put simply survivor bias is  the logical fallacy of ascribing a value to something that has past a given bench mark or hurdle by virtue of longevity. In doing so you overlook those things that have disappeared along the way. For example lets assume I was testing a system on the S&P/ASX200 and I did so using the data as it is today but I set my testing parameters to look back a decade. The problem is that my testing will pick up results that were not part of the index at the time of the testing window. For example the test might pick up a stock that had moved from $0.01 to $100 and I would conclude that the system would have picked up this stock resulting in a massive return. The only problem is that this stock was not part of the index during the testing window. This is a problem in an index such as the S&P/ASX200 because of the extraordinary turnover in its constituents – the index is constantly being turned over so any result my system generates has to be viewed with the knowledge it might not have actually been part of the index and therefore would have been missed if this system was trading in real time

This misunderstanding creeps its way into a lot of the things that are written about markets, take for example this recent piece by The Motley Fool It implies that it is going to tell me how to turn $20,000 into $575,000 which is something I definitely want to know. Imagine my disappointment when its advice was that I should have bought TPG Telecom ten years ago and held it. The basic premise is that you simply buy something you know is going to go up and hold it forever (the expression Holy Grail springs to mind) This is cherry picking of results based upon survivor bias to justify an argument. Despite this I thought it would be good to have a look at the raw data of TPG just to check the numbers and the outcome of this can be seen below.

My numbers come out with some variation from $575,000 but they are close enough. Now if they wanted to give some really good advance they should have said go back in time and buy TPG but then sell it when you have just over $1M in your account. Whilst I was playing silly buggers with data and excel I thought about the companies I would like to buy with the aid of the way back machine and I instantly thought of going back to 1980 and buying Apple. This investment would today be worth around $171M which is not  too shabby. However, if we are going to disappear down the rabbit hole of stupidity we can always look to an investment in Bitcoin. Unfortunately, my data for Bitcoin on goes back to 2013 but $20,000 invested then would today be worth $2.63B. However before you get to cocky remember at its peak it would have been worth around $9.9B.

But all of this pales into insignificance with Ronald Wayne who sold his original Apple stake for $800. This stake would be worth around $100B at today’s market value.

So how do you turn $20,000 onto $575,000….step one invent a time machine.