When Is A Benchmark Not A Benchmark

I have been reading snippets of the Productivity Commissions report into the state of superannuation in Australia and whilst my reading has so far been cursory nothing I have read so far sways me from my opinion that the best solution is to simply nationalise the entire thing and put everyone into an index based fund. This little rant notwithstanding one of the early things that has caught my eye is the definition of a benchmark used by the commission. Their definition is shown below –

This is a point of confusion for me since if I run a fund that invests in the local market my benchmark is the All Ordinaries Total Return Index – it is not an artificial portfolio. The use of a passive benchmark is done to asses whether the manager is adding value over and above what could be achieved by investing in the index. If a manager outperforms the index they are adding value – if they under perform the index they are a drag on performance. Generating a surrogate portfolio does not show this and is not actually standard or worlds best practice for benchmarking as it seems to set an artificially lower bar and lets managers off the hook for their lack of ability.

The problem with this sort of benchmarking rears its head when you begin to look at the number of fund supposedly outperforming this artificial benchmark, This can be seen in the distribution of returns displayed by the report.

This gives a somewhat distorted impression of the performance of the funds within their non representative sample. A simple reading of the graph above would indicate that the majority of funds outperform their benchmark but there are two problems with this as an interpretation. It is a profoundly narrow and biased sample that despite the Commission statement does suffer from sample bias and as already stated the benchmark is wrong. If I were doing this I do something along the following lines –

1. Use a public access data source such as Morningstar – from this database you can isolate individual Australian Superannuation funds.
2. Apply an industry accepted benchmark in this instance the All Ordinaries Total Return Index to the data collected.

If you do this you find that over a 10 year period only 53 of 3055 funds have outperformed the index. This means that instead of it appearing as 63.5% of funds outperform their benchmark you find that 98.27% of individual funds under perform the industry benchmark over a ten year period. Applying the long performance of the All Ordinaries Total Return Index to the Commissions sample produces  a very different result as the benchmark for the period under investigation sits at 8.24%.

This finding tallies with most overseas experience that shows that the majority of funds fail to outperform their industry benchmark over the long term. This is the true problem afflicting retirement savings – the majority of fund managers simply dont know what they are doing and they charge a small fortune for what they do. According to the commission this year Australians will pay $30 billion in fees for which the vast majority will receive no benefit.

One of the Commissions main recommendations is that investors be presented with a list of the top 10 performing funds – this list is selected according to a panel of experts ( god forbid) every four years. The notion of chasing winners is a powerful one and when tested it does work – it completely trounces the idiot idea of chasing losers which believe it or not if is a staple of the investment industry. However, chasing winners is based upon constant rebalancing each year so that each time a new winner is found the funds are moved into it. Having a static list that is only reviewed every four years does not take into account the active approach required for this strategy. For example the table below lists the top ten performing individual funds for the year.

Lets play a thought experiment and assume this approach was in use ten years ago – your drop down list whilst it might not show individual funds such as this would have the same tone and tenor. So we can use it as a proof of concept. Quite rightly you select the top performing fund and leave it at that. However, performance is not static and whilst good managers generate persistence in performance things can change.

In this instance the change has not been too drastic, our fund has slipped from first to second place – hardly a disaster. However, it is possible for a fund to go from first place to last place in the space of a year. It is possible to generate out performance one year and then over the long term completely under perform the benchmark. This is why a passive approach to selecting winners on a regularly basis may not generate the increase in returns that are hoped for – it might be better than what is in place now but I am not convinced it will generate the solutions the Commission hopes for.