I was having breakfast with Scott L out systems tester extraordinaire the other day. Our chice of venue for such outings is a cafe in Brighton. Not because it is half way between our two abodes but rather that it boast a boat ramp. And as everyone knows a boat ramp is the best free entertainment you can have.
Our conversation turned to system design as it often does and Scott had done some interesting work looking at the impact of changing markets on system performance. To illustrate this point he tested a simple generic system applied to the ASX 300, NASDAQ 100, S&P500 and the Russell 2000. It is important to note that exactly the same trading system was applied to each market. The only variable was the market used.
The basic data looks like this.
Whilst this data looks interesting, it is when you begin to pick it apart that some things jump out at you.
Interestingly, this system outperformed the market in all markets that it was applied to. This rate of return whilst modest combined with the very low drawdown would put almost every fund manager in the world to shame. But that is a different issue.
What is really interesting is the Monte Carlo results. In testing a system it is important to make certain that the system is truly robust and that the results achieved are not simply dependent upon the order in which the trades were taken. This is done by what is known as a Monte Carlo analysis which reorders the results to see what happens if trades are taken in a different order. In simple terms Monte Carlo analysis makes certain that you simply did not get lucky.
These results show something very interesting in terms of the robustness of a system when applied to different markets. The most compelling feature of this table is the Russell 200o results. During this simulation the average profit for the Russell 2000 was a paltry $20,977 and even more startling is that this system has the potential to actually go broke when applied to this market.
The chart below shows the pronounced difference between the various markets.
It is important to remember that this is an identical system applied to differing markets. It is the market that is making the difference to the performance not a variable within the system.
The take home message for this particular simulation and for trading in general is that your choice of markets matters.
Very interesting.
For me that raises the question, do all markets at some stage in their lifetimes go through similar stages. ie the Russel at this time period has 220 trades initiated, 67% of which are stopped out at intial stop. At 147 of these trades, thats more trades that got stopped out at intial than the other markets traded at all.
If instead of 220 signals, it only gave a similar number of signals to the others, ie 100 signals with the same stats profile other wise, would it have been as bad?
Is there a time period where the Nasdaq has a similar set of results? Can we recognise when a market is going through a period such as this and step out of it?
As a side note, I reckon I have had that experience a few times, large number of initial entry stops being hit, again and again. It hurt.
I don’t expect an answer to each of these questions, I am thinking aloud in some respects.
Great bit of data to mull over.
Cheers
Without knowing anything about the system it’s hard to see what is so different about the Nasdaq. I guess the point of the article is to illustrate the fact that it can happen not the whys and why nots
As a keen statistician in a previous life I just get the feeling that we are all missing something when we apply detailed (almost forensic analysis) to share price action… how do we become “share whisperers”?…(play spooky music)…
This kind of Monte Carlo analysis is simply bunk.It in no way reflects what happens in the real world.
Mixing up the trade order randomly destroys the serial correlation that always exists in markets thus giving very misleading results.
And you would advise what?
I would advise that you do some research on the subject.
Such as?
Perhaps you could expand upon how you do your testing by making it public.