This Friday we have Kora Reddy of asxiq dropping by as a guest contributer, who for the next three weeks will be going through some of the intricacies of back testing.
Why Backtest a strategy?
Quick answers to the question:
- to determine whether a theory or hypothetical construct is valid in historical testing
- to summarize the overall hypothetical performance of a system and to analyze its various aspects in order to isolate its strong and weak points.
- the purpose of testing a pattern or a trading system is simply to find out what will work best on the basis of what had worked best in the past. You test drive a car before buying it; there is no reason why you shouldn’t test your trading strategy before applying it.
What is Backtesting?
Definition of backtesting from investopedia
Backtesting is a key component of effective trading-system development. It is accomplished by reconstructing, with historical data, trades that would have occurred in the past using rules defined by a given strategy. The result offers statistics that can be used to gauge the effectiveness of the strategy. Using this data, traders can optimize and improve their strategies, find any technical or theoretical flaws, and gain confidence in their strategy before applying it to the real markets. The underlying theory is that any strategy that worked well in the past is likely to work well in the future, and conversely, any strategy that performed poorly in the past is likely to perform poorly in the future. This article takes a look at what applications are used to backtest, what kind of data is obtained, and how to put it to use!
Important metrics to consider in the Backtesting
There are many factors traders pay attention to when they are backtesting trading strategies. A thorough back test of a trading system should include the following information. Here is a list of the most important things to remember while backtesting:
Number of years analysed: Although it is desirable to test as much data as possible, the minimum should be at-least last 4 years of recent historical data. Often people test the previous bull market’s data if they figure that the current market resembles a bull market, and vice versa. The most important data tranche is the most recent one as that is what the current market phase is, as you want your trades to work there.
Number of trades analysed: More important than the number of years analysed is the number of trades analysed. A typical pattern should generate at least 20 to 25 trades over the test period in order to support the statistical significance of back-testing results. It would be wrong to assume that a pattern that had formed only a couple of times in the past is a guide or reference to a good trading opportunity in the future.
You may perhaps have come across the term called accuracy while reading statistics. Accuracy usually increases as the number of samples becomes larger and the measurement of deviation or error becomes proportionately smaller. Accuracy is calculated as follows:
Accuracy = (1- (1/ Square root of the sample size))*100.
This concept can be extended to the number of trades analysed. For example, with a sample of four trades, the error is 50%. If a system has had only 4 trades, whether profitable or loss-making, it is very difficult to draw any conclusions about performance expectations. To reduce the error to 10%, the sample size has to be 100 trades. But this could be tricky in respect of a system that might generate only 3 or 4 trades in a year. To compensate for this, the identical pattern can be applied to other markets and the sample size thus increased. By keeping the sample error to no more than 20%, the risk of small sample size can be minimized.
Percentage winning trades: This is not as important as one might think. In reality, few patterns have more than 70 percent winning trades. Patterns that are correct as little as 35 percent of the time can still be good systems, whereas systems that are accurate as much as 90 percent of the time may be bad systems.
Gross Profit: is the sum of points generated by all profitable trades
Gross Loss: is the sum of points generated by all loss making trades
Total Net profit: is gross profit minus gross loss
Total Net profit %: is the sum of all trades profit or loss in percentage terms add together
Average profit per trade: This measure tells you what the average profit per trade for all the trades has been, minus commission and slippage. The average profit per trade figure is important as it considers all profits and all losses. Some people might question – and legitimately, too – whether, say, a 40-point average profit would vary to a great degree from the underlying XJO value. For example, a 40-point gain translates to less than 1 percentage gain when the XJO is trading above 4,000 levels, as opposed to a 2 percentage gain when the XJO is trading below 2,000 levels. So, it’s important to view the trade details in percentage terms as well.
Median profit per trade: in probability theory and statistics, median is described as the numerical value separating the higher half of a sample, a population, or a probability distribution, from the lower half. The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking the middle one. If there is an even number of observations, then there is no single middle value; the median is then usually defined to be the mean of the two middle values.
The median can be used as a measure of location when a distribution is skewed, so, it’s important to view the median profit per trade (and profit percentage per trade as well) to be in trading strategies favour. For example if the average profit per trade is, let’s say 0.5% and median profit per trade is -0.2%, avoid the system.
Largest single losing trade: This measure indicates how much of the drawdown is the result of a single losing trade. In real-life trading, this helps you adjust the initial stop loss. For example, if the average losing trade was $A 1,000 and the largest single losing trade was $A 8,000, as you would readily guess, a good portion of the average losing trade is borne by the largest losing trade. If you had a better way of managing the largest loser, your overall system performance would be considerably better. You should investigate further the cause for the larger losing trades. In real life trading be prepared to encounter an even higher largest loss, than thrown up by back tested results and brace yourself to handle such situation.
Largest single winning trade: This is more important than the largest single losing trade. Why? Suppose, for example, your total hypothetical profit was $A50,000 , and say $A 37,500 of this is attributed to only one trade (e.g. Short with 5X leverage , as you believed in sell in may strategy at end of April 2012 and covered at end of May 2012), then what you have is a distorted average trade figure. It’s often a good idea to remove such an exceptional single trade from the overall results and re-compute the system performance in order to confirm whether the trading system is actually good enough to trade. In real life trading, be as realistic as possible and be prepared that you may never encounter that largest winning trade derived from the back tested results.
Profit factor: Profit factor is the system’s gross profit divided by gross loss. Look for systems that have a profit factor of 2.5, or higher.
Outlier adjusted profit factor: With any trading pattern, you are going to have one or two exceptional wins. The chances of these trades recurring in the future are very slim and shouldn’t be considered in the overall performance summary. It is often a good idea to remove the largest single winning trade while calculating the outlier adjusted profit factor. And is the way outlier adjusted profit factor is calculated in this book when presenting backtest performance summary.
You might want to consider removing even the top 5% winners. For example, if the number of trades is 40, then remove top 2 largest winners, if number of trades is 60, and then remove top 3 largest winners. Look for trading systems with an outlier adjusted profit factor of more than 2.
Maximum number of consecutive losers / winners: The maximum number of consecutive winning and losing generated is, more often than not, purely psychological. Even using an excellent trading pattern is no guarantee that you will only have winning trades in succession all the time. In other words, there are bound to be a string of consecutive losing trades. But not many traders have the ability to maintain their discipline through four or more successive winning/losing trading trades. Even at the third consecutive loss, you would find many traders ready to abandon their system, thinking that either the system is going through rough patch. To be a winner one would need to weather such storms and be able to take ten or more consecutive losses in one’s stride.
There is another problem that few of the traders encounter, thinking that their system is going through a fluky winning streak after hitting 4 or more consecutive winners. Remember Black Caviar currently holds a winning streak of 23 from 23. The point we are trying to make here is, the human mind cannot relate easily to an unbroken string of successes. All of us expect a failure to happen after a successful run. And once a good run has been broken, we again wait for success. In trading, though, if you are on a clear winning streak, press on. Don’t allow the fear of loss to stop you in your winning streaks, in short as they say in trading manuals, “Let winners run, not the losers”
Gain during max winning streak: is the sum of gains in point terms during the period when the trading strategy had maximum consecutive winners, simply addition of all the profitable trades in point terms.
Gain during max winning streak %: is the sum of gains in percentage terms during the period when the trading strategy had maximum consecutive winners, simply addition of all the profitable trades in percentage terms.
Loss during max losing streak: is the sum of all loss making trades in point terms during the period when the trading strategy had maximum consecutive losers, simply addition of all the loss making trades in point terms.
Loss during max losing streak %: is the sum of all loss making trades in percentage terms during the period when the trading strategy had maximum consecutive losers, simply addition of all the loss making trades in percentage terms.
Maximum drawdown: This is one of the most important aspects of a trading system. A very large drawdown is a negative factor. Maximum drawdown is the largest peak-to-valley loss of a trading system’s historical profit curve. Maximum drawdown can be presented in absolute Dollar terms.
Maximum drawdown (%): As discussed earlier, maximum drawdown is the largest peak-to-valley loss — in absolute Dollar terms — of the trading system’s historical profit. Now, suppose you would like to determine the efficiency of a trading strategy in terms of the overall returns it provided on your starting capital. In that case, we can calculate the maximum drawdown as a percentage of the starting capital.
This above material is reproduced from the XJO Quant : High Probability Trading Setups on ASX 200 Index , Trading game members can avail 30% discount by using “TRGAME” discount coupon.
Part Two will follow next Friday
