Kora Reddy of asxiq is back as a guest contributer with part two of his introduction to backtesting metrics. It follows on from Part One which can be found here.
Payoff ratio (Ratio avg win/avg loss): Payoff ratio is the system’s average profit in Dollar terms per winning trade, divided by the average loss in Dollar terms per losing trade. Unless the system has a particularly high win/loss ratio, we look for high payoff ratios or average profit per winning trade divided by average loss per losing trade
Payoff ratio % (Ratio avg win/avg loss %): Average profit % per winning trade divided by average loss % per losing trade.
Luck Factor: This measure shows how the largest trade compared with the average trade and is calculated by dividing the percentage profit of the largest winning trade by the average percentage profit of all winning trades. The larger the value of the luck factor, the greater portion of the system’s results can be attributed to the largest winning trade, or, luck.
Percent wining months or years: Depending on the time horizon, a trading pattern that averages only one winning month out of twelve, or two winning years out of ten years, is unattractive. You need to look for patterns with at least five profitable months in a year and five profitable years in a ten-year period.
Sharpe Ratio: This is probably the most common measure used by large fund houses in comparing potential investments. The Sharpe Ratio was formulated by Nobel Laureate William F. Sharpe in 1966 as a measure for comparing the performance of mutual funds. This measure was introduced as a reward-to-variability ratio but subsequently came to be referred to simply as the Sharpe Ratio after its originator. Sharpe Ratio was one of the first statistical measures that factored both return and risk into a single formula, thereby giving us a single statistical measure of risk-adjusted return.
Sharpe Ratio = (APR – RFR)/ (StdDevAPR)
Where:
- APR = Annualized portfolio return
- RFR = Annualized risk-free rate (90 – day T-bills are typically used as a proxy)
- Standard Deviation of APR (StdDevAPR) = Annualized standard deviation of the portfolio’s returns.
- For short term trading as our holding period is only one or few days, the RFR can be set to zero.
Sharpe Ratio indicates the smoothness of the equity curve. The higher the ratio, the smoother the equity growth or decline. A Sharpe Ratio value of above 0.5 is considered good, while a value above 1.0 is excellent and a value above 2.0 is considered outstanding.
When choosing a pattern’s reliability in real-life trading, Sharpe Ratio is used on the assumption of a zero risk-free rate of return and at least a Sharpe Ratio of 0.5 from the back testing results.
Pessimistic rate of return: Pessimistic return on margin (PROM) is an annualized yield on margin that is adjusted in a way that pessimistically assumes that a trading strategy will win less and lose more in real-time trading than it did in its historical simulation.
PROM adjusts the gross profit by calculating a new, mathematically adjusted, pessimistic lower gross profit. The first step is to calculate the number of winning trades reduced by its square root or, in other words, adjusted by its standard error. This adjusted number of winning trades is then multiplied by the average winning trade to arrive at a new, adjusted lower gross profit.
PROM next adjusts the gross loss by calculating a new; mathematically adjusted, pessimistic higher gross loss. The first step is to calculate the number of losing trades increased by its square root or, in other words, adjusted by its standard error. This adjusted number of losing trades is then multiplied by the average losing trade to arrive at a new, adjusted larger gross loss.
A new adjusted gross profit, or loss, is then calculated using these adjusted pessimistic gross profit and gross loss values. This is then used to produce an annualized rate of return on margin.
The formula is
- PROM = {[AW x (#WT – Sqrt(#WT))] – [AL x (#LT – Sqrt(#LT))]} / Margin
- #WT = Number of Wins
- AW = Average Win
- #LT = Number of Losses
- AL = Average Loss
- A#WT = Adjusted Number of Wins
- A#LT = Adjusted Number of Losses
- AAGP = Adjusted Annualized Gross Profit
- AAGL = Adjusted Annualized Gross Loss
- A#WT = #WT – Sqrt (#WT)
- A#LT = #LT +Sqrt (#LT)
- AAGP = A#WT x AW
- AAGL = A#LT x AL
- Sqrt = Square root
An example here would help. Assume a $A 50,000 annualized gross profit, 50 wins, $A 20,000 annualized gross loss, 50 losses, and a starting capital of $A 1, 00,000.
As a basis for comparison, let us first calculate an annualized rate of return on the starting capital. This would be a 30 percent annualized return on margin (50,000 – 20,000) /1, 00,000 = 0.3 x 100 = 30%).
In contrast, let us look at the PROM of this strategy:
- Adjusted Number of Wins = 50 – Sqrt (50) = 43
- Adjusted Number of Losses = 50 + Sqrt (50) = 57
- Adjusted Annual Gross Profit = (50,000/50) x 43 = 42,929
- Adjusted Annual Gross Loss = (20,000/50 x 57 = 22,828
- PROM = (42,929 – 22,828) / 1, 00,000 = $A. 20,101
- PROM = 20.1 percent per year.
As you would note, the nearest digits have been rounded off in the above calculations.
This example clearly demonstrates why this measure is termed pessimistic. It assumes that a trading system will never win as frequently in real life as it had in testing and also that the system will lose more frequently in real life than it did in its testing. PROM is a robust measure because it factors in a number of significant performance statistics, such as gross profit, average win, gross loss, average loss, number of wins, and number of losses. Also, PROM by nature of the square-root calculation penalizes small trade samples; that is, the square root of 4 is 2 which 50%, whereas the square root of 100 is 10, which is 10%.
PROM is an excellent measure to compare the performance of different trading models; it is suggested that a trading model’s PROM be at least 50% of the historical returns.
Return on Account: we have used the starting capital of 5000 AUD and taken the total net profit as the basis for calculating the return on account.
For ex: if a trading strategy’s net profit is 1000 points, then return on account is 20%
The rationale for 5000 AUD, the average value of XJO during Jan 2003 till Feb 2013 is about 4480.98 and I’ve considered the nearest digit which is 5000. So with 5000 AUD a trader can buy one unit of XJO, without leverage on most of the days.
Compounded Annual Return %: The compound annual growth rate is calculated by taking the nth root of the total percentage growth rate, where n is the number of years in the period being considered. This can be written as follows:
- Beginning value: 5000 AUD
- Ending value: 5000 AUD plus the total net profit in points generated by the trading strategy
- #of years: is the difference between the first date when the trading strategy triggered its first trade minus the last date when the trading strategy triggered its last trade approximated in years to the 1st decimal
Calmar Ratio: A ratio used to determine return relative to drawdown (downside) risk in a trading strategy, and is calculated as: CAGR/Maximum Drawdown %. The higher the Calmar ratio the better. This ratio helps determine return on a downside risk-adjusted basis.
T-Test: The t-Test is a simple statistical test that tells you how likely these test results are to have occurred by chance alone. A t-Test of less than 1.6 favours chance, above 1.6 and one is more likely to have found something real – a tradable Key Idea. The higher the score given (over at least 20) the more likely one has found a tradable history.
The t-test is calculated as
t -test= square root (n) * (average trade %/ standard deviation of trades %)
A more stringent t-test value too look for is 2.1 for degrees of freedom 25 ( or sample size) as the two tailed P value at t-test of 2.1 and sample size of 25 equals 0.046 which by conventional criteria, this difference is considered to be statistically significant.
Most of the trading strategies presented in this book try to meet the following criteria in the historical backtest performance summary report
- Outlier adjusted profit factor is more than 2.0 and
- Average profit per trade of around 10 points ( leaving aside 4 points as a spread trading costs , a 6 point profit per trade as the trading edge)
- Sharpe Ratio of around 0.5 and
- T-Test is more than 1.6
.This above material is reproduced from the XJO Quant : High Probability Trading Setups on ASX 200 Index , Trading game members can avail 30% discount by using “TRGAME” discount coupon.
“five profitable years in a ten-year period.”
That will be a sobering thought for a few people.
*atleast* Rob 🙂