Notes On Backtesting - What I Like
I’ve covered a lot of material in the past few months on dealing with risk measurements on backtested system data. You can find a review in Making Measurements of Risk and Reading on Risk. Here are some of my conclusions about me, and what I want in a system, what would bring me to a “happy investing place.”
I want to use end of day (EOD) data for trading, I want mechanized decision-making, and I want a system that allows me to be divorced from the day-to-day watching of the markets. At the moment, I am not interested in designing a system specifically for high liquidity, i.e., designed so that tens of millions of dollars could be traded through it without degrading performance. I do have two systems that fit that bill, but liquidity is not a primary concern for me as I don’t manage other people’s money.
Sharpe, Sortino, Alpha, Beta, none of those rock my boat. See the above links for definitions and discussion.
From looking at literally dozens of statistics and metrics applied to backtested trading results, this is what I want. They’re not “hard and fast” rules with razor-sharp cutoff lines, but ballpark measurements. These are just simple rules that don’t involve a lot of mathematical mas-, er, manipulation, to compute.
(1) The monthly and annual standard deviation of returns should be, at a maximum, around that of the overall market (S&P 500). This puts them around 4.5-5% monthly and 17-18% annually, ballpark.
(2) I desire a CAGR (cumulative annualized growth rate) of 20% at a minimum, preferably north of 25%.
(3) I don’t like equity drawdowns (DD) of more than 20% or so. This implies a “floor” on CAGR/DD of 1.00.
If a system meets those three requirements, I’m pretty sure it will score highly on most other metrics I find of interest. Given competing systems that meet those requirements, I would focus on the ones with the highest CAGR/DD ratios first.
That’s not too much to ask for!
December 28th, 2007 at 6:18 pm
OT of backtesting:
I track asset niches using ETFs but would like to expand my universe using some specialty mutual funds. The problem is finding a relatively passive mutual fund index from niches like:
- global engineering & construction firms, or
- asian consumer discretionary
Have you found good speciality mutual funds worth tracking?
December 28th, 2007 at 7:06 pm
I haven’t spent a lot of time looking for mutual funds - although I did use some mutual fund data in assembling backtests for the ETFs that didn’t exist years ago. The problem is their lack of passivity and openness in construction - you’re dead on there.
My screener shows 341 open, no-load, international equity funds available, but many of them are just different classes of the same fund, and the descriptions I see are pretty lame. Thumbing through the first few pages of them, I don’t see what you’d be looking for. I think I’d have to read every description to make sure.
If you have access to foreign exchanges, there are probably ETFs on European and Asian exchanges that do what you’re looking for. I don’t know anything in the U.S. exchanges that slices into “regional sectors.”
Try searching at IndexUniverse.com for ETFs.
Try Morningstar or maybe browse around at fundalarm.com (email them?) for information on the funds.
Best of luck!
December 30th, 2007 at 9:17 pm
I’ve been crunching 125 years of tick by tick data in the grains at the CBOT, KCBOT, and MGEX. I’ve narrowed down my search to 20 patterns I’m looking for. Since I like spreading the grains, I’m trying to optimize my entry and exit points.
Good Blog.
Jeff
January 3rd, 2008 at 8:35 am
Bill - Somewhere along the way I too shifted my comparative focus for evaluating systems to looking at max drawdowns. It’s something that is easily comparable across all systems, unlike some other measures. The one thing I would wonder about is how comparable the StdDev of monthly returns would be to that of a market index just on the basis of trade frequency. If the system trades frequently, then it’s not an issue, but for others it could be.