Just remember, an examination of bet size in isolation presupposes that the other variables in your system, such as choice of instrument, entry rules, and exit rules (including initial and other stops), are already set. Obviously, this post if aimed at the more active system traders …

Using the test data, assemble some basic information at various bet sizes (R), such as the backtested cumulative annualized growth rate (CAGR), maximum drawdown (DD), and Monte Carlo simulated 90% confidence DD and Monte Carlo simulated risk of a trading-ending margin call.

Start at the highest R and look at the Monte Carlo risks of going bust. Do you want to go bust? If not, cross out any options that have non-zero numbers here. If you don’t mind some risk of going back to live in your parents’ basement, cross out only those non-zero numbers that seem “too high” to you (masochist!).

Start at the highest remaining R. Picture in your mind the amount of trading equity you will commit to the system. Now imagine you start trading this thing, and by the end of 2008 you have lost that Monte Carlo simulated 90% confidence DD amount! Would that give you an ulcer? Would a loss of that particular amount make you stop trading your system? Would that loss amount cause bitter arguments with you and your spouse? Then maybe that R level should be crossed off of your list. Continue this process until a DD level is reached that you find tolerable. Now check the CAGR against your predetermined absolute benchmark, and decide if this system is worth pursuing.

Remember, trading is not an arena where you should be concerned with how you think others view you, or to be deluded about how you view yourself. It’s not “manly” or commendable to try to trade a system that’s beyond your own gut-level risk tolerance; the courageous and smart thing is to know what you can tolerate, and what you cannot tolerate.

The answer to what you should do is inside you.

Even if you don’t have access to software that spits out Monte Carlo simulations and equity curves, if you have enough backtested system trades in a spreadsheet, you can get somewhere near the same process. Grab your data and some of the statistics links from my links page, and get cracking. Take your number of trades and percentage of losing trades from test, and assemble a table of confidence intervals around what the system’s percentage of losing trades might be. Use 90%, 95%, 99%, or whatever else you think is appropriate, but using those three will give you a start. Take the “worst case” or highest percentage estimate from each interval, and bring it to another table.

In this second table, use the losing percentage estimates to calculate the number of consecutive losses that might occur with 50%, 90%, 95%, and 99% odds against. Lay out the different confidence interval odds on the left column, the odds against consecutive losses on the top row, and fill the table with results.

For example, suppose my system has 99% confidence of a loss rate no worse than 65%. I want to fill the table cell for my 99% odds against number of trades. Using Excel’s Log function, I take Log(1-0.99,0.65) as equal to 10.69. In other words, with this 99% confidence estimate of my system’s loss rate, I should expect a losing streak of 10-11 consecutive trades to happen about 1% of the time.

When this table is assembled, you’ll have a rough idea of what kind of losing streaks are common and uncommon, and what kind of losing streaks you might expect at an extreme – which you’re certainly GOING to see if you trade for any length of time! Take this data, and multiply those consecutive loss counts by your prospective candidate R amounts, and give ‘em the old stomach test!

Stepping up a bit in complication, if you know the frequency of trades per unit time that your system yields, you could do your own calculation of maximum losing streak over the next ten years of trades, or assemble a spreadsheet Monte Carlo of your own.

Regardless of the math used to see the results, the answer to what you should do is inside you.

Kelly Criterion gives a very incomplete picture of the results, and one that’s not intuitive to a retail trader following the system. For example, a long-term trend-following (LTTF) system may pay winners average twice losers and win 35% of its trades, yielding a Kelly Bet optimum of 2.5% of equity per trade. The volatility of results at that level could be bone-chilling! A money manager can bail on a fund if it draws down too much, losing its investors, but the same drawdown might make the retail trader demotivated, or drive them out of the game if they’re dependent on their trading for income. Much better in my opinion to rely on simpler math that produces intuitive results like drawdown odds, and use those intuitive results and gut feel to determine if a given system is “for you” or not.

There’s one final thing that needs to be said about Monte Carlo simulations and statistical treatments of test data. Monte Carlo is only as accurate as the input data – it’s stringing trades or segments of equity into random orders. There could be, in the next ten years, some event that occurs naturally but just wasn’t in your test data, that blows your system past those parameters. That event could be good for you, or bad for you. You should think about whether you can live with the worst of it. Even using the less sophisticated method from “per trade” data, you should accept the inaccuracy of all backtested results and data derived from them as part and parcel of viewing only a SAMPLE of all potential price action.

If you liked this post, you might be interested in subscribing to my RSS feed. If you prefer, you can get a nightly RSS email update sent on the days that I post! There are convenient “Subscription” icons near the top of the right sidebar.

To view my actual trades and model portfolios for the different systems I track, visit The Rempel Report. If you’d like to become of member of The Rempel Report, you can register here. At The Rempel Report, I track model portfolios for four different mechanical trading systems, disclosing all results (good and bad) at regular intervals. I also track my personal portfolio, and disclose all trades before I make them. Members receive email notification of new posts and can contribute to the site through comments. Registration is still free!

All I know is what I read and interpret from the outside of the industry, and what I guess about human nature. My definition of quant is probably larger than the typical definition. I would include anything that was data-driven, empirical, objective, and deterministic as quantitative, and that would include a lot of value strategies and GARP strategies. I could share a backtest on Price/Operating Cash Flow related to Estimated Earnings Growth and you could see it either as a quantitative strategy that was to be executed mechanically, or you could see it as a GARP screen from which you did your qualitative research, talking to management, reading proxies, understanding the business model, etc.

I think the more typical definition of quant is pigeonholed into a few subcategories, like the high-frequency trading that really is more like glorified computerized market-making, or the derivative strategies on illiquid contracts that require complex valuation modeling to estimate the market value of positions, or black box covariance methods for determining position weighting. There are some quant shops who view stock selection in terms of factor models, and they cross from the “typical” view as I characterize it, into my point of view as to the broad definition of quantitative trading. Like the Bear Sterns EAFE Plus, or the model I examined in the TRPITS posts, I believe they tend to overcomplicate the issue.

I don’t think quant management (from the “typical” definition) is going away anytime soon, and is probably getting stronger as the tools progress. To a man with a hammer, everything looks like a nail, and computer power is pushing the limits of what can be done with the data. Additionally, there’s that fascination with complexity, and the marketing tool that complexity provides, which will manage to sell the product. In Europe and Asia, where these things are less well-explored, I think it’ll grow faster than it’s growing here.

For what it’s worth, I see the same thing happening in insurance ratemaking, where simple approaches are probably measurably better, all things considered, than the complex approaches, but the trend is towards complexity. Nobody believes in blocking and tackling, they all want to use trick plays.

Now, the more traditional ways of choosing stocks, like value, GARP, growth, forensic accounting, etc. are all still effective and pretty much all could be quanitified if you took the time to do it. Given the lack of “flash” in these approaches, and their supposed ease of replication relative to “typical” quant models, I don’t see these gaining in popularity relative to quant.

I find the institutional desire to invest in models that (look as if) only a physics or finance “Piled High and Deep” could have built them to be hilarious, because the “Piled High and Deeps” that run these things all came from the same half-dozen schools or so, all read the same research papers, all know each other, and all have the same data sources to play with. In reality, what they come up with will be rather easily replicated by competitor firms, precisely because of that “groupthink.”

If you liked this post, you might be interested in subscribing to my RSS feed. If you prefer, you can get a nightly RSS email update sent on the days that I post! There are convenient “Subscription” icons near the top of the right sidebar.

To view my actual trades and model portfolios for the different systems I track, visit The Rempel Report. If you’d like to become of member of The Rempel Report, you can register here. At The Rempel Report, I track model portfolios for four different mechanical trading systems, disclosing all results (good and bad) at regular intervals. I also track my personal portfolio, and disclose all trades before I make them. Members receive email notification of new posts and can contribute to the site through comments. Registration is still free!

Risk is the Volatility of an Individual Position.

Some stock speculators, large and small, use an options overlay strategy, such as selling covered calls, in order to mitigate the day-to-day mark-to-market of their individual stocks. In this way, they hope to eliminate the volatility of each holding’s equity.

One common technique is to use recent historical volatility, perhaps a multiple of the width of a Bollinger Band or the Average True Range (ATR) to set the position size. When setting position size via this method, the trader puts a theoretical limit on the day-to-day dollar variance of the position’s value. If the trader can scale the position continuously to adjust for changes in the price volatility, they can have a lot of control on the dollar variability of the position value.

When a commercial enterprise like a manufacturer needs a supply of some basic material, say copper, that enterprise is exposed to the spot price of copper at the time it is needed. Through buying and selling speculative contracts on the metal, or options on those contracts, the manufacturer can, over long stretches of time, smooth out their costs of obtaining copper. This is true for producers as well, since they can smooth out their product delivery price enough to make forecasts and manage their own operating costs which might involve energy contracts for the power needed to mine and transport their copper. These entities are concerned about the volatility of a single position (an input or output) and are participating in the speculative market to control that “risk.”

Risk is the Volatility of a Portfolio.

The futures contract speculator is the liquidity provider to the manufacturer and producer hedgers listed above. They get paid, on net, because the hedgers are more interested in smoothing returns than in generating positive income from their activities, and the inherent leverage of the contracts makes it far more potentially worthwhile than it would be on a cash basis. The mechanized trend-follower who speculates on futures typically uses large baskets of contracts, 20+ or even a hundred different contracts, in order to smooth their results for the portfolio. Think of this as an exercise in blending non-correlated systems.

When trading individual equities, there is a significant risk to any one stock; there could be lawsuits, catastrophes that impact a packing plant, economic outlook changes, or a simple miss of expectations for revenues or earnings. Provided that the overall method of selection has a statistical edge, the speculator can start using the “law of large numbers” by selected many stocks that hold the same selection criteria. Of course, there may be diminishing returns in the selection criteria as it gets less stringent to allow for more choices. A classic “old school” example is Ben Graham’s “Cigar Butt” methodology, which clearly dictates holding upwards of two dozen (if possible) of these NCAV “values” at any one time, because only a few will make it good and pay for the rest.

Some of the more quantitative shops may use something like the “black box” covariance matrix from the “Barra U.S. Equity Long-Term Risk Model” to limit the intercorrelation of their various positions, so that they don’t wind up holding 3 different equities that tend to move together, thinking they’ve mitigated risk – when they haven’t.

Read more about the benefits of diversification.

Risk is the Downside Potential of an Individual Position.

This is similar to viewing risk as the volatility of an individual position, except only the downside variability is counted as a negative.

The “old school” Ben Graham’s “Enterprising Investor” and “Defensive Investor” methodologies would buy stocks only after careful examination of several years’ balance sheets, income statements, and the like. The idea was to find, not just companies that were cheap relative to their current earnings potential, but to find good quality companies where the risk of the stock going to zero was reduced. It’s the value speculator’s, damn! there I go AGAIN, the value “investor’s” concept of “margin of safety.”

Another “trick” used by value-type guys is capitalizing on forensic accounting methodologies. Read more about fundamental analysis for solvency and earnings quality.

The purely technical discretionary trader accomplishes limiting the downside risk through selection of opportunities. Clearly-defined risk/reward parameters emerge, such as pullbacks to current support/prior resistance after a breakout, bounces off of significant moving averages, and the bottoms of breakaway gaps, when one is looking at charts all day. Now, regardless of the predictive value (or non-value, in many people’s opinions) of these levels, it is plain that the levels indicated by the charts present “cut and run” points of price action that DO limit the technical trader’s downside risk on that position.

Risk is the Downside Potential of a Portfolio.

Once again, this is similar to another perspective, but only counting downside variability as a negative. I love the actuarial phrase for this definition: “risk of ruin.”

Mechanized system traders in the futures world think of this number as “heat.” Using an R (risk per position) of 0.5 (percent of equity), a system may dial up the “heat” by having more positions with tighter stops and more leverage, or they may keep the same number of positions and relative stop size, and increase the leverage to have more R than 0.5. In this case the number of positions multiplied by the risk per equals the “heat.”

VAR (value at risk) is one measurement of this “heat” notion that many quants and hedgies use.

Anyone trading unleveraged instruments, like stocks or ETFs from the long side, has the theoretical downside potential capped at their invested equity; using margin accounts to go more than 100% long, or to go short, increases that theoretical risk. Using leverage exacerbates this amount, especially if trading near the minimum margin allowed in a futures account. More than one discretionary trader caught his wanker in a zipper at 10-to-1 leverage during last January’s equity slide! That’s what happens when you don’t know your leverage, you wind up increasing your “risk of ruin.”

When I look at maximum drawdown in backtest, as compared to backtested CAGR, I’m really comparing one measure of downside portfolio risk (drawdown) to one measure of return (CAGR). It’s similar to Seykota’s “bliss ratio.”

Risk is a Mistaken Position.

If the retail schlump “value investor” doesn’t understand the company he’s researching, he might run into this; perhaps it’s an electronics retailer that, on closer inspection, also invests in synthetic fuel plants for the purpose of getting the associated tax credits. In this case, if the “value investor” bought that stock, they introduced the risk of a mistaken position.

The institutional analogy might best be described as “style drift.” They wanted a mid-cap value fund, but the fund manager drifted into holding mostly large-cap growth stocks.

Risk is Third-Party.

The counter-party to your complex derivatives transactions blows up.

The source of your hedge fund’s leverage is in financial trouble and puts the kibosh on your borrowing, forcing a margin call at an inopportune time.

The retail schlump’s broker goes bust. Yes, there are legal protections here up to certain account sizes, but what happens to those with active strategies and open positions?

Risk is Not Achieving My Benchmark.

Of all the risks discussed, this is probably the one that needs the most thought, from the perspective of a retail schlump with an absolute benchmark. Unfortunately, most people don’t give a lot of thought to their benchmark, and don’t understand the difference between an absolute and relative benchmark, much less why they should use an absolute one. Oh, bother.

For the retail schlump who hasn’t yet evolved past comparing himself to the index return, that index’s return is a Mendoza Line that must be crossed. After all, it’s hard to justify the pain and expense of being an active trader if one can’t beat “buy and hold the SPY.”

The institutions seem to have devolved into comparisons to the index, because when they select managers, that’s exactly what they are usually happy with; beating the benchmark index, even if that means losing just less than the S&P 500 loses. Even when using more complex strategies overall, they’ll select the relative benchmark outperformance where they can find it, regardless of absolute performance, and then allocate assets to other classes through cheap index funds or through derivative strategies. Maybe they just never evolved past that …

The most sophisticated retail traders are looking at performance from an absolute perspective, based on either backtested results for their strategy, a necessary return on investments based on careful budgeting, or both. This is where most, darn near all, of the retail schlumps are missing the boat, because you can’t eat relative performance when the index sucks, and they abandon their (occasionally) carefully-crafted strategies when they have periods of underperformance.

If you liked this post, you might be interested in subscribing to my RSS feed. If you prefer, you can get a nightly RSS email update sent on the days that I post! There are convenient “Subscription” icons near the top of the right sidebar.

To view my actual trades and model portfolios for the different systems I track, visit The Rempel Report. If you’d like to become of member of The Rempel Report, you can register here. At The Rempel Report, I track model portfolios for four different mechanical trading systems, disclosing all results (good and bad) at regular intervals. I also track my personal portfolio, and disclose all trades before I make them. Members receive email notification of new posts and can contribute to the site through comments. Registration is still free!

My first exposure to Occam’s Razor was in some now-forgotten science fiction novella I read as a youth, and the idea just “clicked” with me as intuitively correct. In collegiate (or “university”) settings, I was exposed to the benefits of stepwise regression and paring down multivariate models through discarding those independent variables with low Student’s t. I use this paring down or pruning technique at work as well as when examining trading strategies or opportunities. My first question, when faced with complex models, has for a long time been “I wonder how many of those variables actually do most of the work?”

Most people, from what I’ve observed in my interactions, don’t think this way. They fall in love with the “optics” of complex models, because they look good, and tend to impress people who think broadly but not deeply. The common sales technique for model-makers is “model bloat,” adding complexity in order to impress the buyer, while having a deleterious impact on the signal-to-noise ratio of the model. I am not sure if this is intentionally misleading behavior on the part of the model-maker, or if the model-maker is simply falling prey to the same mental flaw of being unduly impressed by complexity.

A case in point, which affects almost all of you, is credit scoring. I’ve seen several of FICO’s models, including one common model which uses thirty-nine variables and impacts the prices y’all pay for certain financial services. From many years of working with it, I strongly suspect that a model with only five to eight of the variables would achieve substantially all of the predictive power contained in their thirty-nine variable model. The “wildly cynical” part of me suspects that FICO made a thirty-nine variable model because they would feel embarrassed trying to sell a five or eight variable model; the less cynical part of me suspects that FICO believes their own bullshit.

Here is an extended quote from the CIA’s Center for the Study of Intelligence, where they reference an unpublished manuscript of a study from 1973. I came across this a few years back, but the entire document is now making the rounds on this “series of tubes” called the internets, and the reminder sparked this post.

Eight experienced horserace handicappers were shown a list of 88 variables found on a typical past-performance chart–for example, the weight to be carried; the percentage of races in which horse finished first, second, or third during the previous year; the jockey’s record; and the number of days since the horse’s last race. Each handicapper was asked to identify, first, what he considered to be the five most important items of information–those he would wish to use to handicap a race if he were limited to only five items of information per horse. Each was then asked to select the 10, 20, and 40 most important variables he would use if limited to those levels of information.

At this point, the handicappers were given true data (sterilized so that horses and actual races could not be identified) for 40 past races and were asked to rank the top five horses in each race in order of expected finish. Each handicapper was given the data in increments of the 5, 10, 20 and 40 variables he had judged to be most useful. Thus, he predicted each race four times–once with each of the four different levels of information. For each prediction, each handicapper assigned a value from 0 to 100 percent to indicate degree of confidence in the accuracy of his prediction.

When the handicappers’ predictions were compared with the actual outcomes of these 40 races, it was clear that average accuracy of predictions remained the same regardless of how much information the handicappers had available. Three of the handicappers actually showed less accuracy as the amount of information increased, two improved their accuracy, and three were unchanged. All, however, expressed steadily increasing confidence in their judgments as more information was received. This relationship between amount of information, accuracy of the handicappers’ prediction of the first place winners, and the handicappers’ confidence in their predictions is shown in Figure 5.

With only five items of information, the handicappers’ confidence was well calibrated with their accuracy, but they became overconfident as additional information was received.

The same relationships among amount of information, accuracy, and analyst confidence have been confirmed by similar experiments in other fields.

How many indicators and inputs are in your trading (or economic) models, and how many of them actually do the heavy lifting? What can you live without and get the same results? Is every filter on your stock screen necessary? Do you really need more information?

Be back in a while.