As it relates to the testing of trading systems, I use “scalability” to describe a concept, although that concept is something I have difficulty testing directly.

The concept of scalability is the general question of how the amount of money managed impacts the returns. More specifically, for each system (or management style, but I really think of those as “insufficiently documented systems”), there are a series of interrelated questions that must be answered. How much money is needed to execute the system with sufficiently low risk of ruin? (which begs another question … ) At what point should the manager stop adding funds to a system? What is the relationship between system returns, volatility, and assets under management, and how does the manager capitalize on this relationship?

In my limited experience, I’ve come across a range of systems with drastically different scalability, but just about every system I’ve come across has some kind of “sweet spot.” Long-term trend-following (LTTF) systems, when executed against a large basket of futures contracts, have a fairly high hurdle for retail traders to use them safely, and run out of room at points I would consider – but that many institutions would NOT consider – stratospheric. Microcap momentum, small cap value, and NCAV systems have sweet spots that only the smallest of fund managers can hit, but any schlub with a retail account can execute these strategies. Other systems may be near infinitely scalable on the large end, but require billions (instead of the mere million-dollars-plus that LTTF requires for relative safety) to execute with a low risk of ruin.

The problem I have in testing is determining exactly what that sweet spot is!

Here’s an example test – running stock trading system X with a list of qualifiers sorted by characteristic Y, with minimum market cap filtered at $1 billion.

I can simulate retail accounts by holding 5, 10, 15, or 20 stocks at a time. I can put together some risk-adjusted return curves by testing more holdings, say 25, or 30 stocks at a time, and from there I may find that, for THIS retail trader, maybe 10 to 20 at a time is the “right” number. But is the idea scalable? If the dropoff in risk-adjusted performance is drastic, then maybe not, but if it isn’t?

I can repeat the test holding 50 or 100 of the top qualifiers. Repeat the test holding ALL qualifiers (effectively not sorting candidates by characteristic Y anymore), and see what the average number of qualifiers, and the minimum number of qualifiers, was over the test period.

I can do all of those same tests with different market cap minimums, like $100 million or $500 million, and see if the numbers change.

I may determine that, for a system X where holding 20 or so stocks looks “just right” to me as “Joe Schmuckatelli, Retail Trader,” that a fund manager using that same system X could potentially hold an average of 344 stocks with market cap $1 billion or more, with 18% monthly turnover, over a test period of 11 years, with outstanding (for a fund manager, not necessarily for a retail trader) results.

While that’s a non-trivial answer, it doesn’t really get directly at the question!!! How much money can that system handle? At what point of AUM does a manager start moving the market, when playing in a sandbox that holds 344 stocks over $1 bil in cap, turning them over at 18% a month?