Historical Backtesting

The standard retirement backtest uses rolling periods. You pick a strategy, such as withdrawing 4% of the starting portfolio and raising the dollar amount with inflation each year, and a length, such as 30 years. Then you run the strategy starting in the first year of the data, again starting one year later, and so on until the last start year that still has a full 30 years of data behind it.

William Bengen’s 1994 paper used this method on US data from 1926, starting a hypothetical retiree on January 1 of each year. The Trinity study of 1998 did the same with 1926–1995 data, using the S&P 500 for stocks and long-term high-grade corporate bonds.

Each period keeps history in its original order. Returns, Inflation and the way stocks and bonds moved together all arrive exactly as they did, through the Great Depression, the high-inflation 1970s and the recoveries after them. That is the method’s great strength: the market path itself is not an assumption.

A backtest produces one outcome per starting year, and the way you summarize them changes the story. Bengen reported the worst case: with a rebalanced half-stock, half-Treasury portfolio, a 4% first-year withdrawal raised each year for inflation always lasted at least 33 years, while 4.25% could run out in 28. That became the 4% rule. Yet in 40 of the 51 start years he charted, 4% lasted the full 50 years his charts showed. A few bad periods set the rule, chiefly retirements that began in the late 1960s, just ahead of the 1973–1974 bear market and high inflation, a textbook case of sequence of returns risk.

The Trinity study reported a success rate instead: the share of periods that ended with money left. A success rate hides how badly the failures failed and how close the survivors came, while a worst case rests on a single period, so read both. Studies of the safe withdrawal rate still quote results both ways.

Both studies tested a generic retiree with rigid, inflation-adjusted withdrawals and judged success mainly at 30 years. A backtest tells you more when it matches your own plan:

A backtest is only as broad as the history behind it, and that history is shorter than it looks. Rolling periods overlap heavily: the 30-year periods starting in 1926 and 1927 share 29 of their 30 years. Seventy years of data yield 41 rolling 30-year periods but only two that do not overlap at all, so a 95% success rate rests on far fewer independent outcomes than the count suggests.

The record is also one country’s, and the most recent start years cannot be judged until their full period has passed: a retirement that began in 2000 will not finish a 30-year test until the end of 2029.

Above all, the future is under no obligation to stay inside the range of the past. FINRA’s communications rule bars broker-dealers from implying that past performance will recur, a fair standard for reading any backtest.

Outside retirement planning, backtesting means applying a trading or allocation rule to past prices to see what it would have earned. It is easy to abuse: try enough rules and one will look brilliant on past data by luck alone.

Under the SEC’s marketing rule for investment advisers, performance backtested by applying a strategy to periods when it was not actually used counts as hypothetical performance. An adviser may show it only with policies designed to keep it relevant to the audience’s likely financial situation and objectives, and with enough information about its criteria, assumptions, risks and limitations. When a financial advisor shows you a backtested chart, ask when the strategy was designed, whether fees such as the expense ratio and taxes are included, and how it has done since it went live.