The Family Office Chronicle January 2024
"Ocean wave washing over the year 2024 written in the sand, revealing 2025 underneath."
Reflect on 25 years of change and opportunity in investing. Learn how to maximize net returns, tackle inflation, and secure your financial future.

Sliver Ring of Time

The Twenty-First Century is No Longer New

Mortality, the awareness of life’s finiteness, occupies a central place in the human experience. Unique among all living beings, humans possess the self-awareness to reflect on their own existence and its inevitable end. As social creatures, we also seek meaning in our relationships with family, friends, and social groups. Due to the explosion of longevity and wealth that occurred in the past century, we today have the challenge and the opportunity to plan for many years spent in retirement, and 25-year anniversaries are a good time to assess where we are.

This Century’s Second Quarter Begins

In 2006, a financial planning firm in Oakland, California, began a longitudinal survey of Baby Boomers, the first of whom turned 60 that year. Among other findings in that inaugural poll was this paradoxical gem: Boomers simultaneously believed they would “never get old” and “would live to 100.” This reminds us of the words of Scottish mathematician Eric Temple Bell, who said, “Time makes fools of us all.”

The average American will experience three quarter centuries; lucky ones will have four. It’s natural to reflect upon these milestones and for investors to take stock of their progress and examine what may need to change. All Baby Boomers (the youngest of whom are now at least 60), all GenXers, and even some Millennials will have a strong sense of where they were when the 20th Century came to a close and this last one began.

Mature Americans will remember that 25 years ago, we were in a general meltdown over Y2K, the fear that the machinery of modern life would break down because most computer programs only had two digits for year entries. The country elected a “compassionate conservative” to swing the pendulum back from a progressive era, and the Dow Jones Industrial Average began the 21st Century at 10,646.

As we begin 2025 and the second quarter of this Century, we see a strikingly similar litany of concerns: Artificial Intelligence has replaced Y2K as the tech boogeyman, Trump Part Deux represents a fundamental turning of the political wheel, and the Dow opened the year at 42,514.

In some ways, everything has changed, yet in others, nothing has. We worry about technology, we argue over policy, and the stock market cycles up and down, sometimes with wild volatility, yet over the long haul, produces significant wealth. Blue chips more than quadrupled in the period, while the S&P 500 produced an average annual total return of 7.93%.

There are no 25-year periods of simple peace and prosperity that we can remember. This past one saw two major bear markets, multiple wars, a pandemic, a global financial crisis, the rise of China, the takeover of e-commerce from old-school retail, and seven of “the most important elections in our lifetime.”

That level of change isn’t terribly unusual: the previous quarter century saw the ignominious end of the Vietnam War, Watergate, two energy crises, historic inflation, a deep recession, the fall of the Berlin Wall, the breakup of the Soviet Union, and the birth of the Internet. The S&P 500 annual total return averaged 16.2% per year in that period.

The French are known for their pithy observations on life, and they give us one that may be useful here: the more things change, the more they stay the same. That sentiment can be a pretty useful remedy for all the lurid headlines and forecasts of doom that make up so much of our news.
The obvious takeaway is that investors are rewarded for taking the long view. When we look at the 50 years between 1975 and the end of 2024, we find that $100 invested in the S&P 500 at the beginning of the period would have grown to $31,605 by December 31 of last year, a total return (including dividends) of 12.24% per year.

Yes, there were periods of high inflation in that half-century, but the net return would have been 5,324% or 8.34% per year. That means your wealth would have doubled in real terms about every 8.6 years. Double anything six times, and you’ll get a big result.

The Difference a Day Makes

Unfortunately, at least so far as investing goes, we don’t experience quarter centuries as one single block of time, but rather as 9,131 discrete days. In every quarter century, some of those days are scary or exciting enough to trigger our fight or flight or FOMO (fear of missing out) instincts. In the market, just a handful of days in or out can make an enormous difference.

A landmark study conducted by Bank of America in 2007, and looking back at data since 1930, found that “if an investor missed the S&P 500’s 10 best days in each decade, total returns would be just 91%, significantly below the 14,962% return for investors who held steady through the downturns.” “Significantly” is a pretty weak term for a disaster on that level.

The great news is that you can be 100% certain to capture the 10 best days in any decade simply by staying put. The not-so-great news is that you will also capture the worst 10 days. However, given the (at least so far) inexorable rise of the market over time, the tradeoff is worth it.

That tradeoff, the psychic wear and tear of bull and bear cycles of recession and recovery, can wear a person out. It wears most people out, in fact, which explains the persistent underperformance of retail investors versus market indexes. However, just because you should avoid market timing doesn’t mean that there’s nothing you can do to improve your financial security. In fact, there’s plenty.

2025 is Here—Time to Get Busy

As you can see from some of the numbers above, the first and most important lesson of stock market investing is that it requires patience. Time washes away the volatility, and the American economic machine relentlessly reinvents itself as a more productive, competitive, and consumer-friendly ecosystem. The gross returns are there for virtually anyone with the patience to capture.

Take note of the term “gross returns” in that previous paragraph. As any financial pro will tell you, it’s not what you make, but what you keep, that matters. And this is where you can exert a profound influence on your retirement security and your legacy. Let’s examine some of the things you can do to maximize your net compounding.

There are a number of corrosive elements that eat away at your gross returns. While they can be examined individually, they actually work together. This can make things a little complex, but not beyond the ability of a typical investor.

The first and simplest attack on your gross returns can come from costs, in the form of management fees, servicing fees, and trading and transactions costs. That is the simple way to think about your portfolio returns, but a more sophisticated and potentially valuable way is to think of costs as another form of investment.

Disciplined and experienced investors will know that all costs are not created equal. As an extreme example, some of the wealthiest and most experienced investors in the world, pension funds, family offices, and billionaires will hire a hedge fund manager for a 2% fee and a forfeit of 20% of any portfolio gains. Clients of these managers are willing to pay these seemingly exorbitant fees because they believe they’ll enjoy otherwise unobtainable returns. Would you pay anyone 2% of your portfolio’s value plus 20% of your gains? You might if you believed you’d earn 25% per year. The net result would be a gain in the 18% range, and you may well believe the manager earned their fees.

In the real world where most of us live, account management fees can also produce a significant return of their own. For example, if an advisor could offset gains with tax loss harvesting, intelligent asset location, and low turnover, they may enjoy significantly higher net returns than they’d otherwise have. Vanguard and other firms have shown that these types of portfolio management disciplines can produce 3% or more than an unmanaged account would. If you paid 1% in costs to receive 3% in benefit, it would be an economically sound investment, would it not?

A second silent killer of returns is inflation. Since 2021, the dollar has been devalued by over 20%. This means the first 25% of your portfolio returns in that period acted simply to keep you where you were, meaning you had zero real growth up to that point. This put a real challenge on fixed income investors, and meant that holders of cash were slowly going broke.

Overcoming inflation requires intelligent diversification and exposure to higher-returning (and therefore riskier) asset classes, like stocks. Many investors who need income from their portfolios make the mistake of thinking they need dividends and interest income from bonds to provide that income. Not true.

More experienced investors realize that income can be generated in a tax-intelligent way from virtually any liquid portfolio, but also realize that managing that portfolio may be beyond their abilities or simply not how they choose to spend their time. This is where the cost of a competent advisor can actually provide another source of net return for you.

More than costs or inflation, taxes are the greatest threat to your portfolio’s growth. A portfolio with high turnover can produce short-term gains, which are taxed as ordinary income in non-qualified accounts. The wealthiest families hire teams of tax and asset management professionals to avoid unnecessary taxes, and find that those tax savings more than justify the cost of the team.

The IRS Code, state tax codes, and tax court case law combine to tens of thousands of pages. All that tax law intersects with the complexities of securities and portfolio architecture to create a confusing and fraught experience for most Americans. These people can find relief through holistic retirement planning, in which tax strategies and portfolio design are blended into a single discipline, with the goal of maximum risk-adjusted returns.

A Good Place to Begin

All this complexity can cause some investors to simply carry on as before, essentially doing nothing because they don’t know where to start. We can help you with that, and suggest a good place to begin is with an analysis of your portfolio.

As part of our family office services, we can provide that analysis for you, free of charge. Armed with your latest portfolio statements, we can assess your total costs, your tax efficiency, your overall diversification, and your total risk exposure. Most investors will have never seen these core performance evaluations.

We call this proprietary report Taxes First, Then Math™. You can use it to help determine if your costs are producing a net benefit for you, if you’re overpaying taxes unnecessarily, if you are over-concentrated in asset classes, or if you are under or over-exposed to downside risk in your portfolio. All of these are good things to know, and could help you increase your net compounding and, therefore, your retirement security.

A silver anniversary symbolizes the lasting but finite influence of 25 years on people and events. The 21st Century is no longer new. We can mark this significant milestone, and we can use this moment to make the next 25 years a more fruitful and productive one for ourselves and our loved ones.