6.2.3 The Danger of Over-Optimisation

Why the search for a perfect strategy tends to backfire

Continuously searching for a marginally better fund, a marginally better factor combination, or a marginally better market-timing signal generally means more frequent switching, more transaction costs incurred along the way, and more opportunities for a costly behavioural mistake to creep in at exactly the wrong moment. The theoretical benefit of finding a genuinely, meaningfully better strategy is very often smaller in practice than the real, tangible cost of the constant searching and switching itself.

Over-optimisation is the pursuit of the mathematically perfect portfolio at the expense of a portfolio that actually works in practice. It manifests as endlessly researching funds instead of investing, frequently switching to marginally "better" options (generating taxes and timing risk), holding many funds to capture every possible factor or theme (creating overlap and complexity), and spending more time on portfolio management than the incremental return justifies.

Why good-enough so often wins out in actual practice

A simple, well-diversified, low-cost portfolio, held with reasonable patience through a full market cycle, has repeatedly outperformed far more complex, frequently adjusted strategies once all the real-world costs and behavioural mistakes involved in that added complexity are properly and honestly accounted for. Simplicity itself has genuine, measurable value that is easy to underestimate in advance.

The opportunity cost of over-optimisation is real. An investor who spends six months comparing flexi-cap funds to find the "best" one has lost six months of compounding. At 12% annual return, ₹1 lakh invested six months earlier produces roughly ₹6,000 more in year one. Over 20 years, that six-month head start compounds to roughly ₹40,000 additional wealth. The difference between the "best" fund and a merely "good" fund over 20 years is likely smaller than the cost of the delay caused by searching for perfection.

Tax and transaction costs from frequent switching amplify the problem. Each switch from one fund to a "better" one triggers capital gains tax on the exit, which reduces the reinvested amount. Over a career of five or six switches, the cumulative tax drag can exceed any performance advantage the replacement funds provided. The best fund you hold today is almost certainly not the best fund with hindsight in 20 years, and that is fine; a good fund held consistently outperforms a sequence of best funds held briefly and switched with tax costs.

How to recognise this pattern in your own investing behaviour

If you find yourself frequently switching funds chasing whichever one performed best most recently, or constantly researching an ever more elaborate strategy without any of the previous versions ever quite feeling finished or settled, that pattern itself is often a more useful and telling signal than whatever specific improvement you happen to currently be chasing.