Suzy is 63. Just retired. Staring down Social Security. She wants to minimize taxes.
She opens a chatbot. Types her details. Gets an answer that sounds so sure of itself.
“Claim now. Convert this. Here is the why.”
It looks right. It even shows the work. So Suzy clicks submit and ignores the financial planner. Maybe it was fine. Probably not. The bot missed that her spouse is young but sick, which changes the math entirely. It ignored that her conversion plan will hike her Medicare premiums in two years.
Suzy might never know she was wrong. And the AI? It won’t call back.
She isn’t alone. People are pouring their wallets into these bots. A 2024 Pew Research Center study found that 34 percent of U.S. adults have used ChatGPT. Among those under 30? The number hits 58 percent.
And some of them are losing money. A 2025 Pearl.com survey of 2,00U.S. adults showed 19 percent lost more than $10 following AI financial advice. Gen Z investors lost money 27 percent of the time.
These aren’t hypothetical risks.
“Confidence is not competence.”
People are already paying the price. As a finance professor, this specific failure mode scares me more than anything else. We are arguing about AI in the wrong way.
The Two Sides of Blind Trust
We usually split AI complaints into two camps.
One group trusts the algorithm too much. Researchers call it algorithm appreciation. You treat the bot like an oracle. The other group trusts it too little. Algorithm aversion. They ignore useful tools because they’re cynical.
I think this is false dichotomy. Both behaviors stem from the same question: can you tell when it is lying?
If an AI fails loudly, you notice. It gives gibberish. You lose trust. You call a human. That is a safe failure.
The danger lies in the quiet failure. The answer is smooth. Confident. Flawlessly articulated. But wrong. You cannot catch it. So you keep DIYing your life while it burns down around you.
With money, this silent failure is the default.
Fluency Masking Errors
Financial advice is treacherous for three reasons.
First, fluency looks like accuracy. If the bot speaks in clear sentences, your brain assumes it understands your context. It doesn’t. A bot can explain a Roth IRA perfectly but miss that your spouse’s medical bills change your tax bracket entirely. It never asks the right questions because it doesn’t know them exist.
Second, AI gets dumbest when it matters most. These tools are great at rote memorization. What is a bond? How does compound interest work? Easy.
Financial life, however, is defined by rare, complex events.
- Exercising stock options
- Alternative minimum tax traps
- Social Security strategies for couples
- Divorce settlements
Market crashes are rare. Therefore, AI has little data to learn from. It ends up most confident where it is most ignorant. We saw this on Wall Street years ago. Traders now warn that AI bots are creating new systemic risks because they optimize for the norm and panic during anomalies. The same applies to your 401(k). Researchers call this a “jagged frontier” of competence. Reliable for the mundane. Useless for the unique. And the unique cases are always the expensive ones.
Third, you can’t check the work immediately. Economics calls this a credence good. Like a doctor’s prognosis. You won’t know if the advice was bad for years. A tax mistake sits until an audit. A retirement withdrawal strategy hurts only when the market tanks later. Without immediate feedback loops, errors compound silently.
Who Loses the Most?
The real tragedy in Suzy’s story isn’t one big blunder. It’s the absence of a conversation. The bot gave her enough confidence to skip calling an expert.
The smoother the tool, the longer you stay in danger.
Who is at risk? A study of a robo-advisory platform in India by my co-author Vishaal Bhulai and me showed a clear pattern. Users skewed young, male, and were smaller retail investors. They signed up in droves when markets got volatile.
Exactly when a wrong move hurts most, the amateurs reach for automation.
There is also a profit motive here. If a platform makes money by holding your attention, it rewards confidence. Confidence keeps you scrolling. It keeps you clicking.
The system isn’t tuned to save your financial future. It is tuned to keep you engaged.
This tension is causing friction in the industry. Wealth management stocks recently slid after reports suggested AI tax tools would cannibalize human planners. The “chatbot reckoning” Bloomberg described is underway. But for the end user, the disruption looks like a slow bleed.
Where to Draw the Line
None of this means AI is evil. It’s a great free tutor. It can explain concepts. It can draft emails. It teaches you the vocabulary to sound smarter when you talk to a professional.
Just don’t let it make decisions for you.
The skill is knowing the boundary.
- Use it for definitions.
- Use it to prepare questions for your advisor.
- Do not use it for high-stakes moves.
If a decision involves large sums, irreversible changes, or complex tax consequences, the bot is likely guessing. Estate planning. Retirement drawdowns. Business structures. These are jagged peaks where the map is blank.
Pick up the phone.
When the answer feels too perfect, be suspicious. Smoothness is the warning light, not the green light. That polished confidence is exactly when you need to walk away and find a human being.
Because someone has to be accountable when the math is wrong.
