A Warning About Trust

While it’s well-known that ChatGPT (and AI in general) can make mistakes, as clearly stated on the website, it’s easy to become complacent.

I’ve been using ChatGPT for months, and it’s been incredibly helpful with only minor errors, none of which were serious. Over time, I developed a strong trust in it.

Recently, I was given a contract regarding a payment I was set to receive, which had tax and pension implications. I had two options to choose from. I ran the details through ChatGPT, asked a lot of questions, and approached the issue from different angles. I spent about 30 minutes reading through everything and cross-referencing ChatGPT’s responses with external sources. I even asked it to summarize official documents from the tax authority’s website.

ChatGPT told me that I could take a certain route that would save me a few hundred euros in taxes. It seemed like a routine matter with small amounts, so I was comfortable with the suggestion.

I was ready to sign the contract and tick the boxes, but something made me hesitate.

Thankfully, I decided to call a qualified financial advisor for confirmation. Without mentioning ChatGPT, I asked him to break everything down from scratch. He explained the process, which ended up with me paying a few hundred euros more in tax than ChatGPT had advised.

I then asked, “What about this option…” and shared ChatGPT’s suggestion (again, without mentioning AI).

The advisor responded, “If you did that, you’d save a few hundred euros in taxes now… but you’d forfeit your right to claim a tax-free lump sum from your pension when you retire. You’d end up paying hundreds of thousands in taxes later. It’s the worst choice you could make—I would never recommend it.”

This was a huge wake-up call for me.

Never get complacent with ChatGPT.

NEVER, EVER trust it for legal, financial, or tax advice.

Yes, LLMs can provide different answers based on how you phrase your question, including punctuation usage. They aren’t reliable and likely won’t be for several years.

This is sound advice for not overly relying on either ChatGPT or a single lawyer, as lawyers can make significant mistakes as well. It’s wise to seek multiple opinions if the stakes are high enough to warrant the time and expense.

It’s not quite that forward-thinking yet; the expertise of a qualified professional still makes the difference, whether that’s fortunate or unfortunate today.

ChatGPT suggested I ask my doctor for a thyroid test, which led to the discovery that I have thyroid cancer. So, is it a good thing that ChatGPT played a role in finding it, or is it bad that it wasn’t caught earlier? Deciding whether something is good or bad often depends on how you choose to frame your perspective.