An investment platform in New Zealand found that ChatGPT was answering customers' questions about its retirement fund with confident mistakes: five funds instead of seven, muddled fees, and a protective feature described as a restriction. The company found the errors by asking, and removed them by correcting the pages the answers came from.
The AI repeats whatever pages it finds
ChatGPT does not keep a fact sheet on each company. When someone asks about a platform's fees, it searches the web, reads a handful of pages and writes one smooth answer from them. If a review site still carries an old fee table, or a help article was written before the product changed, those pages go into the answer with the same confidence as the current ones. The customer reads one clean paragraph and rarely opens the pages behind it, so the old version becomes the one they believe.
This is why the platform's analytics looked healthy. Its site ranked well, and visitors who arrived from ChatGPT stayed longer than visitors from other places. Nothing in those numbers showed what people had read in the answer before they clicked.
It is the old directory problem again
Small businesses have met a version of this. A shop that moves and leaves its old address on a dozen directories sends customers to the wrong street, and the usual fix is to list every place the business appears and correct each one. The cause is the same now: old copies of your facts scattered around the web.
The AI version differs in two ways. A directory shows its wrong address on its own page, where a customer might doubt it. The AI folds the old copy into a sentence with no visible source, so there is nothing to doubt. But the web version of ChatGPT lists the pages it used, so each error can be traced to the page that caused it, which makes the cleanup more exact than guessing which directory is to blame.
What the audit looks like for a small business
Take a physiotherapy clinic that raised its prices and moved to a new street. The owner writes 30 questions a new patient would ask: what a first visit costs, where the clinic is, whether it treats sports injuries, how it compares with the clinic across town. She puts each one to ChatGPT in a private window and records the answer in a spreadsheet next to the correct one.
Two answers show the old price, and both point to a local health directory. One shows the old street and points to the clinic's own page from before the move, which nobody took down. That is three pages to fix, and finding them took about three hours: roughly an hour of asking, then the tracing.
The fixing is slower. The owner has to write to the directory, update her own page or redirect it, and then wait. The investment platform needed three months before its corrections showed up consistently in the answers. The old pages that matter most tend to be the ones no one owns: a price list from two years ago, a help article, a listing an agency created and then left. When conflicting old pages are still live, the AI can pick the wrong one, so the ones you cannot update should be redirected or deleted.
While you are there, check that your site is not blocking AI crawlers through an old robots.txt setting. If the AI cannot read your correct page, it builds its answer from everyone else's version of you.
Then run the same 30 questions again every three months. Answers change as the AI and the web change, and a fixed page can be undone by a new article someone writes about your business. The platform's follow-up audit used the same questions for exactly that reason: it showed which corrections had taken hold and which had not.