AI's Most Expensive Fails: The Chatbot That Invented a Policy and the Company Had to Pay | inicIA
Direct answer
Two fails every business owner should study before putting a chatbot in front of customers. In February 2024, British Columbia's Civil Resolution Tribunal held Air Canada liable for "negligent misrepresentation": its chatbot had told passenger Jake Moffatt he could buy his ticket at full fare and claim the bereavement discount retroactively within 90 days —a policy that did not exist. The airline argued the chatbot was "a separate legal entity responsible for its own actions"; tribunal member Christopher Rivers called the argument "a remarkable submission" and ordered CA$812.02 in payment. A month earlier, in January 2024, the chatbot of British parcel firm DPD insulted a customer and wrote a poem calling DPD "useless" and "the worst delivery firm in the world" after a system update; the company immediately disabled the AI element. The lesson connecting both cases: what your chatbot says, you say.
The Air Canada case: it invented a policy and the tribunal collected
In November 2022, Jake Moffatt lost his grandmother and asked Air Canada's website chatbot about the bereavement fare. The bot told him he could buy at full price and claim the discount retroactively within 90 days of the ticket's issuance. The real policy required requesting the fare BEFORE flying. Moffatt paid CA$1,630 for his tickets and claimed the CA$880 difference. Air Canada refused, and before the tribunal maintained the chatbot was responsible for its own statements. The ruling (Moffatt v. Air Canada, 2024 BCCRT 149, February 14, 2024): Air Canada is responsible for all information on its website —chatbot included— and owed CA$812.02 for negligent misrepresentation. It is one of the world's first rulings holding a company liable for what its AI invents.
The DPD case: the bot that insulted its own company
On January 18, 2024, Ashley Beauchamp —a 30-year-old musician from London— could not track his parcel with DPD's chatbot and decided to test it: he asked it to swear and to write a poem about the company. The bot complied with both: it dropped an "F*ck yeah!" and wrote that DPD was "useless" and "the worst delivery firm in the world". Beauchamp posted the screenshots on X and the post went viral —between 1.6 and 2 million views and more than 23,000 likes. DPD responded: "An error occurred after a system update yesterday. The AI element was immediately disabled and is currently being updated."
3 rules so your agent never costs you
- The bot is the company. Case law has established it: what your agent invents, you sign. No "AI can make mistakes" disclaimer protects you if the bot promises prices, policies or deadlines.
- Ground answers in your real data. A bot that invents policies is a bot without access to your documents. The technique is called RAG: the agent answers by consulting your verified policies, prices and manuals, not its memory.
- Guardrails before launch. Output filters, adversarial testing (trying to break it before your customers do) and human review for consequential answers: prices, promises, deadlines and sensitive data.
Context: a study by the startup Vectara, cited in the DPD coverage, estimates that popular chatbots "hallucinate" —invent information with total confidence— between 3% and 27% of the time. Without guardrails, it is only a matter of time.
Related articles
- What to Do When Your AI Agent Gets It Wrong
- Privacy and Data: What Happens to Your Customers' Information
- Florida Wants Criminal Liability for Chatbots That Help Commit Crimes
Frequently asked questions
Can a company really be held liable for what its chatbot says?
Yes: Moffatt v. Air Canada (2024) established that a company is responsible for the information its chatbot delivers on its website, under negligent misrepresentation. The "the AI said it, not us" argument was rejected by the tribunal.
How do I stop my bot from inventing policies or prices?
By connecting it to your real documents (RAG) so it answers from verified sources, with output filters blocking answers on sensitive topics, and adversarial testing before launch.
Is a disclaimer that "AI can make mistakes" enough?
Not as legal or commercial protection. The notice helps manage expectations, but if the bot promises a price or a policy, the customer has grounds to demand it —as the Air Canada case showed.
What is RAG?
Retrieval-Augmented Generation: instead of answering from memory, the agent first consults your documents (policies, prices, manuals) and answers based on them. It is the difference between a bot that invents and one that cites.
Take the next step
An AI agent without guardrails is a risk with a logo. If you want an agent grounded in your real data, with filters and testing before going live, talk to inicIA on WhatsApp: https://wa.me/17866120930
By inicIA
