← Back to incidents

McDonald's Terminates IBM AI Drive-Through Partnership After Order Errors

Medium

McDonald's ended its AI drive-through partnership with IBM in June 2024 after viral videos showed the system making absurd order errors, including adding hundreds of unwanted items to customer orders.

Category
Agent Error
Industry
Other
Status
Resolved
Date Occurred
Feb 1, 2024
Date Reported
Jun 13, 2024
Jurisdiction
US
AI Provider
Other/Unknown
Application Type
agent
Harm Type
operational
Human Review in Place
Yes
Litigation Filed
No
speech_recognitioncustomer_servicefast_fooddrive_throughviral_failureIBMMcDonald's

Full Description

In 2021, McDonald's partnered with IBM to develop an automated order taking (AOT) system for drive-through locations, aiming to streamline operations and reduce labor costs. The AI-powered system was designed to understand customer speech, process orders, and integrate with McDonald's point-of-sale systems. The technology was initially tested at select locations with plans for broader deployment. By early 2024, viral social media videos began surfacing showing the IBM AI system making significant ordering errors. In one widely shared TikTok video, a customer attempting to order ice cream found the system repeatedly adding multiple servings of ice cream to their order despite corrections. Another viral incident showed the AI adding hundreds of dollars worth of chicken nuggets to an order. Additional reported errors included the system adding bacon to orders from customers who explicitly stated they didn't want bacon, and misinterpreting ambient conversations as additional order items. The errors appeared to stem from the AI's difficulty in accurately processing speech in the noisy drive-through environment. Background sounds from kitchen equipment, multiple people talking in vehicles, varying accents, and the compressed audio quality of drive-through speakers created challenging conditions for speech recognition. The system also struggled with context understanding, failing to recognize when customers were correcting previous statements or when conversations weren't directed at placing orders. On June 13, 2024, McDonald's announced it would end the IBM partnership and remove the AI ordering systems from the approximately 100 test locations where they had been deployed. The company stated it would continue exploring voice ordering solutions with other technology partners, indicating the decision was specific to IBM's implementation rather than abandoning AI-powered ordering entirely. McDonald's emphasized that the technology needed further development before being ready for widespread deployment. The termination of the partnership highlighted the challenges of deploying AI in high-stakes customer-facing environments where errors directly impact customer experience and operational efficiency. The viral nature of the ordering mistakes created reputational risks for both McDonald's and IBM, demonstrating how AI failures in consumer applications can quickly amplify through social media.

Root Cause

The IBM-developed automated order taking (AOT) system suffered from speech recognition errors and context misunderstanding, particularly in noisy drive-through environments with background sounds, multiple speakers, and varying accents.

Mitigation Analysis

More robust speech recognition training on drive-through specific audio environments, implementation of confidence thresholds requiring human intervention for unusual orders, and order confirmation protocols could have reduced errors. Real-time monitoring of order anomalies and automatic caps on quantity additions would have prevented viral incidents.

Lessons Learned

AI systems in customer-facing roles require extensive testing in real-world conditions before deployment. Speech recognition technology faces significant challenges in noisy environments, and robust error detection and correction mechanisms are essential for maintaining customer trust.

Sources

McDonald's to end AI drive-thru pilot with IBM
Restaurant Business · Jun 13, 2024 · news