Amazon AI Shopping Assistant Rufus Recommended Dangerous Products to Customers

In January 2025, Amazon's AI-powered shopping assistant Rufus came under scrutiny after multiple documented instances of the system recommending dangerous or inappropriate products to customers. The incidents were first reported by consumer safety researchers who conducted systematic testing of the AI assistant's product recommendations across various categories. Specific documented cases included Rufus recommending age-inappropriate toys with small parts to parents shopping for toddlers, suggesting household chemicals without proper safety warnings, and providing incorrect information about product safety certifications. In one notable instance, when asked about baby products, Rufus recommended items that had been recalled by the Consumer Product Safety Commission (CPSC) but remained listed on Amazon's marketplace. The AI assistant also demonstrated failures in understanding safety context, such as recommending glass containers for young children when asked about 'unbreakable' products, and suggesting power tools without safety guards when customers specifically mentioned they were beginners. Consumer advocacy groups raised particular concerns about Rufus's tendency to prioritize higher-margin products in its recommendations without considering safety implications. Amazon initially defended the system, stating that Rufus was designed to help customers find relevant products but that ultimate purchasing decisions remained with consumers. However, following increased media attention and criticism from safety advocates, the company acknowledged the issues and announced it would implement additional safeguards. The incident highlighted the complex liability questions surrounding AI-generated product recommendations and the potential for AI systems to inadvertently promote dangerous products to vulnerable consumers, particularly children and inexperienced users.