Michael Cohen Submitted AI-Generated Fake Legal Citations via Google Bard

In December 2023, Michael Cohen, the former personal attorney to Donald Trump, submitted fabricated legal citations to his lawyer David Schwartz that were generated by Google's Bard AI system. Cohen was seeking to have his supervised release conditions modified and asked Bard to provide legal precedents supporting his motion. The AI system responded with what appeared to be legitimate case citations, complete with case names, court jurisdictions, and legal reasoning. Cohen's attorney, David Schwartz, unknowingly incorporated these AI-generated citations into a formal court filing submitted to the U.S. District Court for the Southern District of New York. The filing included three fabricated cases that did not exist in any legal database. The citations appeared professionally formatted and included detailed legal analysis that seemed credible to both Cohen and his attorney. When court staff and opposing counsel attempted to verify the citations, they discovered that none of the referenced cases existed. The fabricated cases included fictional court decisions with non-existent judges and legal precedents that had never been established. This discovery led to immediate scrutiny of the filing and questions about the source of the citations. Cohen later claimed he was unaware that AI systems could generate false information, stating he did not know that Bard could "fabricate cases." He argued that he had relied on the AI system believing it would only provide accurate legal information. His attorney also claimed ignorance about the AI-generated nature of the citations. The incident resulted in significant legal consequences for Cohen. The court imposed sanctions totaling $9,000 for submitting fabricated citations. The judge emphasized the importance of verifying all legal sources and criticized both Cohen and his attorney for failing to conduct basic due diligence. The case became a prominent example of the risks of using AI in legal practice without proper verification procedures. The incident highlighted broader concerns about AI hallucination in professional contexts, particularly in fields like law where accuracy is critical. It prompted discussions within the legal community about establishing guidelines for AI use and the need for mandatory verification of AI-generated content before submission to courts.