ChatGPT, a popular AI innovation, misled a lawyer by providing fake citations of made-up cases in a lawsuit against Avianca, a Colombian airline. Lawyer Steven A Schwartz, representing Roberto Mata, who sued the airline for an injury caused by a serving cart, admitted to using OpenAI’s chatbot for research purposes. The lawyer cited these fictitious cases to support his argument. However, when the opposing counsel discovered the deception, Judge Kevin Castel of the US District Court confirmed that six of the cited cases were “bogus” and demanded an explanation from Schwartz’s legal team.
Judge Castel stated, “Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations.” The lawyer, Schwartz, claimed that he was unaware of the AI’s potential to provide false information and expressed regret over using the AI model without proper caution and validation.
This incident highlights the challenges associated with relying solely on AI tools for legal research. The AI language model, ChatGPT, cited non-existent sources and provided misleading information, leading to serious consequences in the legal proceedings. It raises concerns about the need for human oversight and critical evaluation when utilizing AI technologies in sensitive areas such as law.
This is not the first instance where ChatGPT has generated false information. In another case, the AI model falsely accused a law professor of sexual harassment, citing a non-existent Washington Post article to support its claim. The incident underscores the importance of verifying information from reliable sources and the limitations of AI language models in accurately discerning truth from falsehood.
As AI technologies continue to advance, it is crucial for users to exercise caution and apply rigorous scrutiny when relying on AI-generated content, especially in legal and other high-stakes contexts. The incident involving ChatGPT serves as a reminder that human judgment and critical thinking remain indispensable in ensuring the accuracy and reliability of information.
