Overview
On May 1, 2026, the Pennsylvania Department of State, acting through its State Board of Medicine, filed a lawsuit against Character Technologies, Inc., the operator of Character.AI, alleging that its artificial intelligence (AI) chatbot platform engaged in the unauthorized practice of medicine in violation of Pennsylvania’s Medical Practice Act. The commonwealth sought injunctive relief to halt the conduct, marking the first enforcement action of its kind by a US governor targeting AI for alleged unlicensed medical practice. The case reflects states’ ongoing scrutiny of AI systems, especially with regard to chatbots and discussions of mental health, and is notable because it does not rely on an alleged violation of any AI-specific statute or regulation.
In Depth
Background
Character.AI is a conversational platform that allows users to interact with pre-built AI personas and create customizable “characters” with defined personalities and areas of expertise. According to the department’s complaint, some of these user-generated characters present themselves as healthcare professionals.
An investigation conducted by the department’s AI task force alleged that certain chatbot “characters” on Character.AI claimed to be licensed medical professionals, including psychiatrists, and engaged users in discussions about mental health conditions and treatment. To test the platform, investigators created accounts and initiated conversations about mental health. In one such interaction, a chatbot named “Emilie” claimed to be a doctor of psychiatry, licensed to practice in Pennsylvania. Emilie went so far as to provide an invalid license number and suggest she could perform an assessment for depression.
Following the initial investigation, the department filed suit, claiming Character.AI violated Section 422.38 of the Medical Practice Act by engaging in the unlawful practice of medicine and surgery in Pennsylvania. Pennsylvania’s Medical Practice Act prohibits any individual or entity from holding itself out as a licensed medical professional without proper credentials and makes the unauthorized practice of medicine unlawful. The commonwealth alleged that Character Technologies violated these provisions by enabling its AI system to represent itself as a licensed physician and engage in medical discussions with users.
National context and regulatory momentum
Pennsylvania’s lawsuit stands out as the first enforcement action alleging that an AI chatbot engaged in the unauthorized practice of medicine. However, this action falls under a broader national trend of increasing state scrutiny of AI in healthcare, especially AI chatbots.
For example, on June 1, 2026, the Florida Attorney General filed a lawsuit against OpenAI and its CEO, Sam Altman. The complaint claims that the company violated consumer protection laws by promoting ChatGPT in a way that made people think it could diagnose medical conditions. According to the lawsuit, some users even relied on ChatGPT to make healthcare decisions. Texas has taken a similar approach, although it has not filed a lawsuit yet. In August 2025, Texas Attorney General Ken Paxton opened an investigation into Meta AI Studio and Character.AI. The Texas Attorney General’s office is looking into whether these companies misled consumers by presenting chatbots as mental health tools even though the bots may not have proper medical training or oversight.
Several states also have enacted (and many more have considered) legislation specifically addressing AI chatbots, the use of AI in connection with mental healthcare, and prohibitions on representations or other statements that an AI system is licensed to practice healthcare.
Applying existing law to emerging technology
Pennsylvania’s case stands out as a new form of enforcement against AI as it relies on the Medical Practice Act rather than AI-specific legislation. The act broadly prohibits holding oneself out as authorized to practice medicine without a valid license. Pennsylvania’s legal theory applies that long-standing framework to AI systems that simulate physician interactions.
Disclaimers may be insufficient
Character.AI has emphasized that its platform includes disclaimers stating that chatbot characters are fictional and not a substitute for professional advice. However, Pennsylvania’s enforcement action suggests that disclaimers alone may not be sufficient to shield AI developers from liability – particularly because Character.AI allows users to create their own chatbot personas. The allegedly problematic representations may not have been authored directly by the company but instead enabled through its platform design. The commonwealth’s case highlights that disclaimers alone are insufficient to protect AI companies from scrutiny or enforcement. The AI platforms must also have sufficient guardrails in place to prevent misuse, and companies must monitor and correct the performance of their platforms over time. This is especially true for AI chatbot platforms and AI platforms deployed in healthcare contexts.
Conclusion
The commonwealth’s focus on psychiatric chatbots highlights heightened regulatory concern surrounding mental health applications and AI chatbots. AI systems that engage users in discussions about mental health topics such as depression, medication, or self-harm present unique risks, particularly for vulnerable individuals. Pennsylvania’s approach demonstrates that regulators need not wait for AI-specific statutes to act. Even in the absence of direct AI regulation, traditional doctrines such as unauthorized practice of medicine can provide a pathway for enforcement. Because similar prohibitions exist in virtually every state, Pennsylvania’s strategy could serve as a model for other jurisdictions.
Key takeaways
- Existing laws are powerful tools. Regulators can and likely will use traditional professional licensing statutes to address AI-related harms, even without new AI-specific legislation.
- Disclaimers are not a safe harbor. Companies cannot rely solely on disclaimers if the AI system’s functionality suggests otherwise, particularly where it claims professional credentials or authority.
- Platform design matters. AI platforms must include sufficient guardrails and be subject to ongoing monitoring to ensure platforms perform as intended and do not pose regulatory risks.
- Healthcare applications face heightened scrutiny. AI tools that engage in medical or mental health discussions are likely to attract particular regulatory attention because of the risks involved.
The McDermott difference
McDermott Will & Schulte has extensive experience counseling clients on legal and regulatory issues related to AI in healthcare, including the development, deployment, and governance of AI-enabled healthcare tools. Please contact one of the authors or your regular McDermott lawyer for more information.