Artificial intelligence is transforming healthcare at an unprecedented pace. Hospitals use AI scribes to document patient visits, while insurers increasingly rely on algorithms to automate prior authorization decisions. However, regulators have struggled to keep up with these rapid changes.
A recent report from the Bipartisan Policy Center highlights major gaps in health AI oversight. Many of today’s most widely used AI tools operate outside traditional regulatory frameworks. As a result, providers, patients, and healthcare executives face uncertainty about safety, transparency, and accountability.
Why Most Health AI Tools Escape Oversight
Administrative AI Dominates Healthcare
Many people assume that the U.S. Food and Drug Administration (FDA) regulates most healthcare AI systems. In reality, that is not the case.
Today’s most common AI applications are administrative tools rather than clinical devices. Examples include:
- Ambient AI scribes that record and summarize physician-patient conversations
- AI-powered prior authorization systems used by insurers
- Automated claims review systems
- Revenue cycle management tools
Because these applications do not always qualify as medical devices, they may avoid FDA review altogether. Consequently, patients often use or interact with AI systems without realizing they are largely unregulated.
The Rise of AI Scribes
AI scribes are becoming increasingly popular because they reduce physician documentation burdens. They can automatically transcribe patient visits and generate clinical notes.
However, experts warn that these tools can introduce errors, omit critical details, or even generate incorrect information. Furthermore, physicians often remain legally responsible for any inaccuracies that appear in patient records.
Multiple Agencies Share AI Regulation
A Fragmented Oversight Landscape
Health AI regulation in the United States is highly fragmented.
Several federal agencies oversee different parts of the AI ecosystem, including:
- FDA for medical devices
- CMS for payment and reimbursement policies
- HHS Office for Civil Rights for privacy issues
- Federal Trade Commission (FTC) for consumer protection
- Office of the National Coordinator for Health IT (ONC) for health information technology
In addition, states have their own rules. Therefore, the same AI application may face different regulations depending on where and how it is used.
Confusion for Providers and Developers
This patchwork creates uncertainty for healthcare organizations.
Developers may intentionally design products to avoid medical device classification. Meanwhile, hospitals struggle to evaluate AI products because regulatory requirements differ across agencies and jurisdictions. As a result, healthcare leaders often lack clear guidance on safety standards and liability risks.
How Regulatory Uncertainty Slows Innovation
Clinical AI Faces Greater Challenges
Ironically, regulatory ambiguity may discourage innovation in the very areas where AI could deliver the greatest benefits.
AI tools designed for:
- Early disease detection
- Clinical diagnosis
- Risk prediction
- Treatment recommendations
often face unclear approval pathways and uncertain reimbursement policies.
Consequently, companies may prioritize lower-risk administrative tools instead of developing more advanced clinical applications. Experts describe this trend as a “chilling effect” on clinical AI innovation.
AI Scribes and Prior Authorization Raise Concerns
AI in Prior Authorization
Insurers increasingly use AI to speed up prior authorization reviews and claims processing.
While automation can improve efficiency, critics worry that it may also increase denials or reduce meaningful human oversight. Physicians have expressed concerns that automated systems could generate unfair decisions and delay patient care.
Some researchers describe the situation as an “AI arms race.” Providers use AI to prepare authorization requests and appeals, while insurers deploy AI to review and evaluate them. Unfortunately, this cycle can create more paperwork instead of reducing administrative burdens.
Transparency Remains a Challenge
Transparency is another growing concern.
Earlier this year, policymakers proposed rolling back the ONC’s AI “nutrition label” policy. The initiative was designed to provide hospitals with information about AI systems embedded in electronic health records.
Healthcare leaders argue that removing these disclosures could make it harder to assess AI tools and may increase liability risks.
The Push for a National Framework
Bipartisan Support Emerges
Despite disagreements over details, policymakers from both political parties agree that the current regulatory landscape needs reform.
Many experts support a national framework that would:
- Establish consistent AI standards
- Improve transparency requirements
- Define agency responsibilities
- Protect patient privacy
- Ensure meaningful human oversight
A unified approach could reduce confusion and encourage responsible innovation across healthcare. However, lawmakers still need to determine how federal rules will interact with state regulations.
Conclusion
Health AI continues to reshape healthcare operations, from clinical documentation to insurance approvals. Nevertheless, regulation has not kept pace with innovation.
Today’s fragmented oversight system leaves many AI tools operating in gray areas. As adoption accelerates, policymakers face increasing pressure to create clearer standards that balance innovation, transparency, and patient safety.
The coming years will likely determine whether healthcare can fully realize AI’s benefits while maintaining public trust and accountability.
