Health systems have spent years rolling out AI tools that draft notes, summarize charts, and answer patient messages. The next wave is different: AI agents that can take actions on their own, placing orders, flagging results, closing gaps in documentation, without a person approving each step, prompting several CIOs to tell Becker’s the industry isn’t ready to manage this emerging AI agent oversight healthcare CIOs challenge.
Why AI Agent Oversight Healthcare CIOs Are Comparing This to Managing New Hires
“If we’re going to build agents in a healthcare environment, and we’ve got agents crawling across our workforce, we’ve got to manage them. We’ve got to monitor them. We’ve got to make sure that they’re not drifting and that they’re doing what they’re intended to do,” said Eric Neil, CIO of Seattle-based UW Medicine. Neil compared it to managing a new employee: hiring, training, and then having a manager monitor performance daily, asking how health systems will do the same with AI agents doing work in their place.
Why Oversight Can’t Be an Afterthought
That oversight will likely be some mix of human and automated, Neil said, but either way it can’t be an afterthought. “If you don’t get that in place up front, it’s going to be incredibly difficult to come back in later when you have agents crawling across your environment, automating things and assisting people,” he said.
How Epic’s Agent Factory Platform Shapes This AI Agent Oversight Healthcare CIOs Debate
Right now, access to actually build agents is tightly controlled. Epic’s Agent Factory platform lets health systems take an existing agent, shape it to their own organization, or build a new one entirely, but only a small number of health systems are doing that work today alongside Epic’s own team. “Everybody who’s building agents today is building them with Epic, because everybody’s learning how to do this,” said Rebecca Mishuris, MD, chief medical information officer and vice president of Mass General Brigham, whose system was among the first to build agents through Epic’s early pilot, Factory One, developing ones focused on radiology actionable findings and emergency medicine.
A Narrow Window to Get Governance Right
Her colleague, Eric Podradchik, vice president of digital clinical systems at Mass General Brigham, said tight control needs to hold as access opens more broadly. “Agentic AI is the most exciting thing, and not surprisingly, also the biggest risk,” he said. “We have to build governance around it before we expand.” Broader building access is expected to open to more health systems by the end of the year, a timeline Epic confirmed at its keynote, which Podradchik said leaves a narrow window to get the guardrails right.
A More Optimistic View Within This AI Agent Oversight Healthcare CIOs Discussion
Not every CIO frames it as an unsolved problem. Ryan Smith, chief digital and information officer of Intermountain Health, described Epic’s approach as building governance into the system by design rather than bolting it on afterward, saying Agent Factory “gives organizations a governed way to build their own agents inside the system, addressing any number of potential use cases.”
A Broader Theme in How CIOs View Epic’s AI Expansion
The concern echoes a broader theme in how CIOs are reacting to Epic’s rapid AI expansion this year. Aaron Miri, chief digital and information officer of Baptist Health, has said getting the “sophistication and maturity” to monitor new capabilities matters more than how fast they roll out. For AI agents specifically, several CIOs said that maturity doesn’t fully exist yet, for Epic’s tools or anyone else’s.
Why Scale Makes Agent Oversight Harder Than Monitoring a Single AI Tool
Part of what makes agent oversight harder than monitoring a single AI tool, Neil said, is scale: a chatbot that gives a wrong answer is one bad interaction, but an agent empowered to take actions across thousands of patient charts can compound a mistake before anyone notices.
Why Cost Tracking Compounds This Oversight Challenge
Tracking that activity also means tracking cost, since duplicate or malfunctioning agents can quietly run up token bills as easily as they can cause clinical errors, a concern that dovetails with the budgeting questions several CIOs have raised about Epic’s consumption-based AI pricing.
What This AI Agent Oversight Healthcare CIOs Debate Means Going Forward
With Epic’s broader Agent Factory access expected to open to more health systems by year-end, the industry’s answer to who’s minding the machines is still being built alongside the machines themselves, leaving health systems a compressed timeline to establish monitoring platforms before agentic AI scales significantly across clinical environments. Given Neil’s warning that retrofitting oversight after deployment will prove far harder than building it in from the start, health system CIOs still in the planning stages of agent adoption may want to prioritize monitoring infrastructure investment before expanding their own agent-building activities.
What to Watch Going Forward
As Epic’s Agent Factory access broadens toward the end of the year, industry observers will likely watch whether health systems beyond early pilots like Mass General Brigham’s Factory One program can establish adequate governance structures in time, or whether the “maturity gap” several CIOs described persists as agentic AI scales more widely. Given the dual concerns around clinical drift and token cost overruns, this AI agent oversight healthcare CIOs debate may increasingly shape how health systems structure both the technical monitoring platforms and the governance committees needed to responsibly manage AI agents operating with growing autonomy across patient care workflows.
For more healthcare industry updates, insights and news, visit DistilINFO. Click here to subscribe to stay informed.
