Health systems have spent years running AI pilots. The question now is what comes after scaling them, and four leaders working at organizations ranging from tens of thousands to 5 million patients say the answer to health systems AI at scale requires something most organizations haven’t yet built.
Why Mass General Brigham Measured Burnout for Nearly Four Years Within This Health Systems AI at Scale Question
Mass General Brigham has been measuring the impact of ambient AI on clinician well-being for nearly four years, and Rebecca Mishuris, MD, vice president and chief health information officer, said the results changed how the organization thinks about proof. “When we started this journey almost four years ago, there was no data,” Mishuris said. The system started with 10 physicians testing the technology to confirm it was safe and producing accurate documentation before moving to a broader pilot focused specifically on clinician well-being rather than operational efficiency.
How Mass General Brigham’s CFO Reframed the Goal
“I have a very enlightened CFO who said, ‘I don’t care about the finances. There are real outcomes here,'” Mishuris said, describing how the system measured clinician well-being and professional fulfillment at the start, six weeks, 12 weeks, and within a year using national standard measures. The results showed nearly a 20% reduction in clinician burnout, a figure Mishuris said has no parallel in the broader burnout literature, and the technology “sold itself” rather than requiring active promotion. With that result now proven and sustained, the organization has shifted its measurement focus toward whether patients themselves are experiencing better care.
Why UPMC Prioritized a Unified Data Foundation Within This Health Systems AI at Scale Effort
At UPMC, the prerequisite for scaling AI is different: a unified data foundation. CIO Chris Carmody said the system just completed a milestone that makes the next phase of AI work possible. “We spent the entire weekend converting about 1.4 million patient records,” Carmody said, describing UPMC’s recent Epic go-live. “Our goal is really for the technology to fade into the background, so our doctors and nurses can actually turn their attention to our patients and provide better care.”
UPMC’s 2027 AI Agent Target and Cost Risk Warning
With approximately 120,000 technology users across the system, UPMC has set a concrete internal target for every IT professional to use AI agents to do their job more efficiently by the end of 2027. One early example is prescription refill processing, where natural language processing now handles several million refill requests per year that previously required manual phone-based verification. Carmody cautioned that uncertain token pricing and deploying AI broadly without governance structures can lead to runaway spending: “It can get out of hand very quickly if you open up these tools without structure,” he said.
Why Permanente Federation Starts With Scale Itself Within This Health Systems AI at Scale Approach
For Permanente Federation, scale is where the team starts. The federation serves approximately 5 million members, and Khang Nguyen, MD, chief medical officer of care navigation, said the work of scaling AI is fundamentally about patient care navigation. The federation’s care navigation engine integrates clinical understanding, operational capacity, and patient factors to match patients to appropriate care settings, processing free-text inputs and drawing on multiple signals simultaneously.
The Accuracy Benchmark Behind Permanente’s Care Navigation Tool
A study examining the tool’s performance showed approximately 90% accuracy in routing patients to the appropriate appointment type. “The notion that it is scaled is important,” Nguyen said. “We are involved in our governance structure around what to do.”
Why Stanford Emphasizes Accountability Over Availability Within This Health Systems AI at Scale Framework
Michelle Mello, JD, professor of law at Stanford Law School and professor of health policy at Stanford University School of Medicine, said the question health systems most often skip is whether a tool that works in testing will actually work in the workflow it’s deployed into, and whether anyone is measuring that after deployment. “We can’t just have a room full of experts sitting around and imagine what it would be like to be at this tool,” Mello said, describing Stanford’s process of engaging front-end users, a patient-centered governance panel, and developers directly.
Stanford’s Publicly Available Governance Framework
Stanford has made its governance framework publicly available, including templates, toolkits, and case studies, for other health systems building their own processes. This open publication of governance materials distinguishes Stanford’s approach from organizations that treat their internal AI oversight processes as proprietary, potentially accelerating governance maturity industrywide if other systems adopt or adapt Stanford’s published framework.
Why Measurement Sits at the Center of This Health Systems AI at Scale Standard
Across all four organizations, the measurement question sits at the center of what it means to scale responsibly. Mishuris framed it as a shift in what accountability actually requires: “AI adoption should proceed not at the pace of availability, but at the pace of accountability,” she said. “There is so much that we turn on; the approaches we take after we turn on to make sure it’s delivering value for patients matter.”
Why This Shared Standard Emerged Despite Different Starting Points
Despite arriving at AI scaling from four distinct starting points, clinician well-being at Mass General Brigham, data unification at UPMC, navigation accuracy at Permanente Federation, and stakeholder-driven governance at Stanford, all four organizations converged on the same underlying principle: sustained, post-deployment measurement determines whether AI at scale is actually working, not the initial decision to deploy it.
What This Health Systems AI at Scale Discussion Means Going Forward
Given Mass General Brigham’s nearly four-year measurement timeline before declaring success on clinician burnout, other health systems attempting to scale AI may need to adjust their own expectations for how long rigorous, outcome-based validation actually takes before a tool’s value can be confidently demonstrated. Given Stanford’s decision to publish its governance framework publicly, health systems earlier in their AI scaling journey may find immediate practical value in adapting Stanford’s templates and toolkits rather than building an entirely new governance structure from scratch.
What to Watch Going Forward
As these four organizations continue their respective AI scaling efforts, industry observers will likely watch whether Mass General Brigham’s shift toward measuring patient experience yields results as significant as its documented burnout reduction, and whether UPMC meets its ambitious 2027 target for universal AI agent adoption among IT staff. Given Carmody’s explicit warning about token pricing and governance risk, this health systems AI at scale discussion may increasingly center on cost discipline as much as clinical and operational outcomes as more systems move from pilot programs into enterprise-wide AI deployment.
For more healthcare industry updates, insights and news, visit DistilINFO. Click here to subscribe to stay informed
