Table of Contents
Most health systems don’t have an AI problem; they have an operationalization problem. Underneath that operationalization problem are two journeys running at once, and Stanford Health Care’s experience building ChatEHR Stanford AI operationalization practices offers a case study in navigating both.
The Two Journeys Behind ChatEHR Stanford AI Operationalization
The technology journey, the platform, the data pipelines, the evaluation infrastructure, is, relatively speaking, the tractable one. The organizational journey, changing how teams govern, adopt, and hold themselves accountable to new workflows, is the harder, slower one, and it’s the one most health systems underinvest in. The pilots are everywhere, and the proofs-of-concept are compelling, yet the vast majority of clinical AI tools never leave the lab, dying quietly as demos, grant deliverables, or “cool things sitting on someone’s computer that we can’t put anywhere,” as Lead Data Scientist Juan Banda said.
What ChatEHR Actually Does
ChatEHR is a chat interface embedded directly in the EHR that lets clinicians query structured and unstructured patient data in plain language, questions like “Has this patient ever had a positive biopsy?” or “Summarize the last 12 months of cardiology notes.” Behind that interface sit governed data pipelines built to cut through “250 novels’ worth of data” per patient, as Aditya Sharma, senior manager of data engineering at Stanford Medicine, put it.
Cross-Functional Ownership Within ChatEHR Stanford AI Operationalization
At Stanford, no single department owns ChatEHR. The core team spans data scientists, data engineers, integration experts, nursing informatics professionals, product managers, UX designers, and clinical champions, reporting up through IT, operations, nursing, and compliance. Clinical AI fails when it’s handed off between silos: data science builds it, IT integrates it, operations is expected to adopt it. Stanford instead makes clinical and operational stakeholders co-owners from the start.
Investing in the Boring Infrastructure First
If ChatEHR is the visible tip of the iceberg, the mass beneath it is data engineering, integration, and secure infrastructure, including establishing secure, governed access to clinical data at scale, designing pipelines to unify data across multiple EHR instances and external systems, and embedding the interface inside the EHR with no new portal, login, or workflow toggle. “That investment from leadership makes my job possible,” Banda said.
Evaluation and Monitoring Within This ChatEHR Stanford AI Operationalization Model
Stanford’s team measures accuracy, hallucination rates, and response times, but also the questions clinicians actually care about: Does this save time? Does it catch important details? Does it fit the workflow, or add friction? They study real-world usage, which questions get asked, at what point in the workflow, and how often outputs are accepted or overridden, and feed that back into model tuning and product design. Critically, they align with emerging clinical AI oversight frameworks such as FURM and MedHELM, treating post-deployment monitoring for performance, safety, and equity as a core part of the lifecycle rather than an afterthought.
Why Go-Live Is the Beginning, Not the End
For many health IT projects, go-live is the finish line. For AI, it’s the start of the “long, sometimes exhausting middle.” Stanford’s team names two traps worth guarding against: the “faster horse” trap, where incremental automation builds credibility quickly but leaves the team with no future if that’s all it builds for five years; and the “only ROI” trap. “Some of the most important things we’re learning don’t show up as clean ROI right away,” said Product Manager Duncan McElfresh. “They show up as new workflows, new capabilities, new ways of thinking about the enterprise.”
Keeping Humans in the Loop Within ChatEHR Stanford AI Operationalization
At Stanford, ChatEHR augments clinical judgment; it does not replace it. “You’ve got to be the human in the loop,” said Nerissa Ambers, director of health informatics transformation. “We don’t just let AI run the show here.” The tool has already assisted in multiple cancer diagnoses, a reminder that the goal is to surface what a clinician might otherwise miss, not to make the decision for them.
Why Leadership Air Cover Proved Decisive
Ask the Stanford team what made their work possible, and they don’t start with algorithms. They start with leadership: funding cross-functional teams for real, granting latitude to experiment within security guardrails, and accepting that some projects will fail. In hindsight, McElfresh wishes they’d been bolder: “Ask for forgiveness not permission more often. Be less risk-averse.”
Why the Operating Model, Not the Algorithm, Is What Transfers From ChatEHR Stanford AI Operationalization
The most important insight from Stanford’s experience is that the durable, transferable asset isn’t the AI model. It’s the organizational model. “The technology itself is only half the equation,” said Nikesh Kotecha, head of data science. “The true breakthrough is the organization we are delivering to enable it. Without that time investment, these innovations simply never reach the patient.”
How This Became a Shared Chassis
Other Stanford teams are now building on ChatEHR’s foundation, the secure data access, integration patterns, and governance processes, rather than reinventing the wheel. The flagship product has become a shared chassis other initiatives can build upon rather than replicate from scratch.
What This ChatEHR Stanford AI Operationalization Model Means Going Forward
For the rest of the industry, the choice is coming into focus: treat AI as a rolling series of pilots, or treat it as the beginning of a new kind of infrastructure. Most organizations can eventually solve the technology journey; fewer invest in the organizational one, and the systems that make that investment are the ones that turn AI from a rolling series of pilots into permanent infrastructure.
What to Watch Going Forward
As more health systems attempt to move clinical AI tools from pilot to sustained practice, industry observers will likely watch whether organizations adopt Stanford’s emphasis on cross-functional co-ownership and dedicated infrastructure investment, or continue treating AI initiatives as isolated departmental projects that struggle to scale. Given Stanford’s framing of the operating model itself as the truly transferable asset, other health IT organizations evaluating their own AI strategies may find more lasting value in studying how Stanford structured governance and accountability around ChatEHR Stanford AI operationalization than in attempting to replicate the specific technology alone.
For more healthcare industry updates, insights and news, visit DistilINFO. Click here to subscribe to stay informed.
