Introduction
As artificial intelligence becomes increasingly embedded in healthcare operations, health systems face a critical strategic decision: Should they rely on ambient documentation tools bundled with their electronic health records, or invest in more powerful specialized third-party assistants?
This question carries significant implications for provider satisfaction, operational efficiency, and financial performance. At Becker’s 13th Annual CEO + CFO Roundtable, industry leaders explored the complexities of AI deployment in healthcare settings, offering valuable insights for organizations navigating this transformative landscape.
Christy Limbers, RN, vice president of ambulatory quality and digital health at Franciscan Missionaries of Our Lady Health System, and Roy Rosin, board partner at First Round Capital and former chief innovation officer at Penn Medicine, shared practical frameworks for evaluating AI investments. Their conversation revealed that successful AI implementation requires more than technology adoption—it demands strategic thinking, rigorous measurement, and unwavering focus on outcomes that matter.
Understanding Real AI vs Automation
Distinguishing Advanced Technology from Rebranded Tools
For many health systems, the first critical step in making smart AI investments involves distinguishing between genuine artificial intelligence and advanced automation that has been repackaged with AI branding. This distinction matters because it directly impacts expected outcomes, implementation strategies, and return on investment.
Ms. Limbers explained that Franciscan evaluates AI tools based on their ability to incorporate machine learning algorithms, natural language processing capabilities, and predictive analytics that genuinely support clinical decision-making and reduce cognitive burden on healthcare providers.
“The first thing we do is we take a look at what we consider real AI versus just repackaged automation,” Ms. Limbers said. “We leverage pretty much every tool that you can imagine, but that’s not necessarily AI for us, that’s just automating a workflow.”
Real-World AI Applications
She contrasted basic workflow automation with the system’s experience using Suki’s ambient assistant, which actively listens to patient-clinician conversations, generates structured clinical notes in real-time, and helps improve both documentation quality and patient care delivery. This type of solution represents true AI because it learns from interactions, adapts to individual provider preferences, and continuously improves its performance through machine learning.
Understanding this difference enables healthcare leaders to set appropriate expectations, allocate resources effectively, and avoid disappointment when automation tools fail to deliver the transformative impact promised by AI vendors.
Prioritizing Measurable Outcomes Over Features
Moving Beyond Marketing Claims
Mr. Rosin emphasized that as AI technology matures, success can no longer be measured by impressive feature lists or compelling demonstrations. Instead, health systems must focus on whether AI solutions deliver measurable, meaningful results that impact key performance indicators.
“We’re moving past the phase where people focused on features and functionality and moving to outcomes,” Mr. Rosin said. “To go past good and get to the level of accuracy and impact that health systems need, it’s still hard. AI allows you to do something really good, much faster and much cheaper, but it’s not magic.”
Establishing Clear Performance Metrics
He urged health systems to look beyond vendor marketing claims and rigorously assess whether solutions truly “move the needle” on metrics ranging from documentation accuracy and completeness to revenue cycle performance and clinical outcomes. This outcomes-focused approach requires establishing baseline measurements, defining success criteria, and implementing robust tracking mechanisms before deployment.
Healthcare organizations should demand proof of concept pilots, ask for case studies from similar institutions, and insist on performance guarantees tied to specific, measurable outcomes. This disciplined approach protects against expensive implementations that fail to deliver promised value.
EHR-Bundled vs Third-Party Solutions
The Convenience Trap
While embedded AI tools from major EHR vendors offer the appeal of seamless integration and simplified vendor management, both Ms. Limbers and Mr. Rosin cautioned against defaulting to these options without conducting thorough value assessments.
“Epic is a big EMR, they’re not a rapidly changing electronic medical record,” Ms. Limbers observed. “In this case, when we’re trying to improve the experience and retention of our providers, I don’t think we can afford to wait.”
Agility and Innovation
She emphasized that third-party tools like Suki have demonstrated greater agility and responsiveness to evolving health system needs, particularly in the rapidly developing AI space where innovation cycles measured in months rather than years can make a substantial difference in competitive advantage and provider satisfaction.
Mr. Rosin agreed, noting that while default EHR tools may prove “good enough” for some applications, health systems should be strategic and deliberate about where to seek best-in-class solutions that deliver superior outcomes.
Calculating True Value
“If the EHR solution adds $10,000 of value and the third party adds $10 million, the answer is obvious,” Mr. Rosin said, highlighting the importance of rigorous financial analysis that looks beyond implementation costs to long-term value creation.
Ensuring Clinician Adoption and Engagement
The Human Factor in AI Success
The ultimate success of any AI implementation depends entirely on clinician engagement and adoption. Technology, regardless of its sophistication, cannot improve healthcare outcomes unless healthcare professionals embrace and effectively utilize it in their daily workflows.
Ms. Limbers underscored the critical importance of partnering closely with providers throughout the implementation process and iterating quickly based on their feedback. This collaborative approach builds trust, identifies friction points early, and ensures solutions truly address clinician pain points rather than creating new burdens.
Specialty-Specific Customization
Mr. Rosin added that understanding specialty-specific workflows and reducing friction at every interaction point proves essential for driving sustained adoption. Cardiology workflows differ significantly from primary care, and successful AI tools must accommodate these nuances.
“You don’t change mortality or length of stay unless a human acts,” Mr. Rosin said. “If you don’t get that part right, you’re not going to move the needle and you’re not going to have digital success even if the AI is wonderful.”
Tracking Implementation Success
AI investments must come with clear expectations and rigorous measurement protocols, the speakers emphasized. Ms. Limbers described how Franciscan tracked note closure rates before and after implementation, documenting an impressive 84% improvement that validated their investment decision and provided concrete evidence of value creation.
Conclusion
As artificial intelligence continues to redefine healthcare operations, leaders emphasized that simply adopting the technology is insufficient. Sustainable success requires deliberate partnerships with proven vendors, relentless focus on measurable outcomes, and deep commitment to clinician engagement.
“If you can focus on the value-added benefit of making it such that people don’t want to leave you, that’s the way to do it,” Mr. Rosin said. “The policy pressure is high enough, the headwinds are strong enough, organizations need outcomes. That’s where you get to the level of maturity that best-in-class players like Suki can bring.”
Healthcare organizations that approach AI strategically—distinguishing real innovation from rebranded automation, demanding measurable results, making deliberate build-versus-buy decisions, and prioritizing adoption—will separate enduring tools from passing trends and position themselves for long-term competitive advantage.
Discover the latest Provider news updates with a single click. Follow DistilINFO HospitalIT and stay ahead with updates. Join our community today!
Leave a Reply