India Confronts Critical Questions About AI Healthcare Implementation and Equity
The conversation surrounding artificial intelligence in healthcare typically centers on ambitious promises including faster diagnostic capabilities, scalable healthcare access, and precision medicine deployment at population scale. However, at the inaugural Winter Dialogue on RAISE (Responsible AI for Synergistic Excellence in Healthcare) held at Ashoka University last week, the discussion shifted decisively toward more challenging questions: who benefits from AI healthcare systems, which populations are excluded from these benefits, and how should institutions govern technologies that remain incompletely understood.
The two-day dialogue hosted by the Koita Centre for Digital Health at Ashoka University (KCDH-A) in partnership with NIMS Jaipur, with WHO SEARO as technical host alongside ICMR-NIRDHS and the Gates Foundation, served as an official Pre-Summit Event for the AI Impact Summit 2026. This gathering represented the first in a series of four national RAISE dialogues scheduled across India throughout January, with the opening edition specifically focused on the critical theme of Health AI Policy and Governance.
Persistent “Pilotitis” Prevents Digital Health Solutions From Scaling
If a unifying thread connected the various sessions, it was the substantial gap between technical capability and institutional readiness for AI healthcare deployment. Dr. Karthik Adapa, Regional Adviser for Digital Health at the World Health Organization, warned against what he characterized as the persistent problem of “pilotitis” — the troubling tendency for digital health solutions to remain trapped in experimental pilot programs without ever successfully scaling into public health systems serving broader populations.
Frameworks such as SALIENT, Dr. Adapa argued, were essential precisely because they compel practitioners to think beyond models and metrics, directing attention toward integration challenges, comprehensive evaluation methodologies, and long-term sustainable use rather than short-term proof-of-concept demonstrations. This critique highlighted how many AI healthcare initiatives achieve impressive results in controlled research settings but fail to transition into routine clinical practice where they could deliver meaningful population health impact.
The challenge of moving from pilot projects to scaled implementation reflects deeper systemic issues including inadequate digital infrastructure in rural and underserved areas, insufficient training for healthcare workers expected to use AI tools, fragmented data systems preventing interoperability, regulatory uncertainty about approval pathways for AI medical devices, and limited funding mechanisms supporting post-pilot operational costs. These barriers explain why India’s healthcare landscape features numerous successful AI demonstrations that never reach the patients who could benefit most.
Fundamental Tension Between Average Performance and Equitable Outcomes
The tension between optimization for average performance and equity across diverse populations surfaced repeatedly throughout the dialogue. In his opening remarks, Dr. Anurag Agrawal posed a provocative question that resonated across the conference halls: Would you choose an AI model with higher average accuracy but poor performance for women, or one with lower overall accuracy that demonstrates equity in outcomes across demographic groups?
His larger point crystallized in a phrase that became something of a refrain throughout the event: “AI for Health, not Healthcare for AI.” This formulation captured the fundamental concern that healthcare systems might adapt themselves to accommodate AI capabilities rather than ensuring AI tools genuinely serve patient needs and public health priorities. The risk is that technological imperatives drive healthcare transformation rather than clinical evidence and population health requirements.
This equity challenge manifests practically when AI diagnostic systems trained predominantly on data from certain populations perform poorly for underrepresented groups. For instance, imaging algorithms developed using datasets from Western populations may fail to accurately interpret scans from Indian patients with different disease presentation patterns, body compositions, or genetic backgrounds. Similarly, predictive models for maternal complications might overlook risk factors specific to rural Indian women if training data comes primarily from urban hospital settings.
Real-World Case Studies Expose Fragile Infrastructure and Social Bias
The panels following Dr. Agrawal’s opening demonstrated how complicated the translation from principle to practice genuinely is. Case studies spanning tuberculosis screening initiatives, cancer detection programs, and maternal health monitoring systems across various Indian states showed both significant promise and considerable fragility. Presenters documented fragile data pipelines susceptible to disruption, uneven digital infrastructure creating geographic disparities in AI access, regulatory uncertainty about approval requirements and liability frameworks, and deeply embedded social biases that algorithms can easily reproduce and amplify.
Tuberculosis screening programs using AI chest X-ray analysis, for example, achieved impressive sensitivity in detecting active disease but struggled with implementation challenges including inconsistent image quality from rural health facilities, lack of radiological technician training on proper positioning, intermittent internet connectivity preventing real-time AI processing, and difficulty integrating AI results into existing tuberculosis management workflows. These practical barriers meant that technical accuracy validated in research settings did not translate into operational effectiveness in routine clinical environments.
Cancer detection initiatives similarly revealed how AI performance depends critically on data quality, healthcare worker training, and system integration. AI tools identifying suspicious lesions in cervical cancer screening or analyzing mammography images required not just algorithmic sophistication but also coordinated systems ensuring appropriate follow-up for positive findings, patient education about screening benefits, and sustainable funding for screening programs. Without this broader ecosystem support, even highly accurate AI models failed to improve population health outcomes.
Mental Health Applications Demand Particularly Cautious Clinical Boundaries
Mental health discussions at the dialogue proved particularly cautious about AI applications in this sensitive domain. As Dr. Prabha Chand observed, large language models are “optimized for engagement, not clinical outcomes,” raising concerns about chatbots providing mental health support without adequate clinical oversight. Dr. Smruti Joshi reminded participants that “mental health judgment cannot be fully automated,” emphasizing the irreplaceable role of human clinical expertise in psychiatric assessment and treatment.
The challenge, several panelists argued, is not whether AI has any role in mental health care, but rather how narrowly and carefully that role should be defined — especially when working with vulnerable populations experiencing psychiatric crises or suicidal ideation. AI applications might appropriately support mental health through symptom tracking between clinical appointments, providing psychoeducational resources about mental health conditions, facilitating scheduling and appointment reminders to improve treatment adherence, and identifying patients at elevated risk who might benefit from earlier intervention.
However, critical clinical decisions including psychiatric diagnosis, medication selection and dosing, suicide risk assessment, and treatment planning require human clinical judgment that current AI systems cannot replicate. The risk of over-relying on AI mental health tools includes delayed appropriate care, misdiagnosis or inappropriate treatment recommendations, privacy breaches of sensitive mental health information, and reduced therapeutic relationships between patients and clinicians.
Continuous Validation and Accountability Must Become Standard Practice
Validation methodologies and accountability mechanisms emerged as equally central themes throughout the dialogue. Dr. Mary-Anne Hartley emphasized that imperfect data inevitably produces imperfect models, a particularly acute concern in contexts as diverse as India’s with extreme variations in disease prevalence, healthcare infrastructure, socioeconomic conditions, and population characteristics across regions.
Continuous monitoring of AI system performance after deployment, systematic bias detection and mitigation strategies, and human-in-the-loop systems maintaining clinical oversight, panelists argued, must transition from optional best practices to mandatory standard requirements for all healthcare AI applications. This represents a significant departure from traditional medical device regulation where products undergo pre-market validation but limited post-market surveillance.
Healthcare AI systems differ from conventional medical devices because their performance can drift over time as clinical practices evolve, patient populations change, or data distributions shift. An AI model validated on 2023 data may perform poorly on 2026 patients if disease patterns, treatment protocols, or demographic characteristics have changed. Continuous monitoring enables early detection of performance degradation before patient harm occurs, while human oversight ensures that inappropriate AI recommendations are caught and corrected.
Universities Must Build Intellectual Infrastructure for Responsible AI Governance
Reflecting on broader implications, Dr. Anurag Agrawal returned to the ethical foundation underlying the entire discussion: “The real test of health AI is not peak accuracy in controlled settings, but equitable performance in the real world. If AI systems work well on average but fail women or marginalized populations, we have failed the purpose. We must design AI for health—not bend healthcare to fit AI.”
That sentiment received reinforcement from Vice-Chancellor Somak Raychaudhury, who noted that “Responsible AI in health cannot be built in silos. Universities have a crucial role to play — not only in advancing research, but in creating the intellectual and institutional infrastructure needed to ensure that AI serves public good, equity, and trust at scale.” This perspective positions academic institutions as essential contributors to AI governance beyond their traditional research functions.
Universities can contribute to responsible AI healthcare governance through interdisciplinary education programs training healthcare professionals, data scientists, ethicists, and policymakers, research centers studying AI equity and fairness across diverse populations, policy labs developing governance frameworks based on empirical evidence, public engagement initiatives building societal consensus about appropriate AI roles, and partnerships with healthcare systems enabling real-world AI validation studies.
Four-City Dialogue Series Signals Shift From Hype to Responsible Implementation
RAISE, as described by Aradhita Baral, is intended as “a platform for sustained dialogue rather than isolated conversations.” The initiative’s expansion to IIT Delhi, Bengaluru, and Hyderabad over the coming weeks suggests that India’s AI-in-health debate is finally progressing from hype to homework — transitioning from focus on what is technically possible to what is clinically responsible and socially equitable.
This evolution reflects growing recognition that healthcare AI deployment requires not just technical innovation but also robust governance frameworks, continuous stakeholder engagement, empirical evidence about real-world effectiveness, and explicit attention to equity across India’s diverse populations. The RAISE dialogue series creates structured opportunities for healthcare providers, AI developers, policymakers, ethicists, and patient advocates to collectively shape how AI integrates into India’s healthcare ecosystem in ways that advance rather than undermine public health goals.
