Researchers at NYU Langone Health developed NYUTron, a large language model (LLM) that predicts a patient’s risk of readmission and other clinical outcomes. The code for NYUTron is available on GitHub, allowing other healthcare organizations to train their own LLMs. The collaboration with NVIDIA enabled the model’s development and fine-tuning. NYUTron outperformed physicians in predicting readmission risk. Licensing opportunities are being explored, and future clinical trials will assess the impact of interventions based on NYUTron’s analyses. This advancement has the potential to improve patient care, resource allocation, and reduce readmission rates.
In a groundbreaking development, researchers at New York University’s Langone Health academic medical center have created a powerful large language model (LLM) capable of predicting a patient’s risk of 30-day readmission and other clinical outcomes. This LLM, known as NYUTron, has been deployed at three of NYU Langone Health’s hospitals, revolutionizing the way doctors can identify patients in need of intervention to reduce readmissions. The significance of this achievement lies not only in its potential to improve patient care but also in the fact that the NYUTron code base has been made available on GitHub. This release allows other healthcare organizations to train their own LLMs and equip their physicians with valuable insights derived from patient data.
The Role of NVIDIA in NYUTron’s Development (240 words): NYU partnered with NVIDIA, a leading technology company, to develop and run the LLM using NVIDIA’s artificial intelligence platforms. This collaboration has facilitated the successful implementation of NYUTron, enabling the fine-tuning of the model with local data from specific hospitals. Dr. Eric Oermann, a key member of the NYU Langone Health team, emphasizes the accessibility of this approach, as even hospitals with limited resources can adopt a pretrained model like NYUTron and enhance it with their own data using GPUs in the cloud. By leveraging NVIDIA’s technology, NYUTron’s accuracy has been significantly boosted through the fine-tuning process.
The Training and Capabilities of NYUTron (660 words): NYUTron was trained on an extensive dataset comprising over four billion words of clinical notes from approximately 400,000 patients within NYU Langone Health’s system. The team at NYU Langone Health employed medium-sized models trained on refined healthcare-specific data to accomplish various tasks beyond readmission prediction. Alongside predicting readmission risk, the LLM was utilized to forecast the length of a patient’s hospital stay, estimate the likelihood of in-hospital mortality, and assess the chances of insurance claim denials. Through a webinar, the researchers outlined their approach of treating readmission prediction as a natural language processing task, utilizing high-end multi-node GPU servers to create the LLM.
The Evaluation and Performance of NYUTron (500 words): NYUTron’s predictive capabilities were evaluated against the performance of a group of physicians, and the results were remarkable. In predicting 30-day readmissions, NYUTron was found to be competitive with a small group of physicians, outperforming the median physician in terms of true positive rate and F1 score. The LLM demonstrated a true positive rate of 81.82% compared to 50% for physicians, while its median F1 score reached 77.8% in contrast to physicians’ median F1 score of 62.8%. These findings highlight the potential of AI-driven models like NYUTron to augment the predictive abilities of healthcare professionals and improve patient outcomes.
The Potential Impact and Future Directions (570 words): The release of NYUTron’s code base on GitHub empowers other healthcare organizations to train their own LLMs and adapt them to their specific needs. This democratization of AI-driven predictive models can help physicians gain valuable insights and optimize care delivery. Moreover, NYU Langone Health is exploring the possibility of licensing their models to healthcare providers lacking the resources to build such models from scratch. The next phase for NYU Langone Health’s team involves conducting a clinical trial to assess whether interventions based on NYUTron’s analyses can effectively reduce readmission rates. By leveraging the capabilities of the LLM, healthcare providers can potentially mitigate readmissions, improve resource allocation, and enhance patient care.
