Based on feedback from doctors and nurses in emergency departments and ICUs where sepsis most commonly occurs, researchers at the Ohio State University have developed an AI tool for clinical decision-making to identify patients at risk for sepsis with a novel twist: it queries clinicians for additional information needed to make a more accurate prediction.
Dubbed SepsisLab, the new AI-based tool was built based on the dissatisfaction with an existing AI-assisted tool that only took into account data from patients’ electronic health records with no data input from treating clinicians. SepsisLab can predict a patient’s risk of sepsis within four hours, but over that time it identifies other patient data that is missing, determines how important that information is, and then presents a visual representation of how specific information will affect a patient’s final risk score.
Testing of the new system using a combination of publicly available and proprietary patient data showed that adding 8% of the recommended data improved SepsisLab’s prediction accuracy by 11%. Details of the Ohio State researchers’ work are published in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
“The existing model represents a more traditional human-AI competition paradigm, generating numerous annoying false alarms in ICUs and emergency rooms without listening to clinicians,” said senior study author Ping Zhang, PhD, associate professor of computer science and engineering and biomedical informatics at the Ohio State University.
“The idea is we need to involve AI in every intermediate step of decision-making by adopting the ‘AI-in-the-human-loop’ concept. We’re not just developing a tool—we also recruited physicians into the project. This is a real collaboration between computer scientists and clinicians to develop a human-centered system that puts the physician in the driver’s seat.”
The key feature of SepsisLab is its ability to produce a sepsis risk prediction within four hours of a patient’s arrival, with the system updating its predictions each hour as new data is entered. This approach not only addresses initial uncertainties but also quantifies the impact of missing data on risk predictions.
“When a patient first arrives, many values are missing, particularly from lab tests,” said first author Changchang Yin, a PhD student in computer science and engineering in the Zhang lab. Most AI models account for this data gap with a single assigned value—called imputation. The problem with this, Yin said, is the imputation model could include uncertainty which could then be propagated in the final prediction model.
“If the imputation model cannot accurately impute the missing value and it’s a very important value, the variable should be observed,” Yin noted. “Our active sensing algorithm aims to find such missing values and tell clinicians what additional variables they might need to observe—variables that can make the prediction model more accurate.”
A vital feature of the new tool for removing this prediction uncertainty over time is its ability to provide clinicians with actionable recommendations. Such recommendations could include lab tests that are rank-ordered based on their value in providing a more accurate risk assessment and how this assessment would change based on specific clinical treatments.
“The algorithm can select the most important variables, and the physician’s action reduces the uncertainty,” said Zhang, also a core faculty member at the Ohio State University’s Translational Data Analytics Institute. “This fundamental mathematics work is the most important technical innovation—the backbone of the research.”
In Zhang’s view, the development of human-centered AI tools is the future of medicine, but only these tools can interact with clinicians to allow them to build trust in its recommendations.
“This is not about building an AI system that can conquer the world,” he concluded. “The center of medicine is hypothesis testing and making decisions minute after minute that are not just ‘yes’ or ‘no.’ We envision a person at the center of the interaction using AI to help that human feel superhuman.”
