Heart failure is characterized by weak or damaged heart muscles and causes fluid to gradually accumulate in the patient’s lungs, legs, feet and other parts of the body. This condition is chronic and incurable, often leading to an irregular heartbeat or sudden cardiac arrest. For centuries, bloodletting and leeches were the treatment of choice, and their practice was popularized by barber surgeons in Europe, at a time when doctors rarely operated on patients.
In the 21st century, the management of heart failure has become decidedly less medieval: today, patients undergo a combination of healthy lifestyle changes, are prescribed medications, and sometimes use pacemakers. However, heart failure remains a leading cause of morbidity and death, placing a significant burden on healthcare systems worldwide.
“About half of people diagnosed with heart failure will die within five years of diagnosis,” says Tia Bergamaschi, an MIT doctoral student in the lab of professors Nina T. and Robert H. Rubin. Colin Stultz and co-first author of a new paper presenting a deep learning model for predicting heart failure. “Understanding how a patient does after being admitted to hospital is really important in allocating limited resources.”
paper, Published in Lancet Clinical Medicine By a team of researchers at MIT, Mass General Brigham, and Harvard Medical School, they share the results of the development and testing of PULSE-HF, which loosely stands for “Prediction of Changes in Left Ventricular Systolic Function from ECG of Patients with Heart Failure.” The project was carried out in the Stoltz Laboratory of Massachusetts Institute of Technology Abdul Latif Jameel Clinic for Machine Learning in Health. Developed and retrospectively tested across three different patient cohorts from Massachusetts General Hospital, Brigham and Women’s Hospital, and MIMIC-IV (a publicly available dataset), the deep learning model accurately predicts changes in left ventricular ejection fraction (LVEF), which is the percentage of blood pumped out of the heart’s left ventricle.
A healthy human heart pumps about 50 to 70 percent of the blood from the left ventricle with each beat, and anything less than that is a sign of a potential problem. “The model takes (an electrocardiogram) and produces a prediction of whether or not over the next year there will be an ejection fraction of less than 40 percent,” says Tiffany Yao, an MIT doctoral student in Stoltz’s lab, who is also co-first author of the PULSE-HF paper. “This is the most dangerous subset of heart failure.”
If PULSE-HF predicts that a patient’s ejection fraction is likely to worsen within a year, the physician may prioritize the patient for follow-up. Thus, lower-risk patients can reduce the number of hospital visits and the amount of time they spend having 10 electrodes attached to their bodies for a 12-lead ECG. The model can also be deployed in low-resource clinical settings, including physician offices in rural areas that typically do not have a cardiac sonographer performing ultrasounds on a daily basis.
“The most important thing that distinguishes PULSE-HF from other ECG methods for heart failure is that instead of detecting, it predicts,” Yao says. The paper notes that, to date, there are no other ways to predict future LVEF decline among patients with heart failure.
During the testing and validation process, researchers used a metric known as the area under the receiver operating characteristic curve (AUROC) to measure the performance of PULSE-HF. AUROC is typically used to measure a model’s ability to differentiate between classes on a scale from 0 to 1, where 0.5 represents random and 1 is perfect. PULSE-HF achieved AUROC levels ranging from 0.87 to 0.91 in all three patient groups.
It is worth noting that the researchers have also built a version of PULSE-HF for single-lead ECG devices, meaning that only one electrode must be placed on the body. While 12-lead ECGs are generally considered superior for being more comprehensive and accurate, the performance of the single-lead version of PULSE-HF was just as strong as the 12-lead version.
Despite the elegant simplicity behind the idea of PULSE-HF, like most clinical AI research, it belies its painstaking implementation. “It took years (to complete this project),” Bergamaschi recalls. “It has gone through many iterations.”
One of the biggest challenges the team faced was collecting, processing, and cleaning the ECG and echocardiogram data sets. While the model aims to predict a patient’s ejection fraction, labels for the training data were not always readily available. Just as a student learns from a textbook using an answer key, classification is crucial to helping machine learning models correctly identify patterns in data.
Clean linear text in the form of TXT files usually works best when training models. But echocardiogram files usually come in the form of PDF files, and when PDF files are converted to TXT files, the text (which is broken up by line breaks and formatting) becomes difficult for the model to read. The unpredictable nature of real-life scenarios, such as an agitated patient or a loose lead, also skewed the data. “There are a lot of artifacts that need to be cleaned,” Bergamaschi says. “It’s kind of a rabbit hole that never ends.”
While Bergamaschi and Yao acknowledge that more sophisticated methods can help filter data for better signals, there is a limit to their usefulness. “At what point do you stop?” Yao asks. “You have to think about the use case – is it easier to have this model that operates on data that’s a bit messy? Because it probably is.”
The researchers expect that the next step for PULSE-HF will be to test the model in a future study in real patients, whose future ejection fraction is unknown.
Despite the challenges inherent in bringing clinical AI tools like PULSE-HF to the finish line, including the potential risk of prolonging their PhD by another year, the students feel that the years of hard work have been worthwhile.
“I think things are partly rewarding because they’re challenging,” Bergamaschi says. “A friend said to me, ‘If you think you’re going to find your calling after you graduate, and if your calling is truly your calling, it will be there in the extra year it takes you to graduate.'” … The way we are measured as (machine learning and health) researchers is different from other machine learning researchers. Everyone in this community understands the unique challenges that exist here.”
“There’s a lot of suffering in the world,” says Yao, who joined Stoltz’s lab after a health event made her realize the importance of machine learning in health care. “Anything that tries to alleviate suffering is something I consider a valuable use of my time.”
(Tags for translation) MIT Jameel Clinic
