For all the hope that artificial intelligence (AI) brings to health care, there are perils: patient privacy concerns, public trust, legal and ethical considerations, the fear of biased algorithms, and how to handle the need for massive amounts of the right data to feed AI algorithms. We discuss all of that with experts from across the world of pediatric cardiology.
This episode was taped live during the 8th World Congress of Pediatric Cardiology and Cardiac Surgery in Washington, D.C.
Carol Vassar, producer and host
Featured Guests:
Rima Arnaout, MD, Assistant Professor in Medicine, University of California San Francisco
Devyani Chowdhury, MD, Cardiologist, Nemours Children’s Hospital, Delaware
Mark Friedberg, MD, Pediatric Cardiologist, Labatt Family Heart Centre, Division of Cardiology, Hospital for Sick Children Toronto, Canada
Babar Hasan, MD, MBBS, Professor of Cardiology Chair, Division of Cardio-thoracic Sciences
Sindh Institute of Urology and Transplantation (SIUT)
Pei-Ni Jone, MD, Medical Director of Echocardiography, Professor of Pediatrics, Pediatric Cardiology, Lurie Children’s Hospital, Northwestern University Feinberg School of Medicine
Charitha D. Reddy, MD, Clinical Assistant Professor, Pediatrics – Cardiology, Stanford Medicine Children’s Health
Additional Episodes In Our AI Series:
Artificial Intelligence in Healthcare: The Promise (Part 1 of 3)
Artificial Intelligence in Pediatric Healthcare: A Primer
Artificial Intelligence in Pediatric Healthcare: Dr. Mark Friedberg, Hospital for Sick Children, Toronto
Artificial Intelligence in Pediatric Healthcare: Dr. Charitha Reddy, Stanford Medicine
Artificial Intelligence in Pediatric Healthcare: Dr. Pei-ne Jone, the Ann & Robert H. Lurie Children’s Hospital of Chicago
Artificial Intelligence in Pediatric Healthcare with Dr. Babar Hasan, Aga Khan University, Pakistan
Artificial Intelligence in Pediatric Healthcare with Dr. Rima Arnaout, University of California San Francisco
Episode 36 Transcript
Carol Vassar, Host/Producer:
Welcome to Well Beyond Medicine, the Nemours Children’s Health Podcast. Each week, we’ll explore anything and everything related to the 80% of child health impacts that occur outside the doctor’s office. I’m your host, Carol Vassar, and now that you are here, let’s go.
Music:
Well Beyond Medicine
Dr. Mark Friedberg, Hospital for Sick Kids, Toronto:
It’s a tool, and like any other tool, we’re going to need to learn as time evolves how to use it. It will make us better physicians, just like every technological advance has made us better physicians, but ultimately, it’s going to be up to us to know how to use it, what the pitfalls are because there’s nothing in life. There’s no magic. Every tool has pitfalls.
Carol Vassar, Host/Producer:
That’s Dr. Mark Friedberg, pediatric cardiologist at the Labatt Family Heart Center in the Division of Cardiology at the Hospital for Sick Children in Toronto, with a reality check on the use of artificial intelligence in healthcare. It’s a tool or a set of tools with great promise to improve healthcare delivery across the globe. But AI isn’t without perils and pitfalls, as we’ll find out in this episode.
As AI becomes part of a broader national conversation, the public has concerns about its use in their doctor’s offices. According to a 2023 Pew Research Center study, it states that six in ten US adults would feel uncomfortable if their own healthcare provider relied on artificial intelligence to do things like diagnose disease and recommend treatments. Four in ten Americans in that same study think AI and healthcare and medicine would reduce the number of mistakes, though a majority say patient-provider relationships would suffer. And among those surveyed who see bias based on race or ethnicity as a problem in health and medicine believe AI has the potential to improve the situation. However, there is also concern among those working in and with AI that such technologies, if not built right, would exacerbate bias rather than reduce it, according to Nemours Children’s Health cardiologist Dr. Devyani Chowdhury.
Dr. Devyani Chowdhury, Nemours Children’s Health:
So AI can create bias, okay? So, that has been shown in many studies. The moment you put ethnicity of the patient in, you’ll say, “Okay, this population is at risk for this.” But then you’re already introducing a bias into that. So there was this big paper in the Lancet that just came out how AI can actually cause colonization again because the data lives in the lower middle-income countries. High-income countries want that data to develop their algorithms, but at the same time, you’re not looking at the end-user over there.
Like language, for example, is a big part of colonization. So we want to reverse that, right? So AI can help solve that problem of colonization because if you have a person who lives in Senegal and speaks their local language, AI should be able to talk that so that it doesn’t just become a proprietary of the English-speaking high-income countries. So I think the whole world is now looking at the ethics of that, right? So, for AI algorithms to work, it’s really important that there is smart, thoughtful people who actually pick the right things to put in.
Carol Vassar, Host/Producer:
Training an AI model is an important part of the development process. Training, simply put, is teaching a machine-based algorithm to perceive, interpret, and learn from data to come up with accurate decisions. Doing so is far from simple and requires data, massive amounts of carefully curated data to create an AI model that works well in a clinical setting.
To prevent bias within an AI algorithm, developers, including clinicians, data scientists, AI engineers, researchers, and computer programmers, must be diligent in choosing the right training data so as to make the algorithm generalizable and applicable to large swaths of patients, according to Charitha Reddy, clinical assistant professor of pediatric cardiology at Stanford Medicine in Palo Alto, California.
Dr. Charitha Reddy, Stanford Medicine:
You need to have a baseline dataset that you use to allow the model to learn what are the kind of insights what are the things that you’re trying to measure or replicate or do that humans already do. So, the same way that a human trainee who’s going through medical school or fellowship is learning to look at thousands and thousands of echo images, a model needs to do the same thing. They can just do it faster and speedier than a human can.
And so you want to be able to expose the model to huge amounts of data, but the truth is that looking at it right now when we train the models, we’re training them based on our perceptions. And yes, there may come a point where the models identify things that are new, but it still needs human oversight to be able to put that to the actual patient.
And there are some subtle things that we can’t quantify to be able to train a model. So, thinking about the ethical standards of how do we make sure that models are truly generalizable to a broad population? We’ve already seen that some AI models, when they’re developed in non-medical contexts, have shown racial biases just based on subconscious biases from how they’ve been trained. So how do we make sure that when we train our models that we avoid those biases by being inclusive of broad swaths of population or demographics and make sure we’re representing everybody?
How do we make sure that we test those models everywhere and then also make sure that we reach the highest ethical standards of, okay, if we’re implementing this at the bedside, who’s going to oversee that’s correct or that we’ve had good success with this? Or how do we make sure we inform patients that, “Hey, this prediction was made by a model. It is 90% correct, but here’s the 10%,” just the way that we consent anybody for a surgical procedure or anything else, how do we make sure we build in those same practices for our pediatric patients?
Dr. Babar Hasan, Aga Khan University, Pakistan:
One has to understand how these algorithms are created.
Carol Vassar, Host/Producer:
Dr. Babar Hasan, associate professor and consultant pediatric cardiologist at the Department of Pediatric and Child Health at Aga Khan University in Karachi, Pakistan.
Dr. Babar Hasan, Aga Khan University, Pakistan:
How these algorithms are trained, what data went into it? And that clarity has to happen. That’s some of the work that we have been doing around use of fetal dopplers to predict high-risk fetuses in pregnant women. And this is the work that has been done in Pakistan. And in fact, we did a very interesting comparison of a population out in a high-income country, again, around prediction of babies who will be born or fetuses who will be born being small at birth, and then we did it for the population in Pakistan. What is interesting is that the AI algorithm will behave very differently based on what data goes into it.
So I think as long as we understand that those limitations, any scientific paper has a limitation section. So that’s kind of the limitation section of work around AI. If we understand that biases can be created, just the fact that there is a computer and an algorithm that will basically decide a therapy and the algorithm is unbiased actually may not be true because you have to understand what data went into it went into it. So if we understand the limitations of what goes into it and actively address those limitations, if we bring people from the lower income country as equal partners in developing contextually related AI solutions, definitely it has a role, and it’ll definitely improve the healthcare inequity. There’s no doubt about that.
Carol Vassar, Host/Producer:
Underscoring the need for mammoth amounts of data for model training purposes and to work toward preventing AI bias is Dr. Pei-Ni Jone, director of the echocardiography laboratory at the Heart Center of the Ann & Robert H Lurie Children’s Hospital of Chicago.
Dr. Pei-Ni Jone, Ann & Robert H Lurie Children’s Hospital of Chicago:
You can have bias if you have limited dataset to train the algorithm. That’s the power of multicenter collaboration. If you have one center that’s doing the training, what this thing with edge computing is you have five centers or seven centers that are feeding their data into this edge computing, and then the AI model sort of adjusts itself to try to fit the different types of data that are coming from different center.
So, I think, in that sense, it is quite powerful to reduce the amount of bias that you could potentially have. I think the bias that we see is if you keep feeding the data the same data from one center and not validating it in another center, that becomes problematic.
Carol Vassar, Host/Producer:
But where does all this data come from? Dr. Charitha Reddy.
Dr. Charitha Reddy, Stanford Medicine:
We’re still in those early phases of figuring out how do we overcome the gathering of data. How do we make sure that we create data sets that are representative of the whole group of all of the demographics that are represented on the adult side? It’s relatively easy for a single institution to pull together, say, 10,000 echo echocardiograms. It’s not that easy on the pediatric side to do that for a few different reasons.
One is that if you think about congenital heart disease, the frequency or prevalence of congenital heart disease is low compared to the general population. And then each institution might only see a certain type of diagnosis, say 500 times in five years or something along those lines. And so in order to really get a dataset ever in the thousands range, you either need to pool data, go back multiple years, or figure out a way to really work with other institutions.
There are some technologies that allow you to augment data sets, but I think to really train a robust model, you need to be able to figure out how to share amongst institutions or figure out a way to augment those data sets such that they exist for people to do research on. But I think the areas that we also need to focus on as we develop this infrastructure are the ethical standards. Who’s going to be responsible that if an AI model identifies something incorrectly, or how do we make sure that we still have good human oversight?
I think that’s where people get concerned. Is AI going to replace the job of a human physician? As we as a whole group in pediatric cardiology move towards in every area in electrophysiology or echocardiography as we move towards it, we should all be thinking about that anytime we come up with a project thinking about AI and machine learning and deep learning is how are we making sure that the standards that we’re setting forth are equal amongst all of our organizations? How are we making sure that there are standards for AI research in pediatric populations, and really, how do we make sure that we hold each other to those standards, and how do we test these models in different populations to make sure that it is working out for everybody?
Dr. Mark Friedberg, Hospital for Sick Children, Toronto:
We lag often in medicine with the ethics behind the technology. I think in humanity in general, right?
Carol Vassar, Host/Producer:
Dr. Mark Friedberg.
Dr. Mark Friedberg, Hospital for Sick Kids, Toronto:
We invent a bomb, and then we’ve got to decide how we use it and what it is, and it’s like that. And there are examples with AI where you can either go wrong because the software is recognizing something wrong, or it can impact decision-making in and of itself. It can impact how people use that information for decision-making, for example, or the output can be different. And there was, again, another great example given with ChatGPT, how you can change an echo report completely and make it sound more young or more old, or take data from the medical chart.
So there’s a whole array of possibilities that are technologically possible that I think we as humanity are going to have to decide how we use, where we use. So, for example, and this is not for imaging, but it’s one of the things I gave in a recent talk. One of the examples, you can take data, or one study took retrospective data from just regular visits to the family physician and took a whole bunch of parameters and predicted outcomes going forward from data that they had over the last two years. So you can imagine then how that can impact people’s ability to get insurance or to travel. So we’ve got to decide how we use all of these very powerful tools. They are still tools, and it’s going to make us better physicians if we use them properly.
Carol Vassar, Host/Producer:
Swept up amongst the enormous amounts of data used to train AI models for healthcare is personal data collected from various sources. This produces challenges to protecting patient privacy, but it’s on the radar of many working with AI models, including Dr. Charitha Reddy.
Dr. Charitha Reddy, Stanford Medicine:
I think that’s at the forefront of everybody’s mind is maintaining privacy, particularly as children are vulnerable populations. So, to make sure that there’s not really a way to re-identify or make sure that we’re supporting their privacy is a big challenge. And I think the legal and ethical barriers that we need to work on are really manifold, but I think there’s a few different ways that people are working on it.
The American Society of Echo has a pediatric module for their registry where they’re working on trying to figure out how we can put together at least starting with the reports and then thinking about how we can put together the images as well. I think the ways that we have to think about this as a group: are we willing to figure out the legal portions of it first? So how do we work with our legal teams, with each of the hospitals, to ensure that the data is as private as possible?
How do we work with either industry partners or other people to make sure that we have great automated anonymization software to really get rid of that and make it as safe as possible? At the same time, how do we make sure that this is clinically valuable? If you delete all of the identifying data, which is important for patient privacy, how do you make sure that you link it back to the clinical outcomes to make it as relevant as possible when you’re developing research?
So, you need to figure out a system in which you somehow link each echocardiogram to a clinical outcome without necessarily compromising the patient data. I think we’re still in the growing phase of figuring that out. Each part has probably been figured out in slightly different areas, or there are models on the adult side or on the pediatric side that we can group from. But I think putting it together is still a work in progress.
The model exists to share data, and there’s been great work that’s been shown where multiple institutions pool their data for specific lesions like hypoplastic left heart syndrome and things like that, and they compare clinical outcomes. They compare hospital length of stays and things like that, and they’ve been able to come together to create group recommendations for institutions to follow for these patient populations to theoretically say, “Hey, we’re all across the country. We’re all following these same guidelines. Is this helping our outcomes?”
So, there is a model to do that. To do that for pediatric echo can be a challenge because in addition to sharing the clinical reports and outcomes, you actually need to share the images and the data storage and the ability to figure out where you’re going to store that kind of high amounts of data is a bit of a challenge. In addition, while there have been some legal ways to figure out how to make sure we maintain patient privacy while sharing the data, it’s easy to anonymize written reports because there’s maybe one field or two fields that can be used to identify a patient.
Sometimes, in the pediatric echocardiogram data, there’s been challenges where it’s either burned into the image itself or erasing the metadata can be a challenge. And so, maintaining patient privacy has been a barrier to really creating large pediatric echo data sets that can be shared. Because right now, one of the ways that people do it is on a lesion-by-lesion basis. And so teams come together, and they say, “Hey, we want to evaluate, say, aortic stenosis,” and then all of the hospitals come together, but they have to manually delete those images or manually delete the metadata. So it becomes a huge time sink.
Carol Vassar, Host/Producer:
To ensure patient data privacy while sharing information for the purpose of creating AI algorithms, healthcare institutions will often enter into a data usage agreement or DUA. However, these often take up to a year to put in place, which can set back development of the algorithm significantly. However, there is a decentralized data-sharing model using edge computing and federated learning that allows each institution to oversee its data, ensuring it never leaves its firewalls. Dr. Pei-Ni Jone.
Dr. Pei-Ni Jone, Ann & Robert H Lurie Children’s Hospital of Chicago:
Edge computing and federated learning is that you are in charge of the data in your center. It actually doesn’t leave your center. It just feeds modals up into the cloud. And then you could have that algorithm tested. This is like a digital twin kind of computing that I’ve just recently learned about. What it is, it will twin the ultrasound system that we have at our center and then put it into the cloud. And then so each of our centers is in control of our data.
So this new edge computing, which I’ve learned, is actually really nice right now because, at first, we had to worry about DUAs, which can take a long time. So DUA are data use agreements that can take a long time between centers, sometimes six months to a year, and that usually is the holdup. And so waiting to do that it would put you behind for another year. But right now, with this new cloud computing and edge server with digital twins cloning the ultrasound machine, our data resides with us. So it’s actually going to be an interesting, innovative path to go forward.
Carol Vassar, Host/Producer:
Occasionally, the barrier to achieving the more perfect AI model is in the nature of the non-AI technology. So says Dr. Rima Arno, assistant professor of medicine and a member of the Baker Computational Health Sciences Institute at the University of California, San Francisco.
Dr. Rima Arnaout, UCSF:
Ultrasound of all the imaging modalities, the most common imaging modality worldwide next to X-ray. But the spatial resolution then it’s a noisy image. It can be acquired freehand from somebody moving the probe around. So, the quality of the images you get can be very variable. The number of images that gets stored is very variable. The different types of ultrasound machine manufacturers, software, all of that stuff affects the variability of the data. So that’s a big challenge.
Carol Vassar, Host/Producer:
For Dr. Pei-Ni Jone and so many others in the field, the additional obstacles to incorporating AI into pediatric healthcare are twofold: community and provider education and a shortage of trained specialized workers.
Dr. Pei-Ni Jone, Ann & Robert H Lurie Children’s Hospital of Chicago:
Finding a data scientist who understands how to do echocardiography as I’m an imager by training, and so they’re good with still images right now, but they’re not very good with moving images. So, it’s very rare to find a data scientist who can actually do the moving imaging for echo. As echo is moving images of the heart. So I’ve actually met with six data scientists and discussed with them, “Could you help me with our research in developing the algorithm for view classification for recognizing normal heart and abnormal heart?”
If we could use AI to leverage that, to recognize what’s really normal what’s abnormal, then we can spend our time focusing on what’s abnormal. Because majority of time, normal echoes doesn’t take very long to read, but their tie-in’s intensive. We also have shortages of sonographers right now in the workforce, and so if we could help AI to train our sonographers who are brand new, then we don’t need to really spend six months of teaching them at the bedside how to scan.
Dr. Charitha Reddy, Stanford Medicine:
I think one of the other big barriers is really figuring out how do you get something from the bench to the bedside.
Carol Vassar:
Dr. Charitha Reddy
Dr. Charitha Reddy, Stanford Medicine
So realistically, you can publish something, you can put it out, and it often goes into a paper and then doesn’t really go anywhere else. How do you get it into the clinical practice? Right now, so much of what we do in pediatric echo is vendor-dependent. So we talk a lot about our ultrasound vendors or our echocardiogram reading systems, our PAC systems. They’re all kind of owned by a certain industry partner or industry company.
So how do we make sure that when we develop something that either it’s vendor-neutral so that it can really be utilized amongst all of those vendors so that one hospital isn’t left out because they use a different vendor or something like that. So, how do we really make sure that what we’re developing is vendor-neutral? And then the other is how do we make sure that it doesn’t feel like when you develop something that you’re developing it in a silo, that it’s your intellectual property, but that you want to share it with everybody else?
I think that part in pediatrics, we have to figure out a way to do that, and I think we’re open in pediatrics sometimes more than other places because we realize that the only way to move forward is to share with our other partners because we want to really make the health of children better overall. And so I think that’s one of the biggest things that we have to work towards. That’s the biggest barrier is getting it actually to the clinical bedside.
I think the thing is that change is coming. I think we’re excited to figure out ways to implement AI and machine learning for our patients and improve outcomes. That’s the biggest. I think that would be the only reason to really implement AI is really to we want to improve outcomes for our patients and find these patterns. So I think for the patient things to look forward to, are we going to be able to make some of these processes easier? Are we going to be able to provide care in rural locations that previously didn’t have access to experts because we can implement this technology out there?
Or for parents that are caring for really ill children at home, can we supply them with some technologies that allow them to assess their children at home with a relatively good accuracy, at least to flag, “Hey, this patient is sick,” or, “Hey, you need to call your doctor,” in a way that gives them a little bit more safety or a bigger safety net. So I think that’s where we should really be looking forward to some of the technology helping patients
Music:
Well Beyond Medicine
Carol Vassar, Host/Producer:
Looking forward, what is the future of AI and pediatric healthcare? Our experts weigh in on the final part of our three-part series on artificial intelligence in Pediatric Healthcare. Thanks for listening to the second of three episodes on AI and pediatric healthcare with me, Carol Vassar, and our guests, Dr. Devyani Chowdhury, Dr. Mark Friedberg, Dr. Charitha Reddy, Dr. Pei-Ni Jone, and Dr. Rima Arnaout.
Where do you see AI working toward a better and healthier generation of children? Let us know by leaving a voicemail on our website, nemourswellbeyond.org. You can also leave your comments and feedback on the AI series or any of our other podcast episodes, as well as listen to previous episodes, subscribe to the podcast, and leave a review. That’s nemourswellbeyond.org.
Thanks, as always, to our production team for their help with this series: Che Parker, Cheryl Munn, Susan Masucci, and Alison Michich. Until next time, remember, we can change children’s health for good, well beyond medicine.
(Singing.)
