By Paul Nicolaus
September 9, 2019 | Last year, IDx-DR became the first-ever so-called "autonomous AI" system cleared by the FDA to provide a screening decision without the oversight of a doctor. Since then, IDx Technologies, the company behind the product, has begun to roll out this tool designed to detect diabetic retinopathy.
Over 30 million people are living with diabetes in the US alone. "The disease itself is bad," said Michael Abramoff, a University of Iowa ophthalmologist and computer scientist, "but the complications make it even worse."
One of those complications is diabetic retinopathy, the leading cause of blindness in American adults. It sets in when high levels of blood sugar damage the tiny blood vessels of the retina—the thin, light-sensitive layer of tissue that lines the back of the eye.
"People with diabetes fear nothing more than going blind," he said. "They fear it more than losing a limb or even dying, so it’s a terrifying complication."
Individuals with all types of diabetes are at risk for diabetic retinopathy, according to the National Eye Institute. The early stages usually have no symptoms, however, and the disease often progresses unnoticed until it impacts vision.
"We also know that in almost 95% of cases, it can be treated well if diagnosed early," Abramoff added, "so early detection and diagnosis is key." But according to the Centers for Disease Control and Prevention (CDC), as many as 50% of patients aren’t getting their eyes checked or aren’t diagnosed early enough for treatment to be effective.
It's a dilemma that Abramoff has worked to address. His research, which has mainly focused on the detection of diabetic retinopathy using smart cameras and AI algorithms, led him to establish a company called IDx Technologies nearly a decade ago.
His tool, the IDx-DR, is a software program designed to be used by health care providers to detect more than mild diabetic retinopathy (mtmDR) in adults who have been diagnosed with diabetes. This is accomplished by capturing images using a retinal camera.
Patients are asked to look at a green spot, he explained. There's a flash, and the retinal image is taken. The operator is required to have no more than a high school graduation level of education, as well as specific training by the company to use the robotic camera.
"There's an assistive AI that helps the operator take good quality images at the right area of the retina," he said, and it indicates when to retake if the image is out of focus, for example. This process typically takes a few minutes.
From there, an algorithm analyzes the images and searches for lesions specific to diabetic retinopathy. According to Abramoff, "the AI makes a diagnosis right then and there, which takes less than a minute."
If the image quality is sufficient, the software provides one of two results: either mtmDR is detected and the patient should be referred to an eye care professional, or mtmDR is not detected and the patient should be retested in a year.
Approval Granted, Limitations Highlighted
In April 2018, the IDx-DR became the first-ever so-called autonomous AI system cleared by the US Food and Drug Administration (FDA) to provide a screening decision without the need for a clinician to interpret the image or results.
The FDA evaluated data from a clinical study (doi: 10.1038/s41746-018-0040-6) that included 900 patients with diabetes and demonstrated 87% sensitivity and 90% specificity at detecting mtmDR using the IDx-DR.
"While this study is undoubtedly a milestone, and an important benchmark for future research, it is also important to touch on some of its shortcomings," Eric Topol of The Scripps Research Institute and Pearse Keane of the National Institute for Health Research (NIHR) Biomedical Research Centre for Ophthalmology wrote in an editorial piece (doi: 10.1038/s41746-018-0048-y) that accompanied the study.
Although recruitment did occur from 10 primary care sites, the study is still relatively small in diagnostic accuracy terms, they pointed out, and additional clarity is needed on the study end points. The sensitivity end point agreed upon with the FDA was 85%, which was met, but the confidence intervals of this estimate ranged from 81.8% to 91.2%.
In addition, 40 participants enrolled in the study were excluded from the analysis because their images were later found to be of insufficient quality. "The authors attempt to address this by considering a worst-case scenario where all such images are incorrectly graded and repeating the analysis," they wrote, but the sensitivity would be 80.7% (76.7%-84.2%) using this approach. "They note that this calculation rules out a pre-specified inferiority hypothesis of 75%, but do not highlight that the superiority end point would no longer be met."
There are clinical limitations that extend beyond the methodological caveats highlighted. The diagnostic system requires the use of a retinal fundus camera called the Topcon NW400, for example, which comes at a cost of roughly $18,000. "Besides the expense, it remains to be determined how and where it would be implemented," Topol and Keane added. "Will primary care clinics incorporate retinal screening into their practice?"
The editorial's co-authors made the point that this isn't exactly an "autonomous system," considering a human still needs to perform the task of actually obtaining the images.
And they raised the question of whether or not patients will directly benefit from the use of this type of AI system. The FDA approved Computer-aided detection (CAD) systems for mammography in 1998, they noted, but nearly 20 years later a large study (doi:10.1001/jamainternmed.2015.5231) concluded that CAD doesn't improve the diagnostic accuracy of mammography. Prospective interventional studies are needed, they noted, to address this issue.
(Topol declared an advisory role for Verily Life Science and Keane noted a consulting relationship with DeepMind in the competing interests section of their editorial.)
The IDx-DR should not be used with pregnant patients who have diabetes, according to the FDA announcement, or with patients who have a history of laser treatment, surgery, or eye injections. Nor should it be used with patients who have "persistent vision loss, blurred vision, floaters, previously diagnosed macular edema, severe non-proliferative retinopathy, proliferative retinopathy, radiation retinopathy, or retinal vein occlusion."
Iowa Sees Initial Rollout
The University of Iowa Hospitals and Clinics became the first site to make use of the system, and Yumi Imai, an endocrinologist affiliated with the university, has been involved with implementation at the diabetes clinic.
If there is not an existing record within the past year, she explained, then diabetic patients are typically asked when they've had their most recent eye exam. If a patient is not up to date on their yearly exam, the screening can be handled the same day using the IDx-DR.
This avoidance of a separate appointment helps increase the number of patients who receive their eye exam and screening. Imai estimated that it captures about 20% to 30% of patients who make their way to internal medicine or primary care to receive diabetes care but do not make their way to an ophthalmologist.
Imagine you are someone with diabetes, Abramoff said. You go in for your regular diabetes management visit or follow-up with your doctor or nurse practitioner. They record your weight and take your blood pressure. They perform a blood draw for your A1C and handle similar lab tests. And then you discuss how things are going with a physician.
If a diabetic eye exam is recommended, you are referred to an eye care provider. That's a separate appointment, potentially months away, that may involve travel. The appointment may be rescheduled, or you may miss the appointment.
There are all sorts of reasons why people don't make it to that eye exam. Part of it is a cost problem, he said, and part of it is an access problem. The vast majority of patients do go through their diabetes management systems, however, so if the diabetic eye exam is made available right then and there, within minutes, that access problem is solved.
Beyond that, the cost for the diabetic eye exam is lowered, Abramoff said, because the doctor is taken out of the equation. From the clinical trial, he added, "we know AI is more accurate than me, as a retina specialist."
The IDx-DR can be used anywhere in the world, he said. The only thing needed is an operator with a high school education, a robotic camera, a little bit of space, and patients.
The machine itself is "very easy to operate," Imai said, and the operator "can be easily trained." According to her, the workflow has been smooth and patients seem happy that they don't have to make an additional visit to an ophthalmologist to receive an eye exam.
In terms of drawbacks, Imai pointed out that about five to ten percent of patients have small pupils, which can prevent a diagnosis using the machine. If those percentages get smaller, she said, it would be even more useful.
There will always need to be monitoring to ensure the machine is functioning appropriately, she pointed out, but so far, she and colleagues are "confident that the machine is functioning and helping us do the diagnostics."
Reimbursement Issues and Other Hurdles
What began at the University of Iowa Hospitals and Clinics has since expanded. Abramoff indicated that many places around the country are now using it as well, typically in primary care endocrinology clinics. The rollout has presented a variety of challenges along the way, and among them are reimbursement related issues.
"Nothing in the healthcare system was prepared for autonomous AI," Abramoff said. The system is built around doctors that carry out work and get paid for that work. Since there is no doctor effort required to make use of the IDx-DR, there was no mechanism in place to deal with payor reimbursement.
Recently, however, the American Medical Association's (AMA) Current Procedural Terminology (CPT) Editorial Panel created a code for retinal imaging with automated point-of-care, which will facilitate the billing of IDx-DR.
There is a process between the creation of CPT codes and actually billing and collecting, he said, so there is a need to address who is going to pay for what, when, until the code goes into effect at the start of 2021. But the most important step on the path to payment has been set.
"There were a host of other hurdles that we needed to address for providers to be able to use this in the clinic," he said, like who orders what, when? Or how do you integrate it with the electronic health record (EHR)?
"One by one we have been clearing these hurdles," he added, and gradually reaching a point where there are essentially no more obstacles left to overcome. "This can just be employed like any other technology."
What's Next?
Artificial intelligence is not new, explained Abramoff, and it's not new in healthcare, either. The FDA has previously cleared AI-based tools, but these have been assistive AI technologies that help a clinician make a diagnosis.
What is new is the approval of an AI-based tool where the technology makes the diagnostic decision by itself without clinician oversight. From his perspective, the IDx-DR is a game-changer that can lower costs, improve quality, and improve access.
"I think if we do it right—if we do this transparently and in an accountable fashion—then it will have tremendous benefits for us as patients and as taxpayers and for the healthcare system."
But people also have concerns, and rightfully so, he said. Is it safe? Is it efficient? Is it accessible? Is it integrated into the healthcare system? Does it improve patient outcomes? Will doctors lose their jobs?
"We just need to be very careful and deliberate how we do this," he said. "People have valid concerns. We need to be transparent and accountable for what people care about and are concerned about—and very open." There are benefits, he added, but there are also risks. And people need to know about that.
Looking ahead, Abramoff has his sights set on addressing diseases beyond diabetic retinopathy, such as macular degeneration and glaucoma.
Paul Nicolaus is a freelance writer specializing in science, nature, and health. Learn more at www.nicolauswriting.com.