Latest News

The Unleashing Of AI In Real-World Diagnostics

By Deborah Borfitz

September 6, 2018 | Despite months of speculation about a potential human backlash against artificial intelligence (AI), focused largely on unknowable safety hazards and jobs that will fall victim to automation, AI-driven healthcare technologies are already being quietly embraced in real-world healthcare settings around the country. Machine smarts is getting a well-deserved round of applause where it improves disease detection and treatment, and the daily life of overwhelmed pathologists and diagnosticians around the country.

Consider Prognos (formerly Medivo), which has amassed possibly the world’s largest clinical dataset on the planet comprising nearly 18 billion test results on about 175 million patients. The Prognos Registry is now being used both as a disease prediction tool and to better match therapies to patients, according to Co-founder and Chief Medical Officer Jason Bhan, M.D., who spent the first decade of his career as a practicing family medicine physician. The company started with the proverbial low-hanging fruit—compiling lab results data on 50 common or otherwise costly disease areas from labs around the country.

Lab test results are “critical and actionable” information used for almost every healthcare decision and, relative to other clinical datasets, concentrated in fewer hands, notes Bhan. But many “pretty significant decisions,” including how to deploy resources and qualify patients for programs, still rely on often dated, inaccurate claims data and prescription information.

To get closer to its ambitious, 25-year vision to “eradicate disease,” Prognos spent more than five years applying AI, machine learning, and simple logic to its super-sized registry to come up with different ways to identify patients somewhere in their journey—i.e., a newly diagnosed diabetic patient, someone whose blood cancer has just gone into remission or a patient who has failed miserably on the first-line drug treatment. In 2016, Prognos started using machine learning algorithms to predict when those patients would reach a place where they’d need a new drug, develop a complication, or fail a therapy, says Bhan.

The next step was to apply that same intelligence to a larger data set that included lab test results as well as claims and prescription data, continues Bhan, to build predictive analytics models of interest to various players in the industry. Prognos then began helping risk-bearing payers forecast when patients would be the sickest and require more intervention, and pharmaceutical companies identify strong candidates for hard-to-fill clinical trials—especially patients in their networks with rare and often undiscovered diseases. “Generally, we just plug directly into whatever customer relationship management system they’re already using,” Bhan says.

Prognos also started assisting labs in organizing, normalizing, standardizing, and otherwise getting their data into an AI-ready format to boost the quality of the test results data they were sending to physicians. These included pilots capturing data elements discretely rather than in free text fields that could not be queried to produce meaningful results, Bhan notes, including the prevalence of small cell cancers and melanomas among skin samples being tested.

“We don’t always have clients on the receiving end when we start digging through and evaluating a condition of interest,” says Bhan. That was the case a few years ago, when Prognos proved Hepatitis C prevalence was being underreported and it found pockets of patients who weren’t being treated simply because therapeutic options didn’t exist when they were diagnosed. Prognos has made similar strides with rare diseases and cancers, he adds. It also has uncovered evidence that the incidence of certain diseases, including prostate cancer, are misrepresented in registries of the Centers for Disease Control and Prevention.

Prognos works with laboratories of every size and specialty, including both major national reference labs and 26 different pharmaceutical brands. Five major payer groups have more recently signed on and Bhan expects more will be added to the mix as they see the value of the Prognos Registry to improve the way they engage patients, communicate with providers and prioritize where and when they act to improve patient outcomes. For Cigna, Prognos recently discovered undetected health risks among their covered members with complex disease who were in need of targeted care management.

Better Cancer Care

Since early 2014, Memorial Sloan Kettering (MSK) Cancer Center has done genomic profiling of tumors and matched normal DNA in more than 27,000 cancer patients—and the dataset recently expanded to include those with hematologic malignancies such as leukemias and lymphomas. Patients of the center routinely consent to targeted tumor sequencing via its MSK-IMPACT test, which can detect both inherited germline mutations and tumor-specific alterations, says Michael Berger, associate member of the Department of Pathology and associate director of MSK’s Marie Josee & Henry R. Kravis Center for Molecular Oncology.

The dataset supports the development of “basket studies” enrolling patients who share a specific genomic alteration that can be targeted by an investigational drug, or a drug approved for another cancer type, regardless of where the cancer originated. MSK researchers led a basket study of the drug vemurafenib, already known to be effective in melanoma patients, expanding its FDA-approved use in 2015 to lung cancer and two rare disorders— Erdheim-Chester disease and Langerhans cell histiocytosis. Similarly, the FDA last year approved the immunotherapy drug pembrolizumab for patients with all tumor types having a certain biomarker (microsatellite instability) in their genome. “This idea of testing drugs across many different histologies and correlating response with the genomic signatures is definitely catching on,” Berger says.

Key drivers of the clinical genomics program at MSK are to help design better trials and identify the patients most likely to respond to investigational new therapies. A 2017 study in Nature Medicine found 11% of those who had their tumor sequenced enrolled in a clinical trial at MSK based on a mutation that was detected, Berger says, and that figure has since grown by several percentage points. And nearly 37% of tested patients had at least one “actionable” mutation widely viewed by clinicians as treatable with available drugs.

MSK-IMPACT has also “incidentally” discovered inherited mutations associated with cancer susceptibility, leading to genetic counseling for patients and their family members and, in some cases, early detection and intervention, Berger says. A 2017 study found that half of clinically-meaningful pathogenic germ line mutations identified among the first 1,000 patients would have been missed based on current genetic screening guidelines, Berger says. “It has been one interesting case after the other” for genetic counselors at MSK, he adds.


The informatics system built around the MSK-IMPACT is designed to maximize the clinical utility of the information being generated, says Berger. This includes a large dataset of tumor mutations (some with no functional role whatsoever), and their known clinical and biological significance, which get automatically annotated in test reports. Oncologists are notified of clinical drug trials their patients may be eligible for based on one or more tumor mutations, and the alerts can be timed to coincide with routine visits. New computational methods, including machine learning, are also being deployed to “squeeze even more information out of the genomic data.”

Berger’s lab is currently focused on building a classifier to predict a cancer’s tissue of origin, which remains the basis of most clinical treatment. Mutations and alternations have been observed in different frequencies in different types of cancers, he explains, making it possible to create a diagnostic decision support tool for the roughly 5% of patients with metastatic disease treated at MSK where the original primary site can’t be identified.

Improving Diagnostic Accuracy

Machine learning meets dermatology at health technology company VisualDX, inventor of a decision support tool aiding physician interpretation of medical conditions affecting the skin, hair, and nails at the point of care. “Evidence suggests that 10% to 20% of all diagnoses are wrong, which is frightening to patients,” says CEO Art Papier, M.D. VisualDx puts symptom-based information at the fingertips of primary care and emergency medicine physicians faced with the “daunting challenge of knowing all of medicine.”

VisualDX is built on a “highly juried and reviewed” database of 100,000 professional medical photos that have accumulated over the past two decades, says Papier. They’ve helped train an algorithm to interpret any new image uploaded to a cloud-based server or, thanks to Apple’s Core ML machine learning framework, taken and analyzed directly on an iPhone or iPad. The software returns multiple diagnostic possibilities as well as a “best match” to the most likely disease or condition depicted based on the latest medical research. Information on over 1,000 medications, including drug reactions and side effects, is also readily available to physicians.

The software empowers generalists to recognize unusual presentations of common diseases, improving diagnostic accuracy and the appropriateness of referrals to specialists. Worried patients can see firsthand how clinical decisions are reached and feel reassured their doctor isn’t just “winging it from memory,” Papier notes. According to a recent survey, VisualDX saves users an average of 14 minutes per day (26 minutes among nurse practitioners and physician assistants).

Physicians decide which images are of sufficient quality to be added to the database, and medical librarians constantly comb the medical literature to ensure the veracity of other information in the product, which currently includes 3,000 diagnoses and hundreds of thousands of relationships between symptoms and diagnoses. “Medical diagnostics is not big data,” Papier notes. “Typically, physicians only see about 500 diagnoses their entire career.”

Currently, customers of VisualDX include 2,300 hospitals and large clinics around the world. “Physicians much prefer to reach for their smartphone or search on a desktop for information than going down the hall to grab a book,” says Papier. The medical software can be integrated with most any EHR and, ultimately, may be fully embedded in the most widely-used systems. The foundational database will further improve once the outcomes of patients can be tied back to how they presented, revealing new trends and associations.

Meeting Demand in the Lab

ARUP Laboratories, a large national reference laboratory and nonprofit enterprise of the University of Utah and its department of pathology, is endeavoring to automate some of the nearly 400 microscopy-based diagnostic tests on its menu. These tests range from infectious diseases, fecal parasites, histopathological examination of potentially malignant lesions, and fluorescence in situ hybridization (FISH) tests that searches for genetic abnormalities associated with cancer—all of which require extensive technical expertise that is in increasingly shorter supply, according to Program Manager Orly Ardon, PhD. Given the volume of tests being done at most labs, turnaround times and patient care can benefit from computer-assisted diagnostics.


“Most of those tests are still being done the same way they were 100 to 200 years ago just by taking a specimen, smearing it on a glass slide and looking at it under the microscope,” Ardon says. With digital imaging, whole slides instead get digitized and allow image analysis software to help technologists and pathologists identify regions of interest and help interpret what they’re seeing. Fluorescent signals of varying intensity can be picked up on FISH slides by a machine learning computer program comparable or better than the most highly skilled pathologist sitting for hours in a dark room, she adds.

Despite the obvious patient care advantages, wider adoption of this form of AI has only recently been considered by the lab industry, says Ardon. Hesitation to change and invest in test improvements are being overshadowed by shifting economics highlighted by competitive and reimbursement pressures and an aging population that continues to ramp up demand.

Only some of the current microscopy tests have a digital image analysis solution today, at least among large academic labs. Development of new machine learning tools takes time and resources, but the added benefit of computer vision allows the detection of subtle features that are not apparent to human observers and can aid classification of microbial or cancer cells and better weed out the negative specimens. Technologists and pathologists gain time they can devote to slides that test positive and their professional training, improving the work experience in the lab. In another five years, she predicts, “many more specialized lab tests will have a computer doing at least part of the diagnostic work.”