February 16, 2023 | Efforts to use digital endpoints to modernize clinical trials have been slowed by an assortment of challenges, including lack of data standardization, regulatory uncertainty, and how to demonstrate their value to clinical study teams. But the field will be getting a big push in the right direction in another few months when the first hard evidence will be published demonstrating digital endpoints can live up to their promise of delivering smaller, speedier trials.
These were among the themes emerging from a panel discussion at last week’s Summit for Clinical Ops Executives (SCOPE) in Orlando, moderated by Rinol Alaj, director of digital health technologies at Regeneron. Weighing in on the conversation were Lada Leyens, Ph.D., global regulatory lead within clinical trial innovation and digital health at Roche; Michelle Crouthamel, head of digital science at AbbVie; Guangchen Ruan, senior advisor and data scientist within the advanced analytics group at Eli Lilly and Company; and Jeremy Wyatt, CEO of ActiGraph.
The topics up for discussion were themes of a roundtable discussion the day prior, notes Alaj, who kicked off the session with a question around terminology. “Digital biomarkers” have been alternatively referred to as “digital measures,” “patient-reported outcomes” (PROs), and “eCOAs” (electronic clinical outcome assessments), and the variance has been belaboring constructive dialogue.
“How are we going to standardize evidence generation if we don’t standardize language?” Leyens rhetorically asks the group. “Because we collected digitally doesn’t change the fact that a biomarker is a biomarker, and a clinical outcome assessment is a clinical outcome assessment. Collecting it digitally just means we have to validate an extra thing—the tool we use to collect it.”
Leyens is now trying to align people on terminology across industry, a task that within Roche alone took about six months. “It comes down to the evidence we need ... to validate these measures,” she says, since biomarkers and clinical outcome assessments get validated through different regulatory pathways. They also get used differently in trials.
Definitions on the European side are currently highly misaligned with what officials with the U.S. Food and Drug Administration (FDA) are using, says Leyens, an ex-regulator with the European Medicines Agency (EMA).
Crouthamel, who serves as lead of TransCelerate’s novel digital endpoint team, acknowledges passionate beliefs on both sides of the word war. But when such measures are being used to support a drug approval, one of three efficacy criteria is the goal: “to prove your drug makes the patient feel better, function better, or live longer.”
The FDA website indicates digital biomarkers don’t measure outcomes, she continues, noting that in oncology only a biomarker with a proven link to ultimate survival can be called a surrogate biomarker. “The same thing applied here. That’s why there are two different mechanisms [for biomarkers and eCOA] to get things approved.”
Critically, “meaningfulness and concepts of interest” should link to the outcome—and sponsors and regulators should be aligned on this upfront if the objective is to use a digital means to support a primary or secondary endpoint in the IND pathway, Crouthamel says. Abbie will sometimes tell the FDA its interest in measuring tremor using digital sensors, only to be told “it is only one element of the disease manifestation” and therefore insufficient.
“There is just no firm answer yet” from the agency on what could be considered a biomarker versus an outcome, she continues. The FDA currently has passive measurements using digital health technology featured in an eCOA table, but differing opinions surface when engaging regulators in live dialogue.
Part of the confusion here is that the digital markers aren’t as straightforward as measuring, say, blood pressure, says Wyatt. When talking instead about ActiGraph’s specialty, accelerometry and movement, the difficulty is in explaining how a random waveform translates into a patient’s behavior.
It’s not simply a matter of producing a measurement, Wyatt says. Movement can be variably measured based on the desired level of precision, amount of sampling done, and inter- and intra-device consistency. And accelerometry still lacks standards.
When it comes to concepts of interest that matter to patients, he adds, “we [erroneously] try to apply the standards for PROs... [but accelerometry] is a continuous time measurement.” Patients must therefore be questioned differently about their understanding of what’s being measured and what it means.
ActiGraph’s biosensor solutions are often used by study sponsors as exploratory endpoints in clinical studies, “but we’re seeing an uptick in [their use as] secondary and primary endpoints,” says Wyatt. “In fact, right today, ActiGraph is being used as a primary [endpoint] in a pivotal phase 3 [trial].”
Getting digital endpoints qualified for use in a clinical trial is a lengthy process, says Ruan, who favors the idea of getting started by running a pilot study to generate some preliminary evidence and help build the confidence, and interest, of clinical teams in using a digital device to collect extra data from patients. “I also think it’s critical to have that validation inside of the clinical trial.”
Guidance on novel digital endpoint approval is available from the Clinical Trials Transformation Initiative, continues Ruan. Psychometric validation, concerning how a digital endpoint relates to existing measures used for drug approvals, would best be done by looking across indications versus at a single compound with inclusion/exclusion criteria that would limit the patient population size.
It is likewise important to repeatedly try out a novel endpoint in different contexts but for the same population, thereby “pre-specifying” what will later be done in a clinical trial setting, she says. It is also helpful when looking for clinically meaningful change using the novel endpoint to start with an approved drug with known efficacy.
Ruan also points out that the operational feasibility of using a novel digital endpoint can’t be properly tested outside of the clinical trial setting. “We [wouldn’t] really know what we’re facing, later on, if we try to deploy it in... a [more challenging] free-living environment.”
Usability is another key component, which the FDA included in its 2021 draft guidance on digital health technologies, Alaj adds. Patients’ perception of a device can change throughout the course of a study such that “patches may be good for 14 days versus wristwatches... for long-term longitudinal studies.” He encourages companies to invest in user experience researchers to be part of endpoint validation programs.
Cognitive impairment can impact patients’ ability to interact with technology and that can in turn have a “statistically meaningful impact on the data,” points out Wyatt. Likewise, individuals with a dexterity problem may be challenged to press a button on a phone to get to and open an app. “So, those workflows are important considerations around usability and validation.”
With the EMA, companies can bring their questions to a specially assembled innovation task force and, after they have a validation plan in place, go through a “very flexible” qualification advice meeting to learn if a novel measure can be used in a late-stage trial, Leyens shares. In the U.S., the process happens inflexibly through IND meetings. The FDA is also less likely than the EMA to apply “common sense” rationale when it comes to establishing patient relevance.
Interestingly, she adds, the EMA recently began insisting that Roche do the validation work in the context of clinical trials “to show sensitivity to change.” It’s a trend that has likewise been seen with projects done with the Innovative Medicines Initiative.
“Gold standards are not-so-gold standards,” says Leyens. “Sometimes you don’t have anything that you can compare with that is good enough, and then the question is if you cannot compare with the not-so-gold standard is it because your measure is bad or because the not-so-gold standard was really bad, and regulators recognize it.” In some cases, the FDA has recommended that the comparator be some other type of data altogether.
The Roche team also has a lot of questions about validating across context of uses, Leyens says, and “regulators are not open to that yet.”
ActiGraph has partnered with Bellerophon Therapeutics on a phase 3 trial testing INOpulse (inhaled nitric oxide) for the treatment of fibrotic interstitial lung disease in patients at risk for pulmonary hypertension. Moderate to vigorous physical activity, as measured by actigraphy, is the primary endpoint for detecting patient improvement.
In the phase 2 study, where actigraphy was an exploratory efficacy endpoint, INOpulse was well tolerated and associated with maintenance of physical activity and improved symptomatology, Wyatt reports. “It’s a digital endpoint with a lot of industry validation... [that has] showed significant change even over traditional PROs as well as a six-minute walk test. But the key thing [here is that] it also correlated greatly to meaningful aspects of health for the patient.”
In meetings with the FDA, Wyatt says, he was on point to explain particulars of the digital endpoint such as how activity monitors get calibrated and raw data “gets reduced to a form that we can apply statistics to.” Bellerophon was cleared at the end of 2022 to reduce the number of patients in the phase 3 study by about 50%, which represents a “massive payoff” for the company. Top-line study results are now expected in mid-2023.
Meanwhile, Leyens says her colleagues will be publishing data this May showing a better-than-50% reduction in sample size based on “clear signals with very small standard deviations” from a digital endpoint.
At AbbVie, change management remains the key challenge, reports Crouthamel. In-house clinical teams need to be shown multiple cases where a digital endpoint has been used to deliver faster, smaller studies—and the audience needs to expand beyond operational staff to include medical doctors overseeing global development and are focused on scientific evidence.