March 1, 2023 | Growing interest in the use and utility of digital health technologies (DHTs) was on full display at the recent Summit for Clinical Ops Executives (SCOPE), highlighted by a pair of co-occurring sessions on their use and utility in clinical trials. While outsourcing professionals were learning from Merck’s experience in evaluating and selecting DHT suppliers, digital data specialists were getting an update on progress with the multi-sponsor Watch-PD study comparing sensor-based symptom measures with traditional clinician-rated assessments in patients with early, untreated Parkinson’s Disease (PD).
Merck has established an entire resource center housing a DHT and innovation playbook, decision tree to help internal teams understand trigger points for various assessments, lean evaluation matrix to be sure they’re asking vendors the tough but necessary questions, an innovation inventory facilitating knowledge-sharing, and a transition checklist to streamline the transition of projects from early-stage pilots to clinical trials. It all began with the development of an overarching DHT and innovation framework by an advisory group comprised of people from different cross-departmental teams, according to Sonali Bhatnagar, associate director of clinical innovation and digital health, in the company’s R&D sourcing and procurement division.
The need for a comprehensive solution was spurred by a litany of DHT-related appeals and proposals related to trial-specific needs or intriguing new technology or business imperative, she says. For a company the size of Merck, it could be “anything and everything... [and] we can’t go after every shiny object.”
Today’s reality, says Bhatnagar, is that people have variable enthusiasm for DHTs depending in part on their functional area, first-hand experience with various products, differing priorities, and time-related pressures. Among commonly voiced concerns are the risk of inadvertently running afoul of regulatory requirements or missing a critical step and clinical projects either losing momentum or not failing fast enough.
The cross-functional team assembled to address the chaos—and provide a “line of sight” to tractable, near-term technologies—included representatives from global digital analytics technologies, global clinical supply, procurement, IT, quality assurance, pharmacovigilance, data management, and early- as well as late-stage operations, she says. The end-to-end methodology they came up with “could be adopted by any company.”
The first job of the cross-functional advisory group was to “talk through the business case” for DHTs and come up with a framework for understanding the scope of work (SOW) on a project and the critical related variables, explains Bhatnagar. Next, they discussed minimum necessary testing, risk assessments, contractual clauses, and governance alignment for evaluating vendors as quickly as possible.
At the decision phase on a pilot-tested DHT, the group convenes to hear from business stakeholders about what did and didn’t work well and determine if it makes sense to move into a clinical trial, she continues. Assuming all goes well with subsequent assessments and contractual activities, the new technology vendor is then onboarded within Merck’s systems and oversight meetings begin with relationship management colleagues.
The DHT and innovation playbook provides visual and textual guidance “so people know exactly who to go to, what questions to ask, [and] at what stages,” says Bhatnagar. The “very short, very succinct” decision tree tool was developed next to ensure teams were neither under- nor over-evaluating suppliers.
All the tough questions teams should be asking vendors are outlined in the extensive lean evaluation matrix, she says, and “not only from a capabilities perspective but [in terms of] what kind of appetite they have for getting involved in future clinical trials.” The innovation inventory is intended to help break the silo mentality, which tends to plague big companies, by enabling people to learn about their colleagues’ experience working with suppliers rather than always starting from scratch.
The transition checklist then ensures all the important variables get factored in as projects move toward implementation in clinical trials. “As impressive as this sounds, it’s all a learning cycle,” Bhatnagar says, “so we do a lot of after-action reviews with our vendors... [and] look at [their] system from time to time to see if it is working efficiently.”
Bhatnagar shares a sampling of some of the critical questions sponsors should be asking contracted suppliers to “identify any showstoppers,” as delineated in Merck’s lean evaluation matrix. One of them revolves around the necessity of being “audit-ready at any point in time” by asking suppliers for data flow diagrams, algorithm information, and data transformation information. All regulatory documentation needs to be ready, be it for an inspection by the Food and Drug Administration, Occupational Safety and Health Administration, or Environmental Protection Agency.
From an audit trail perspective, sponsors need end-to-end visibility to certain records and have a process for obtaining them, she continues. They also need to know suppliers “have the appetite” for their record-keeping obligations, notably under FDA 21 CFR Part 11.
Equally important is to ask suppliers about their quality check steps and establish how frequently data will get transferred to the sponsor, says Bhatnagar. Other useful questions get at their blinded/masked data experience and disaster recovery and business continuity plans. And if an otherwise viable vendor does not have a mitigation plan in place, internal stakeholders need to decide if it is a risk worth taking.
Sponsors will also want to ask for any relevant information about suppliers’ system development life cycle process for creation of their databases and tools, she adds.
From a procurement perspective, it’s important to build in enough negotiation time with suppliers to avoid eleventh-hour frustrations and unwanted commitments, Bhatnagar shares. She further advises understanding the extent to which subcontractors are supporting the company and how they’re qualified—and, if there’s a chance that they can be acquired by someone else, what’s the potential impact and backup plan.
Be sure milestones and deliverables are connected to payments, she continues, and tie that into the SOW. “Things happen, but we want to be sure we’re getting the business done and also holding each other accountable.” A risk-based approach allows for moving as fast as possible while abiding by internal policies.
Some vendors, unless they are well established, may not have a process in place for adverse event handling/key performance indicators, Bhatnagar adds, although it may not matter depending on the SOW and complexity of a project. But it should be clear if one company or the other, or both, owns the intellectual property. Similarly, expected outcomes of the project need to be spelled out.
When it comes to contracts, the usual templates may not be a good fit with new technology vendors, she notes. IT procurement colleagues may have a more suitable option.
At the contracting stage, keep in mind that what might be okay from a legal and compliance standpoint today (perhaps due to a pressing need) may not be tomorrow. Bhatnagar suggests preemptively having those conversations with internal stakeholders and externally with suppliers to get “on the same page” about the basis of the relationship.
Implementing New Technologies
As DHTs move from the pilot phase to a device feasibility or experimental medicine study, a top concern is always whether the technology works as expected, says Bhatnagar. This would include any transmission gaps and potential failure modes, and the potential regulatory implications in the markets of interest.
Whether a DHT is scalable to a clinical trial from a cost perspective also needs to be calculated, she adds. “What kind of supply assurance do we have? Does the team have the resources?” Plans for building connections to other labs and technologies, and any required assistance, need to be part of that conversation.
Lastly, says Bhatnagar, measures need to be established to ensure success throughout the engagement. These would potentially include oversight meetings, governance meetings, and interactions with relationship managers to ensure roles and expectations are aligned.
Merck has achieved significant internal buy-in on its DHT and innovation framework by getting the initial buy-in of senior management and, as needed, doing periodic “road shows” for people who are resistant to the approach, Bhatnagar says. It has been especially valuable to showcase success stories of their peers in the same group and, as needed, to have one-on-one conversations to understand perceived obstacles and offer support and reassurance.
Precompetitive development of digital measures in Parkinson’s disease was the focus of a contemporaneous presentation by Jie Shen, Ph.D., director of digital science at AbbVie. Specifically, he described headway being made by the Digital Drug Development Tools (3DT) initiative of the Critical Path for Parkinson’s Consortium.
Parkinson’s disease is named for English surgeon James Parkinson who in 1817 first described the symptoms in six individuals in a published essay entitled “Shaking Palsy,” which is what the condition was called for 60 years, begins Shen. It’s a neurodegenerative disease and, although tremor is the first identified symptom, many parts of the body are later impaired and these can include nonmotor skills such as cognition, sense of smell, dermatological and gastrointestinal issues, and pain.
PD has no cure, Shen says, and the drugs developed to date are designed either to relieve symptoms or modify the disease to slow down its progression. This highlights the importance of early detection.
One of the challenges for PD drug development is how to measure disease progression, which tends to happen slowly and fluctuate, says Shen. Given the huge variability and the fact that most patients only visit the clinic every few months, it can be difficult to interpret the data to know when they are declining. Current clinical measures, focused primarily on motor features, also don’t adequately reflect the multidimensional nature of worsening PD.
“Digital measurement might be a gamechanger here, because [it’s] empowered by a digital health tool that can be administered by patients on their own at home,” he continues. “That allows the measurement to be at a much higher frequency and also, because it is a more objective measurement, we would expect a smaller variability.”
Clinical measures of disease progression have traditionally included bradykinesia (slowness of patients), tremors, finger tapping, gait, voice, and speech, says Shen. Efforts are emerging around cognition and passive data collection. “The goal is to develop individual measurement associated with those symptoms, and also trying to get a consensus score to detect the whole disease.”
Shen cites previous work by University College London (CloudUPDRS), Johns Hopkins University (HopkinsPD), Sage Bionetworks (mPower), and Roche (Roche PD mobile app). Most recently, as part of the Critical Path of Parkinson’s Consortium, a group of companies began working together on the Watch-PD study, originally sponsored by Biogen and Takeda, and agreed to share data, learnings, and success. The collaboration has facilitated dialogue with regulatory agencies, helping to promote the development, validation, and adoption of digital measurement in the drug development process for PD, he says.
Watch-PD is a non-interventional, observational study involving both an in-clinic assessment and at-home test. Every two weeks, participants conduct a series of tasks (bradykinesia, mobility, speech, and cognition) and complete a set of surveys at home, explains Shen. The devices include the Apple Watch and iPhone powered by Digital Artefects, a platform for running decentralized trials.
Participants have in-clinic study visits every three months, in addition to their regular check-ups, when they undergo a quantitative motor assessment (including bradykinesia, gait and balance) that is part of the MDS-UPDRS that has been used to measure PD progression for more than 40 years. As they perform each prescribed task, a clinician or assistant assigns them a score between 1 and 4 (best to worst). The motor examination, combined with the MDS-UPDRS sections measuring non-motor and motor experiences of daily living, produce an overall score reflecting disease severity.
Measurement and Validation
Movement-related symptoms are normally measured digitally using two types of sensors—an accelerometer capturing acceleration in three different directions and a gyroscope capturing rotation speed, says Shen. Patients might wear six different sensors, including one on each hand and foot as well as on the chest and lumbar area, capturing their movement while they conduct the various tasks. The data are then analyzed to reveal distinct patterns.
Once the raw data is collected, signal processing converts the continuous digital measurements into “epoch-level” features, after which one of two strategies can be deployed, Shen continues. A heuristic algorithm might be used to directly get the pattern from the raw signal. Alternatively, a machine learning algorithm could be used to associate different features with a desired outcome, and the information would then be aggregated to reflect the so-called digital endpoint.
Analytical and clinical validation precede adoption of a digital endpoint, he points out. The former is focused on how the digital measurement endpoint performs against a pre-specified standard, including correlation with ground truth, as well as its reproducibility. The latter looks at the applicability of the digital endpoint for a specific patient group, including baseline group difference analysis (correlation with disease severity) and sensitivity to the longitudinal change.
For the in-clinic portion of the Watch-PD study, bradykinesia data was collected by having participants do three tasks on both sides of the body, shares Shen. The first was supination/pronation, where they were instructed to rotate their hand 10 times as fast as they could. The second was toe tapping, where they were asked to lift your toes 10 times as fast as they could. The third was leg agility, where participants would lift their legs as high as they could and then stomp on the ground as hard as possible, again repeated 10 times as fast as they could.
This produced six different MDS-UDPRS scores, to which a machine learning-based measures were applied to extract important features (e.g., frequency and amplitude) while participants were doing the tasks, he says. These were then associated with their MDS-UPDRS score.
A separate set of participants was used to validate the algorithm by measuring the agreement between ratings made by the machine learning model and the clinical measure outcomes. The algorithm “not only counted the exact same ratings; it also gives partial credit to the close-enough ratings,” Shen reports, and overall attained moderate agreement with MDS-UPDRS scores.
PD-associated tremors, including both resting and postural types, were captured by a wearable sensor, he continues. But in this instance, the movement is “relatively unique” and thus was relatively easy to capture using heuristic methods. As evidenced by a 2007 study out of Switzerland (IEEE Transactions on Bio-Medical Engineering, DOI: 10.1109/TBME.2006.886670), it is possible to quantify the magnitude of tremors capturing only the frequency between 3 and 6 hertz.
Clinical validation was accomplished here by comparing the baseline tremor score generated by the digital measurement and the associated MDS-UPDRS score. The tremor digital endpoint had “relatively good performance” based on that correlation, says Shen.
Gait impairment is a relatively complex symptom covering many different features, each of which could be extracted with a digital measurement, he continues. Stride length symmetry was identified as the most sensitive feature, “clearly separating the PD patient from the non-PD patient... [over a] six-month period.”
Digital measurements have “great potential” in PD and many other progressing diseases, concludes Shen, as evidenced by the many efforts in this space and precompetitive collaborations that are catalyzing progress. Among the challenges are how to aggregate high-frequency data when there isn’t a gold standard against which superiority can be demonstrated and the lack of industry consensus on an analytical and clinical validation framework.