This blog series shares what’s happening day-to-day in our Innovation Lab. VeraSci created the Innovation Lab to pair scientific knowledge with cutting-edge technology to improve the tools and measurements available for clinical trials. We originally presented these results at the Clinical Trials on Alzheimer’s Disease (CTAD) Digital Event. Click here to download the poster.

Measurements from validated wearable technologies have the potential to revolutionize clinical trials by facilitating site-less clinical trial designs. However, for this to become a reality, endpoints collected using wearables must undergo the same clinical and technological validation processes required by currently accepted tools. VeraSci’s Innovation Lab has been conducting an NIH-funded study examining the relationship between real-time measurements of gait and actigraphy collected by industry-grade wearables and established cognitive and functional endpoints that are sensitive to decline early in the Alzheimer’s disease continuum.

The preliminary data presented at CTAD and summarized below suggest correlations between remotely captured actigraphy and gait measures and established assessments of cognition and function, suggesting wearables can provide meaningful information regarding cognitive health and may ultimately contribute to continuous assessment of cognition and function in clinical trials.

About the Study

The study has enrolled 38 older adults (55+), including 26 healthy controls and 12 age-matched individuals with subjective cognitive decline. Participants attended two in-person study visits separated by one week of passive data collection using two wearable devices. At the first visit, participants completed well-established motor, cognitive, and functional assessments, including the Computerized Finger Tapping Test (CTAP), Timed 25-Foot Walk Test (T25-FW), Timed “Up and Go” (TUG), VeraSci’s Brief Assessment of Cognition (BAC), and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). At the second visit, participants repeated a portion of the in-office walking assessments.

The two wearable devices used by participants were the Empatica E2 wristband and the Moticon Science Insoles. The Empatica E2 wristband provided continuous measurement of motor activity (i.e., actigraphy), galvanic skin response, skin temperature, and sleep. The Moticon Science Insoles passively collect gait and movement data. The insoles characterize the subject’s gait across a gait cycle and within a specific phase of a gait cycle (see figure below).

phases of the gait cycle

Results

Participants with subjective cognitive decline performed significantly worse than the healthy controls on the BAC and the VRFCAT, indicating objective impairments in the subject cognitive decline sample.

Cognitive scores by group
VRFCAT scores by group

Participants with subjective cognitive decline performed worse on a portion of motor functioning assessments administered at the first visit. They were significantly slower than the healthy control group at completing the TUG. On the CTAP, individuals with subjective cognitive decline produced significantly fewer button presses with their non-dominant hand and using both hands, but the difference for the dominant hand was not significant.

For the Empatica E2 device, compliance rates during the at-home portion of the study were significantly lower for individuals with subjective cognitive decline than the healthy control group. Correlational analyses across the combined sample indicated several associations between E2 endpoints and cognition (see table below). There was a strong positive correlation between cognition/function and daily activity/movement. The average waking activity level was positively correlated with BAC verbal fluency, verbal list learning, symbol coding, and VRFCAT completion time.

The Moticon Science Insoles revealed that participants with subjective cognitive decline exhibit different gait parameters than healthy control subjects. Correlational analyses across the combined sample revealed associations of gait measures with functional capacity and cognitive endpoints (see table below). These results suggest that lower gait efficiency is associated with reduced functional capacity.

correlation between empatica and cognitive measures

Conclusions

These preliminary findings highlight the challenges of integrating wearable technologies into clinical trials for participants with even mild cognitive defects.

Despite compliance issues, preliminary data indicate moderate correlations between remotely captured actigraphy and gait measures and established assessments of cognition and function, suggesting wearables can provide meaningful information regarding cognitive health and may ultimately contribute to continuous assessment of cognition and function in clinical trials.

Learn more about VeraSci’s experience in Alzheimer’s clinical trials.

Dr. Luca Pani is the former Director General of the Italian Medicines Agency and a former member of the Board of Directors of the Committee for Human Medicines (CHMP) and of the Scientific Advice Working Party (SAWP) for the European Medicines Agency (EMA) in London. Luca is a recognized expert in basic and clinical pharmacology and regulatory science with particular emphasis on health technology assessments linked with novel negotiation strategies for the reimbursement of precision medicines. He is a Professor of Pharmacology and Clinical Pharmacology, University of Modena and Reggio Emilia in Italy and a Professor of Clinical Psychiatry, University of Miami.

It is important for anyone developing therapeutics that could be used to treat pediatric populations to take into account both FDA and EU regulations. EU Regulations seek to improve children’s health by: increasing high-quality, ethical research of medicines for children; increasing the availability of authorized medicines for children; and increasing information about medicines without conducting unnecessary studies on children or delaying medicines’ authorization for adult populations.

 

When are Pediatric Investigation Plans (PIPs) Required?

Pediatric development is mandatory in the EU for new products unless a waiver is granted. Waivers can be either product-specific or class-specific and only apply to specific conditions and dosage forms. Deferrals can be granted that allow studies in children to be completed after applying for marketing authorization in adults. For existing products, a Pediatric Investigation Plan (PIP) is mandatory when seeking a new indication, route, or dosage form if the product is protected by a patent or a Supplementary Protection Certificate (SPC). Off-patent products already authorized in the EU that do not have a valid SPC do not require PIPs. They are also not required for new medicinal products in certain groups, including traditional herbal products, homeopathic products, generic products, hybrid products, and biosimilar products. There are three types of waivers:

  • Total waivers apply to all pediatric subsets of specific conditions.
  • Partial waivers apply to some pediatric subsets or some indications but still require a PIP.
  • Some classes of products for a condition or all products for a condition may have a class-waiver.

The legal grounds for obtaining a waiver are (1) a product is not safe or effective in pediatric populations, (2) the disease or condition occurs only in adult populations, or (3) there is no significant therapeutic benefit in pediatric populations. The EMA may grant a deferral to avoid delaying marketing authorization in adults and may allow one or more studies in the PIP to be completed after the marketing authorization application (MAA). Deferrals are granted at the study or measure level. In some cases, regulators may require that sponsors initiate pediatric trials prior to marketing authorization, but may grant marketing authorization for adult populations prior to the completion of the pediatric trials. All types of deferrals must establish completion dates. Incentives are available for all correctly completed PIPs, and additional incentives are available for voluntary development through the Pediatric-Use Marketing Authorization (PUMA) program. “Correctly completed” means that the studies performed comply with the PIP agreed to in the compliance statement of the marketing authorization (MA), the results of studies are in the Summary of Product Characteristics (SmPC) and the patient’s leaflet, and the product is authorized in all member states (except for PUMA candidates). Non-orphan products can receive a 6-month extension on patent protection. Orphan products can receive two additional years of market exclusivity. Products submitted under PUMA can receive eight years of data protection, two additional years of market protection, and a partial EMA fee exemption for a year. Incentives are provided even when the PIP results are negative, but not when the results are inconclusive or when a waiver is granted.

What Does a PIP Cover?

PIPs are designed to collect data on efficacy, safety, and age-appropriate formulation. The PIP will outline studies needed and may include nonclinical studies in juvenile animals (i.e., toxicology, pharmacodynamics, pharmacokinetics, carcinogenicity, genotoxicity), safety, and proof-of-concept studies, dose-finding studies, and efficacy studies. PIPs will also provide timelines for each study. For new products, PIPs should be developed by the end of phase 1 clinical trials in adults. Amendments to the PIP may be made throughout phase 2 and phase 3. Data from adult studies cannot provide a full picture of the safety of a product in a pediatric population. Animal studies and clinical trials in pediatric populations are needed to determine the impact of a therapeutic on organ development, growth, maturation of function, neurobehavioral development, and realization of potentials. It is essential to ensure that pharmacovigilance mechanisms are adapted to meet the specific challenges of collecting safety data in a pediatric population, including data on possible long-term effects. Sponsors may be required to set up a risk management system or conduct specific post-marketing studies in instances where there is a particular cause for concern. Sponsors should start thinking about pediatric development early. Getting an agreement on a PIP with EMA typically takes 8-12 months and needs to be done before studies in children begin. 

Contact Us to Learn More

One of the most frequently used assessments in clinical trials of Parkinson’s Disease is the Movement Disorders Society version of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Producing high-quality data is crucial for demonstrating efficacy for new products in development. A recent study published in Movement Disorders examined frequency of MDS-UPDRS rater errors in the Parkinson’s Progression Markers Initiative (PPMI) cohort.

The study looked at the motor examination in Part III performed by site investigators who had completed online training for the MDS-UPDRS. In PPMI, investigators record ratings on a paper case report form and then enter the data into an electronic data capture (EDC) system. Inconsistencies were flagged based on the following criteria:

  • Resting tremor amplitude without constancy (or the reverse)
  • Hoehn and Yahr Stage 2 ratings with unilateral symptoms
  • Opposite lateralization of resting tremor or rigidity between upper and lower extremities

Based on these criteria, inconsistencies were identified for 11.8% of participants at baseline or first follow-up. The study recommends direct data capture, such as an electronic clinical outcome assessments (eCOA) platform as a mechanism to reduce inconsistencies and errors. One way eCOA can reduce errors in this situation is by eliminating transcription errors that occur when one value is written on the case report form and then incorrectly entered into the EDC system.

More importantly, eCOA allows for real-time data checking that can flag likely errors, prompting raters to address entries that appear inconsistent or fall outside expectations. For example, if severe postural instability was recorded without any gait disturbance, a flag would be raised to reconcile or reconfirm this unusual combination. Systems can also be configured to check for unlikely changes over time, such as reverse lateralization. These real-time checks are important because investigators can confirm the values during the visit rather than relying on memory if an anomaly is found later.

At VeraSci, we believe that well-designed, science-driven processes and technologies are the best cure for sluggish timelines and noisy data.

Contact us to see a demo of our Pathway eCOA system and find out how we can improve the quality of data in your next clinical trial.

References

In a recent article published in Neurology, the Dementia with Lewy Bodies (DLB) Consortium published new consensus research criteria for prodromal DLB, or DLB prior to the onset of dementia. These new criteria have the potential to significantly impact clinical trials and the development of drugs to treat, delay, or prevent DLB.

Background

Dementia with Lewy bodies (DLB), the most common form of dementia after Alzheimer’s disease, is characterized by progressive cognitive decline with fluctuating alertness, parkinsonism, and psychiatric disturbance. Many patients also experience sleep disruptions, visual hallucinations, and other psychotic symptoms. It is believed that DLB symptoms are related to the aggregation of a misfolded protein, alphasynuclien, into the eponymous Lewy bodies which disrupt neuronal functioning. Lewy bodies in the brainstem are thought to cause motor disturbances, while Lewy bodies in the occipital cortext and frontal lobes are related to visuospatial deficits and executive dysfunction respectively. Alzheimer’s neuropathology, including accumulation of amyloid-beta plaque and neurofibrillary tangles are also found in many patients with DLB and associated with cognitive deficits.

In 2017, the DLB Consortium published a consensus report outlining diagnostic criteria for DLB. The first core criterion was dementia. While establishing cognitive deficits as central to DLB, this had the unfortunate effect of delaying formal diagnosis of many patients who evinced the hallmark constellation of neuropsychiatric disturbances, but had not yet shown the magnitude of decline in cognition or functional independence requisite for dementia. As such, timely intervention with disease-modifying and neuroprotective therapies in DLB clinical trials was limited.

Criteria for Prodromal DLB

In 2020, new research consensus criteria for prodromal DLB was published in Neurology. This landmark publication acknowledges characteristic symptoms that may present years before the full DLB syndrome can be diagnosed. Prodromal DLB includes not only cognitive deficits, but motor signs and symptoms, sleep disorders, autonomic dysfunction, and neuropsychiatric disturbance. These clinical manifestations can begin as early as 15 years before formal diagnostic criteria for dementia are met. Three variants of prodromal DLB are described—mild cognitive impairment (MCI-LB), delirium onset (delirium onset-DLB), and psychiatric onset (psychiatric onset-DLB). The consortium proposes diagnostic criteria for probable and possible MCI-LB intended for use in research settings. The research criteria for MCI-LB includes essential elements, core clinical features, and proposed biomarkers.

Impact on Clinical Trials and Development of Therapies for DLB

Identifying patients in the very early stages of any neurodegenerative disease is critically important for developing effective therapies which can be implemented before the extent of brain injury is unrecoverable. The new research criteria for prodromal DLB allows the timely identification of patients at risk for transitioning to the full DLB dementia syndrome. By enrolling these early staged patients into clinical trials, there is the potential to slow the disease progression and prevent the onset of dementia. Because patients would be enrolled at a point in illness when they are more capable of adhering to study procedures, retention is likely to be higher as well in trials focused on prodromal DLB. This all bodes well for future clinical trials in DLB, enabling standardization across studies and early intervention in a clinically challenging neurodegenerative disease and bringing hope to patients with DLB and their families.

References

As the COVID-19 pandemic halted many in-person clinical trial visits, sponsors were faced with difficult decisions about whether to delay, cancel, or modify their on-going clinical trials. One option for some trials was to replace in-person visits with remote or virtual visits. Here, we will share VeraSci’s experience helping one of our sponsors with the transition to remote visits during the pandemic.

Background 

The trial in question was a large, global, phase II trial in schizophrenia. The pandemic related lockdown began about two years into the trial and dramatically impacted the ability to conduct in-person assessments at numerous sites. One of the key assessments used in the trial was the Brief Assessment of Cognition (BAC). The BAC is a performance-based outcome measure typically administered on a tablet by a rater during a patient’s visit to the clinic. The challenge was to develop a method of remote administration to be used during the pandemic.

Considerations at Multiple Levels

Making the switch to remote visits was a time-sensitive, multifaceted problem. Chief among our concerns was whether the assessment could be performed in a way that maintained scientific integrity. It was important to ascertain whether conducting the assessment through a telehealth visit would preserve the validity of the test. Regulatory considerations were equally important, however, and regulatory guidelines were emerging and evolving rapidly at the time. VeraSci was in close contact with regulatory agencies, and Dr. Rich Keefe, VeraSci CEO, was closely involved in the organization of the International Society of CNS Clinical Trials (ISCTM) Working Group meetings used to disseminate information about remote assessments from FDA and EMA to industry. We worked closely with Dr. Luca Pani, VeraSci VP of Regulatory Strategy & Market Access Innovation, to make sure our plans were as aligned as possible with emerging regulatory guidelines. There also were important data privacy and security considerations. It was essential to select solutions that met data privacy and security requirements, as well as ensuring patients’ privacy needs were being met. Finally, there was an array of practical concerns that needed to be addressed, such as technology, connectivity, training, testing materials, and data management.

Modifying the BAC for Remote Administration

Preparing the BAC for remote administration required a subtest-by-subtest consideration of what could be done in a way that was both practical and scientifically valid. Some of the biggest challenges were the Symbol Coding and Token Motor tasks, which required the use of a touchscreen. We considered several options for implementing these two tasks but ultimately decided to eliminate those subtests to meet the timeline requirements. We determined that the remainder of the subtests could be administered remotely with modifications. Using a telehealth format that incorporates videoconferencing technology was vital because it allowed the rater to monitor the patient’s environment and behavior during testing. Importantly, the modified assessment still enables the rater to compute a valid composite score.

BAC Remote Administration Deployment

The first step in deploying the BAC for remote administration was to develop the materials and procedures needed. This included developing a rater training process, developing technical support procedures, creating an administration manual, and modifying data entry procedures as well as identifying, qualifying, and procuring a secure platform for videoconferencing.

Remote Administration Deployment Process

The next step in deployment was rater training. This involved identifying the sites and raters that needed to perform remote administration of the BAC, distributing necessary materials to conduct the assessment, and distributing telehealth platform credentials.

Finally, as we began rolling out remote administration, we took a number of steps to make sure everything went smoothly. We provided on-going rater and technical support, conducted rater debriefing calls, supplied full data review with feedback to raters, and we evaluated rater and patient surveys. We used all the information we collected as feedback to refine our processes.

Results

As this study is currently on-going, we have been collecting formal and informal feedback from raters and patients. Overall, the responses have been very positive. Patients and raters have appreciated feeling safe while still doing the assessments. Knowing that we are concerned about their safety has led to an increased sense of engagement. We have not had anyone express privacy concerns. We have had some feedback from raters that there is a learning curve to managing both the technology for the assessment and the videoconferencing technology simultaneously. Most have felt that with practice, over time, they have become more comfortable. Most importantly, raters have reported that they do not believe there is any compromise to the validity of the testing or the data.

For more information about using remote assessments in your next clinical trial

As the COVID-19 pandemic made in-person visits difficult or impossible, sponsors and CROs turned to remote visits and remote assessments to keep trials running. This is an approach to which several regulators have signaled interest and VeraSci has endorsed. After careful consideration, we’ve moved forward with transitioning multiple on-going trials to remote assessments.

Increasingly, however, we’ve noticed a trend with some in our industry suggesting remote assessments can be added as an option in protocols to use any time a resurgence of COVID-19 happens to avoid missed visits or protocol deviations. These suggestions and recommendations ignore the complexities of implementing remote assessments in a way that will be acceptable by regulatory guidelines. While regulators have shown a willingness to be flexible during the pandemic, the validity of the data must be maintained. Downplaying the complexities involved in properly implementing remote assessments creates a risk that your trial data will not be considered acceptable to regulators.

How to Decide If Assessments Are Appropriate for Remote Administration

Not all assessments are suitable for remote administration. Each assessment needs to be considered individually in the context of the trial, and some may require modifications. Patient Reported Outcomes (PROs) can almost always be administered remotely because, by definition, they require patients to provide responses without external support. Many clinical interviews with patients and observers that do not involve any physical assessments are also appropriate for remote assessment based on reasonable clinical judgment. For some assessments, such as those that involve the assessment of non-verbal emotional expression, video is essential for an accurate assessment. In other cases, these can be completed by phone. In all cases, the appropriateness of remote assessment of clinical interviews relies on participants having a private space for completing the interview.

Traditional performance-based assessments (PerfOs) such as tests of cognition, motor function, and functional capacity can often be administered remotely, but typically require some modification. The level of modification varies widely, ranging from none needed for a completely verbal assessment to extensive modifications for assessments involving manipulation of objects. In some cases, the need for object manipulation makes remote administration impossible. Specific timing needs can also impact the feasibility of remote assessment. It may be necessary to send materials to subjects in advance of the session or to have someone with the subject to assist with administration. Similar to clinical interviews, these assessments often require videoconferencing. These assessments also require the subject to have a private space free of distraction and have the additional burden of the electronic equipment and often the data plan.

Scales involving a physical exam that include touching the subject, such as assessing rigidity or strength, are the most significant challenge for remote administration. For scales that include these elements, it may be possible to do a partial administration that does not include these items. For example, proxy scales for the assessment of medication side effects have been utilized by eliminating items that require hands-on assessment. Decisions regarding remote administration of scales with these elements are the most complex and require thorough consideration of the protocol, importance of those measurements, and the impact of not assessing them on the scale’s validity.

The appropriateness of remote administration of any assessment may require input from those who developed the instrument. During the COVID-19 era, some authors of tests or rating scales have weighed in on the appropriateness of their scales for remote administration. Some authors have provided specific suggestions for modification, and some have provided specific guidance indicating that their scale is not appropriate for remote administration. We always recommend consulting with the authors of an assessment to get their input on any modifications under consideration. A thorough literature search may also be necessary to determine whether there is any precedent for remote administration and if any validation research has been completed.

Switching Between Modes of Administration

We’ve also seen recommendations to allow individual subjects “ping-pong” between remote and in-clinic assessments, meaning that they could move back and forth between remote and in-clinic assessments multiple times based on conditions at the time of each visit. Although no rule currently exists, there is no regulatory precedent which introduces some risk for sponsors moving forward with this approach. Switching back and forth between remote and in-person assessments would require a thoughful justification for the multiple changes to allow for data interpretability in the context of, among others, the ICH E9 addendum framework.

It will be essential for investigators to document why they chose to use remote assessments because many remote assessments are not supported by good evidence of equivalency with the original measures.

Assessments Needed to Demonstrate Safety

Assessments which are designed to detect safety signals require special attention. When it comes to safety related assessments, it is likely better to err on the side of collecting incomplete data rather than collecting no data. Even if an assessment could only be partially administered or would rely on a patient to tell the clinician about something the clinician would normally observe directly, it may be better than receiving no safety information at all.

Data Analysis Considerations

Finally, when it comes time to analyze the data, special care must be taken to account for changes in the mode of administration appropriately. Sponsors will still be expected to follow existing regulatory guidance for the statistical analysis of clinical trial data, particularly as it relates to intercurrent events, estimands, and intent-to-treat principles as well as proper handling of missing data.

Do you have questions about adding remote assessments to a new or on-going trial? Contact us to discuss and get support in designing your trial.