Volume 12, Issue 7 pp. 1324-1333
Research Article
Open Access

Validity and Reliability of Clinical and Patient-Reported Outcomes in Multisystem Proteinopathy 1

Lindsay N. Alfano

Corresponding Author

Lindsay N. Alfano

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Department of Pediatrics, The Ohio State University College of Medicine, Columbus, Ohio, USA

Correspondence:

Lindsay N. Alfano ([email protected])

Search for more papers by this author
Megan A. Iammarino

Megan A. Iammarino

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Natalie F. Reash

Natalie F. Reash

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Linda P. Lowes

Linda P. Lowes

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Department of Pediatrics, The Ohio State University College of Medicine, Columbus, Ohio, USA

Search for more papers by this author
Lindsay Pietruszewski

Lindsay Pietruszewski

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Kathleen Adderley

Kathleen Adderley

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Lauren Humphrey

Lauren Humphrey

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Audrey B. Knight

Audrey B. Knight

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Christopher L. Steiner

Christopher L. Steiner

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Melissa A. Smith

Melissa A. Smith

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Zarife Sahenk

Zarife Sahenk

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Department of Pediatrics, The Ohio State University College of Medicine, Columbus, Ohio, USA

Department of Neurology, The Ohio State University College of Medicine, Columbus, Ohio, USA

Search for more papers by this author
Anne M. Connolly

Anne M. Connolly

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Department of Pediatrics, The Ohio State University College of Medicine, Columbus, Ohio, USA

Department of Neurology, The Ohio State University College of Medicine, Columbus, Ohio, USA

Search for more papers by this author
Momen Almomen

Momen Almomen

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Search for more papers by this author
Eleonora S. D'Ambrosio

Eleonora S. D'Ambrosio

Jerry R. Mendell Center for Gene Therapy, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA

Department of Neurology, UMass Chan Medical School, Worcester, Massachusetts, USA

Search for more papers by this author
Nathan Peck

Nathan Peck

Cure VCP Disease, Warner Robins, Georgia, USA

Search for more papers by this author
Allison Peck

Allison Peck

Cure VCP Disease, Warner Robins, Georgia, USA

Search for more papers by this author
First published: 28 April 2025

Nathan Peck and Allison Peck co-senior authorship.

Funding: This work was supported by Cure VCP Disease Inc.

ABSTRACT

Objective

Valosin-containing protein (VCP)-associated multisystem proteinopathy 1 (MSP1) is caused by variants in the VCP gene. MSP1 results in various phenotypes including progressive myopathy, Paget's disease of bone, frontotemporal dementia, amyotrophic lateral sclerosis, and parkinsonism, among others. Our study aimed to validate functional clinical outcome assessments (COA) and patient-reported outcomes (PRO) to inform clinical care practices and future clinical trial design. In addition, we evaluated the test–retest reliability of these COAs within clinics and remote environments.

Methods

Patients completed a battery of COA and PRO across a 2-day traditional onsite visit and a 2-day remote visit within their home environment. All COA and PRO deemed safe and feasible to complete based on participants' level of function and/or home environment were collected at each visit.

Results

Forty-six total patients enrolled in our study, 34 in our full study and 12 in an expanded remote-only cohort. Functional COA measured decline over reported disease duration in this cross-sectional group and significantly correlated with PRO (rho > 0.5, p < 0.001). Differences in lower and upper extremity involvement were noted across variant groups. Performance of functional COA was reliable and safe within and across onsite and remote testing environments (ICC > 0.7, p < 0.001).

Interpretation

Functional COA and PRO are valid and reliable to measure abilities in participants with MSP1. Testing can be completed reliably within the home, which could expand equitable access to clinical care and/or future clinical trial participation. Prospective longitudinal data collection is ongoing to understand outcome sensitivity and meaningful change over time.

1 Introduction

Multisystem proteinopathy 1 (MSP1) is a rare progressive disorder resulting in various phenotypes including progressive myopathy, Paget disease of bone (PDB), frontotemporal dementia (FTD), amyotrophic lateral sclerosis (ALS), and parkinsonism, among others [1]. MSP1, previously known as Inclusion body myopathy associated with Paget disease of bone and frontotemporal dementia (IBMPFD), is caused by variants in the gene encoding valosin-containing protein (VCP) [2, 3]. The retrospective, collaborative “VCP International Study” has identified characteristic patterns of muscle involvement via magnetic resonance imaging (MRI) techniques, as well as genotype–phenotype relationships across this heterogeneous cohort [4, 5]. While the most common variant reported is c.464G>A (p.Arg155His), the c.463C>T (p.Arg155Cys) has been reported to demonstrate earlier symptom onset compared to other variant groups [5]. Expert consensus standard of care guidelines have been developed to improve the consistency and proactive clinical management of patients, as awareness of this complex multisystem disorder continues to grow [1].

There are no approved treatments to halt or slow the progression of disease, with the exception of PDB, which may be managed with the use of bisphosphonates [1, 6]. Understanding the underlying pathophysiology of MSP1 is underway, while efforts to identify potential treatment pathways or approaches are in early stages [7-10]. There is a need to identify methods to quantify strength, function, and participation in patients with MSP1 to 11 proactive management of the disease and enable data-driven design of future clinical trials. The purpose of this study was to prospectively establish the validity and reliability of functional clinical outcome assessments (COA) in patients with MSP1. In addition, we sought to validate the remote collection of these COA utilizing live video conferencing.

2 Methods

Our prospective natural history study was approved by the Institutional Review Board at the Abigail Wexner Research Institute at Nationwide Children's Hospital (NCT04823143). Participants were recruited through the Cure VCP Disease registry as well as Nationwide Children's Hospital neuromuscular clinics; participants gave written, informed consent to join the study. Those enrolled were required to have a genetically confirmed variant in the VCP gene, be over the age of 18 years, and be willing and able to consent and complete study procedures. Participants were excluded if they were currently or had previously participated in an interventional clinical trial or exhibited any other illness or comorbid conditions that would interfere with the person's ability to complete study procedures safely.

The full natural history study is a planned 24-month longitudinal study with onsite clinic visits at baseline, 12 and 24 months; and remote visits were also completed at baseline continuing on a 6-month basis. The following reports on the baseline results of this ongoing study. Our full study protocol included 2-day onsite visits at Nationwide Children's Hospital in Columbus, Ohio, and an additional 2-day visit remotely via live video conference within the home environment, for a total of 4 baseline visits. Participants were randomly selected to complete onsite or remote visits first. All visits during the baseline phase were completed within a 2–4 week window, as generally disease progression would be expected to remain stable. Similarly, spreading visits across this timeframe was intended to minimize fatigue and the burden of repeated testing.

In addition to the full study, enrollment was expanded to include a small remote-only cohort in which participants completed all testing visits (i.e., 2-day baseline and 6-monthly visits up to 24-months) in their home via live video conference with a physical therapist. The aim of this subgroup was to increase enrollment across a diverse spectrum of disease progression and geographic location by reducing the burdensome requirement of travel to participate.

2.1 Onsite Visits

Participants performed a battery of functional testing at each in-clinic visit in a standardized order which included: the 100-m timed test (100 m); North Star Assessment for limb girdle-type dystrophies (NSAD) which includes the time to rise from floor (TTR) and 10-m walk/run (10 m), 4 stair climb (4SC); timed up and go (TUG); performance of upper limb (PUL); ACTIVE workspace volume (ACTIVE WSV); 9 hole peg test (9HPT); and spirometry including forced vital capacity with percent of predicted calculated using Global Lung Function Initiative reference values, forced expiratory volume in 1 s (FEV1), and maximal inspiratory (MIP) and expiratory pressure (MEP) [11-20]. Participants completed all testing that was safe to attempt based on their level of strength and function at each visit. All functional study procedures were completed by physical therapists (PT) with expertise in evaluating and providing care for patients with neuromuscular disorders. This team of PTs regularly establishes inter-rater reliability to the industry-standard expectations for multi-site international clinical trials [21]. A physical exam was completed by expert neurologists specializing in neuromuscular disorders at each onsite visit. Participants who opted-in to an optional biobanking sub-study provided blood, serum, and urine samples for future biomarker discovery.

Participants also completed several PROs, including Patient-Reported Outcomes Measurement Information System (PROMIS) global health, upper extremity (UE) and lower extremity (LE) function or mobility, cognitive function; Inclusion body myositis functional rating scale (IBMFRS); Rasch Overall ALS Disability Scale (ROADS); Eating Assessment Tool 10 (EAT-10); Communicative Participation Item Bank short form (CPIB); Speech Handicap Index (SHI) [22-28]. Participants completed each PRO at all visits.

2.2 Remote Visits

A reduced battery of functional COA was attempted remotely to include only those assessments deemed to be safe and feasible to complete within a home environment [29, 30]. These included the NSAD, TUG, PUL, 9HPT, and FVC. Assessments such as the 100 m and 4SC were deemed not feasible to complete remotely as our team expected very few homes would have available space for a 25-m straight path or a standard set of 4 stairs to mimic those available within the clinic.

Every effort was made to standardize the equipment used for remote study visits to match the clinic environment, which included shipping an aerobic step, hand grip myometer, PUL kit, 9HPT board, and spirometer directly to the home. These kits were assembled and shipped directly to homes by Cure VCP Disease. Prior to remote study visits, Cure VCP Disease coordinated a pre-site video conference to discuss expectations for the study visit and answer participant questions. Additionally, they reviewed all equipment to ensure it had been received, was in good working order, and was set up properly as appropriate.

During the remote study visit, the evaluating PT helped the patient, family, or caregiver identify appropriate spaces and furniture (e.g., chair, table, bed or couch, floor space) to complete each study assessment that would most closely match the clinic environment. If suitable space or furniture was not identified within the home, those items or tests were not completed or scored during the study visit. All participants were provided a cell phone tripod and encouraged to have a caregiver, family member, or friend available during their visits for safety and to assist with any needed logistics of completing the remote visit (i.e., equipment and video management). The remote visits were conducted via live video-conferencing with a PT providing instructions for equipment placement and/or movement to be completed. Timed tests were assessed with the start occurring when the caregiver, family, or friend said “GO” and stopped when the task was completed [30].

2.3 Statistical Analysis

Data analysis was performed with SPSS software (IBM SPSS, Chicago, IL, USA; Version: 29.0.0.0). Descriptive statistics were used to understand participant demographics. Independent t-test statistics were used to compare baseline age, age at symptom onset, and disease duration of our cohort. A general linear regression model was used to understand the impact of disease duration on the performance of COA. Spearman correlation coefficients with Bonferroni correction were used to understand the relationship between COA and PRO. Test–retest reliability of COA and PRO was evaluated using intraclass correlation coefficients (ICC2,1) and Bland–Altman plots.

3 Results

A total of 46 patients enrolled in our study and completed at least 1 study visit; 34 in the full study and 12 in the remote-only cohort. Demographics at that visit are included in Table 1, with the number of participants exhibiting each unique variant in the VCP gene listed in Table S1. Our cohort included 23 female patients (50%) with no significant difference in age at baseline, symptom onset, or disease duration at baseline across sex or cohort (e.g., full vs. remote-only cohorts). Three participants reportedly enrolled during a presymptomatic phase of muscle weakness (all female, age range 28–46 years). While no muscle weakness was reported, none of these participants ran at normative speeds and did not achieve full NSAD scores, demonstrating early deficits in standing on heels, tiptoes, and squatting [11]. For those with reported muscle weakness at baseline, the most common initial presenting symptom was muscle weakness. Table 1 lists the prevalence of other MSP1-related diagnoses in our sample at baseline. Of note, our sample was primarily presented with muscle weakness and/or PDB without other comorbid conditions such as ALS or FTD at baseline. Table 1 lists the prevalence of other MSP1-related diagnoses in our sample at baseline. Of note, our sample was primarily presented with myopathy and/or PDB without other comorbid conditions such as ALS or FTD at baseline. Half of the cohort (N = 23) reported concerns with bladder and/or bowel function. Of those reporting bladder concerns (N = 18), a majority (N = 10/18, 55%) reported both urgency and incontinence as a problem (mean age: 54.0 ± 11.9 years) with the remainder reporting mild to moderate urgency or increased frequency of urination only. Of those reporting bowel concerns (N = 11), a majority (N = 6/11, 55%) reported experiencing both bowel urgency and incontinence with the others reporting bowel urgency only. Five participants (11% of the full cohort) reported issues with both bladder and bowel incontinence (mean age: 49.8 ± 12.9 years).

TABLE 1. Baseline demographics and presenting symptoms of our cohort including mean and ranges of age, symptom onset, and disease duration (since first reported symptom) in our full and remote-only cohorts.
N Mean ± SD Range
Age (years) 46 53.6 ± 9.3 28.5 to 72.9
Female 23 51.6 ± 9.9 28.5 to 71.2
Male 23 55.5 ± 8.3 44.4 to 72.9
Full cohort 34 51.9 ± 9.3 28.5 to 71.2
Remote-only cohort 12 58.1 ± 7.8 47.5 to 72.9
Symptom onset (years) 42 45.3 ± 7.5 35.0 to 67.9
Female 20 45.2 ± 7.9 36.0 to 67.9
Male 22 45.3 ± 7.3 35.0 to 59.0
Full cohort 31 44.8 ± 7.9 35.0 to 67.9
Remote-only cohort 11 46.6 ± 6.2 36.0 to 56.1
Disease duration (years) 42 9.4 ± 6.4 1.5 to 24.0
Female 20 8.5 ± 6.4 1.9 to 23.8
Male 22 10.1 ± 6.5 1.5 to 24.0
Full cohort 31 8.6 ± 5.8 1.5 to 23.8
Remote-only cohort 11 11.6 ± 7.7 1.7 to 24.0
N %
First presenting symptom
Muscle weakness 38 83
Paget's disease of bone 5 11
Presymptomatic 3 6
Diagnosis at baseline
Muscle weakness 41 89
Paget's disease of bone 12 26
Cardiomyopathy 3 6
Parkinsonism 1 2
ALS 0 0
Charcot–Marie-Tooth 0 0
Frontotemporal dementia 0 0

All functional COA were highly correlated (rho > 0.7; p < 0.001) with the exception of PUL and TTR. This is not unexpected and would be evidence of divergent validity as these COA measure completely different functional constructs. Similarly, ACTIVE WSV was significantly correlated with PUL (rho = 0.73; p < 0.001) as both are measures of upper extremity strength and functional reaching ability. Ambulatory functional COAs were also highly correlated with PRO, including PROMIS UE & LE/mobility scales, IBMFRS, and ROADS (rho > 0.7; p < 0.001). The PUL was moderately correlated with these scales as well (rho = 0.5–0.6, p < 0.001).

The mean and range of performance of each COA can be found in Table 2. All COA were feasible and safe to complete in the clinic and remote environments (if included in the battery of testing based on testing location and participant ability). There were no study-related adverse events recorded during the baseline phase. Figure 1 highlights the relationship between key COA and reported disease duration in our cohort, with all outcomes being associated with reduced performance with increasing disease duration.

TABLE 2. Mean and range of performance of clinical outcome assessments (COA) across the cohort.
Mean ± SD Range
100-m (s) 97.1 ± 65.2 28.1 to 317.2
100 m (%) predicted 19.7 ± 15.7 0.0 to 57.0
100 m velocity (m/s) 1.1 ± 0.96 0.0 to 3.5
NSAD 32 ± 14 5 to 54
TTR (s) 8.5 ± 6.4 1.7 to 26.1
TTR velocity (task/s) 0.15 ± 0.14 0 to 0.59
10 m (s) 8.4 ± 4.1 3.3 to 18.9
10 m velocity (m/s) 1.1 ± 0.9 0.0 to 3.0
4SC (s) 5.2 ± 3.8 1.9 to 15.9
4SC velocity (steps/s) 0.9 ± 0.6 0.0 to 2.1
TUG (s) 8.6 ± 3.5 4.3 to 16.4
TUG velocity (m/s) 0.6 ± 0.4 0.0 to 1.4
PUL 33 ± 8 12 to 42
ACTIVE (m3) 1.2 ± 0.6 0.2 to 2.3
ACTIVE scaled score 114.7 ± 62.5 12.1 to 224.9
FVC (%) 84.7 ± 21.7 37.6 to 156.9
MIP (cmH20) −82 ± 22 −119 to −23
MEP (cmH20) 112 ± 45 37 to 217
Details are in the caption following the image
Cross-sectional performance of clinical outcome assessments (COA) by reported disease duration (first reported symptom) in years. (A) Individual performance of 100-m timed test velocity with dotted lines at 4.0 m/s (i.e., running speed), 1.2 m/s (i.e., lower limit of functional community walking speed), and 0.8 m/s (i.e., lower limit of household walking speed); (B) performance of NSAD; (C) timed up and go (TUG); and (D) performance of upper limb (PUL) across the cohort.

The 100 m was attempted at onsite visits, and 8 of 34 participants (23.5%) were unable to complete this test safely without an assistive device. Figure 1 highlights the variety of gait speeds present in our cohort, with 7 participants walking at transitional speeds below the threshold of functional household ambulation, 6 walking at functional household speed only, and 13 achieving velocities considered safe for ambulation within the community. No participant achieved a 4.0 m/s running speed, indicating there was no ceiling effect of this assessment [11].

The NSAD provided a useful and informative score for all patients completing this assessment in our cohort as no one scored the maximum (e.g., 54 points) or minimum (e.g., 0 points) on this scale. Seven patients (15%) scored ≥ 50 points nearing the ceiling effect, and similarly, the lowest score was 4 points, which approached a floor effect. Three participants were missing total NSAD scores in the home environment due to difficulty transferring safely to supine with the helper available during the visit or the perceived burden of transferring out of their wheelchair. Based on discussions with these patients, it is likely their performance would have been near the floor effect of the NSAD due to their reported limited movement ability in supine; however, this could not be objectively assessed. Our participants could be classified into functional groups based on NSAD performance (Figure 2). Those in our ambulatory cohort who could also complete TTR (i.e., rising from the floor without the use of furniture or other external assistance) demonstrated a mean NSAD score of 40 ± 8 points. Those who were still ambulatory but had lost the ability to complete TTR had a mean score of 22 ± 6 points, and the non-ambulatory cohort NSAD score showed a mean of 9 ± 3 points.

Details are in the caption following the image
Functional groupings of (A) NSAD score in ambulatory patients with valid TTR time (circles), without the ability to perform TTR (squares), and nonambulatory participants (triangles); and (B) PUL score in those with the strength to reach overhead (circles), perform mid-level activities like bringing hand to mouth (squares), and those independently completing tabletop activities only (triangles).

Our cohort performed four timed functional tests, including the TTR and 10 m (performed as part of the NSAD), the 4SC, and TUG. The most challenging task was TTR, with 21 of 46 (45.6%) participants unable to perform this transfer independently. An additional 4 patients took ≤ 3 s to complete the test, falling within the ceiling effect of this assessment. Thus, the TTR was only cross-sectionally informative in 48% of the cohort. As demonstrated in those with other neuromuscular diseases exhibiting proximal weakness, TTR is the most challenging functional task and is typically the first lost as the disease progresses. Twelve participants (26.0%) were unable to complete the 10 m safely without the use of an assistive device. Capturing a valid time for 10 m was challenging in remote home environments, as 18 participants (39.0%) could not attempt this test due to space constraints within their home. Four participants completed the assessment in ≤ 4 s, indicating a slight ceiling effect in the strongest of the cohort.

Similar to the 10 m, 13 participants (28.3%) were unable to complete the TUG as they could not rise from the chair independently and/or walk safely without an assistive device. No one completed the TUG in ≤ 4 s, indicating a limited ceiling effect of this assessment in our cohort. This was the second most challenging task, as it was most likely to be lost after TTR. In those able to complete the TUG, 10 participants (30%) took > 11 s (range: 11.1–20.9 s), putting them at a higher risk for falls [31-34]. A threshold of > 12 s seems to put participants at a high risk of being unable to complete the TUG independently. Similarly, the risk of losing independence with TTR increases with TUG time > 9 s, as 50% of these patients are unable to complete TTR, compared to 100% of those with a TUG time < 9 s being able to complete TTR. Lastly, just 6 participants (18%) attending onsite visits were unable to complete the 4SC. An additional 6 completed the 4SC in < 3 s, indicating a ceiling effect in the strongest participants.

No one in our cohort scored < 10 points on the PUL at baseline, indicating no floor effect of this COA. Conversely, 17 participants (37%) achieved a score ≥ 39 points, indicating a ceiling effect in those without upper extremity involvement. The PUL entry item classified participants into functional reaching groups, including those with overhead reaching ability (N = 26; mean PUL score: 31 ± 2.5 points), mid-level abilities (N = 18; mean PUL score: 26 ± 4.6 points), and tabletop only (N = 2, mean PUL score: 15 ± 4.9 points) (Figure 2). Upper extremity involvement is more common in the p.Arg155His and p.Arg155Cys groups, with only 2 of 16 patients reaching a ceiling effect on the PUL at baseline, compared to 9 of 11 patients with p.Arg159His or p.Arg159Cys, and 2 of 3 patients with p.Arg93Cys variants (Figure 3). This amounts to 79% of those combined cohorts achieving a ceiling effect on the PUL at baseline, with no significant difference between age at baseline or disease duration across variant groups.

Details are in the caption following the image
Cross-sectional performance of (A) NSAD, (B) 10 m velocity (dotted lines: 1.2 m/s lower limit of community walking speed; 0.8 m/s household walking speed), (C) PUL, (D) FVC percent of predicted (dotted lines: 70% as lower limit of normal performance; 50% as need for nocturnal ventilatory support) plotted by disease duration (years since first reported symptom) and variant group: p.Arg93Cys (red circles), p.Arg155Cys (green circles), p.Arg155His (green squares), p.Arg159Cys (blue circles), p.Arg159His (blue squares), and all other variants (black circles).

Assessment of forced vital capacity (FVC) could be completed safely within the onsite and remote environments. Just 8 participants (17.4%) had FVC percent of predicted dropping below 70% (e.g., the threshold for typical FVC% for older adults), though only 2 had values below 50%, indicating an increased risk for nocturnal hypoventilation [1, 35, 36]. Maximal pressures were collected in participants with onsite visits; most had MEP and/or MIP greater than 60 cmH20, with 4 participants (14.2%) falling below this average threshold [1, 35, 37]. Eight participants were using overnight BiPAP support at baseline, and 2 additional reported use of CPAP as management of sleep apnea.

3.1 Test–Retest Reliability

All COA and PRO demonstrated significant and high test–retest reliability within and across testing environments (Table 3). The 9HPT, TUG, and PUL had slightly more variability across testing environments; though, as Figure 4 illustrates, this difference appears to be driven by just a few outliers. In discussions with participants at visits, this variability is likely driven by differences in chair height (for the TUG) and/or table height differences (for the PUL and 9HPT). The assessment most frequently not able to be administered in the remote environment was the 10 m assessment due to space constraints. There were a few homes with limitations on floor space or changes in flooring that made finding a 3-m track for the TUG not feasible at one or more visits. Similarly, for those participants with more advanced weakness (N = 3), it was not feasible to complete a full NSAD, as transferring to supine was deemed unsafe or too much of a burden at the visit (Table 3).

Details are in the caption following the image
Bland–Altman plots to assess individual test–retest reliability of the (A) TUG at onsite visits, compared to (B) TUG across onsite and remote environments; and (C) PUL at remote visits, compared to (D) PUL across onsite and remote environments.
TABLE 3. Intraclass correlation coefficients (ICC) of the test–retest reliability of performance of each functional clinical outcome assessment and patient-reported measure. Reliability of performance across onsite, remote, and the first onsite versus remote visits is reported. p < 0.001 across all reported ICC.
Onsite Remote Across
100 m 0.98
NSAD 0.99 0.99 0.97
10 m 0.98 0.97 0.96
4SC 0.93
TUG 0.97 0.97 0.90
PUL 0.98 0.99 0.93
ACTIVE 0.83
9HPT 0.90 0.96 0.88
FVC 0.80 0.95 0.91
Patient reported measures
Global health 0.97 0.93 0.96
IBMFRS 0.82 0.97 0.76
UE 0.89 0.98 0.91
Mobility 0.97 0.99 0.97
Cognitive 0.97 0.94 0.93
ROADS 0.95 0.97 0.93
SHI 0.59 0.98 0.94
CPIB 0.71 0.91 0.80
EAT-10 0.93 0.87 0.96

4 Discussion

The findings from our study suggest that functional COA validated in other neuromuscular conditions are valid and reliable for use in patients with MSP1. The included COA measures both gross motor, upper extremity, and respiratory function across our heterogeneous cohort. All outcomes demonstrated significant decline with increasing disease duration, though sensitivity and meaningful change over time will be evaluated in future studies. The 100 m was a more challenging assessment, though it could be safely completed by more than 75% of our cohort traveling to onsite visits. No participant could run at an average speed, and there was quite a range of gait velocities captured across our group [11]. The NSAD and PUL could be captured in virtually all patients. There may be a slight floor effect of the NSAD in very progressed, nonambulatory patients as a few patients declined transferring into supine due to concern of difficulty breathing, decreased ability to move against gravity, or the burden of the transfer. We did not find a floor effect with the use of the PUL in our cohort, but there was a significant ceiling effect due to variability in timing of upper extremity weakness across variant groups. Interestingly, our findings support those identified retrospectively in that the p.Arg159His group may demonstrate a slightly later onset of weakness and the p.Arg155His and p.Arg155Cys groups have an earlier onset of upper extremity involvement than those variants at the 159 or 93 residue [5, 38]. Use of the 100 m, NSAD, and PUL assessments clinically or in clinical trials would provide a reasonable overview of ability across functional domains while limiting the impact of floor and/or ceiling effects in assessment.

These COA were highly reliable within and across environments. To our knowledge, this is the first study to evaluate the novel approach of remote, in-home assessments via live video-conferencing in adult patients with a neuromuscular disorder. We sought to understand the feasibility of this approach in an adult cohort at risk for frontotemporal dementia and who may also have less experience with video-conferencing software and other technology (i.e., spirometer software application). We attribute the success of our methods to the collaborative approach to study design and implementation with the patient-led advocacy group, Cure VCP Disease. The co-founders of Cure VCP Disease sourced, organized, and shipped standardized equipment kits to all consented participants. In addition to the kits, they provided individualized pre-visit calls to each participant to prepare them for remote visits. These pre-visit calls included an overview of the activities that would be performed (if that participant was unfamiliar or had questions), an orientation to the study equipment kit contents and process for use, and setup of the required software for spirometry testing with validation that all was in working order. In addition, Cure VCP Disease provided lessons learned, reminders, and overall support to all participants as needed across the course of the study. Due to this collaborative partnership, all visits could be completed safely and efficiently within the home environment via live video-conferencing software. All participants were instructed to have a family member, friend, or caregiver present during visits to ensure safety and assist with testing and video capture as needed. There were no visits interrupted during this baseline phase due to internet connectivity issues or other concerns.

It is important to note that the functional relevance of the COAs used in our study can be considered tools for both proactive management of patients with MSP1, as well as data-driven clinical trial design. Functional COAs selected for inclusion in our study have known relevance to disease in patients with neuromuscular disorders, and many were developed with direct patient input. The 100 m, for example, if performed prospectively in a clinical environment, can be used to understand how a patient is walking or maybe beginning to note restrictions in walking throughout their community or home. As patients are reaching lower thresholds for walking speeds, these measurements can inform care discussions about assistive device or mobility equipment use. Similarly, as patients take longer to complete the TUG (e.g., 9 or 12 s) asking about loss of ability to rise from the floor or get out of a chair independently can aid in proactive prescription of adaptive equipment. The NSAD functional groups may, similarly, be a useful tool in these discussions. Considering clinical trial design and readiness, our study identifies key floor/ceiling effects across each COA, which can be helpful in designing future enrollment criteria and/or selection of efficacy endpoints.

We acknowledge there were some limitations to our studies. Our full study sample could be biased toward stronger participants as we required travel to a single site in Columbus, Ohio. However, due to the impressive test–retest reliability across testing environments, we were able to expand enrollment to include participants with more progressed disease who felt travel to our site was too burdensome or infeasible. Similarly, in this rare disorder, our remote-only cohort enabled enrollment of those from other regions outside the United States, including Europe, South America, and Australia. To begin understanding and quantifying the impact of MSP1 on motor function, we selected COA from available assessments validated in other neuromuscular conditions. It is possible future research could focus on the development of novel assessments or tools to quantify disease and progression of disease in MSP1. Similarly, our study focused on the muscle weakness phenotype in MSP1 and while we collected medical history and other information on other body systems, a prospective study to evaluate the onset and progression of other phenotypes, and the relationship between body system involvement is key to fully understanding the impact of MSP1 for patients.

Functional COA and PRO are valid and reliably quantify functional abilities cross-sectionally in participants with MSP1. Testing can be completed reliably within the remote, home environment, which has the potential to expand access to clinical care and/or reduce the burden of future clinical trial participation by reducing the frequency of travel to a clinical trial site. Prospective longitudinal data collection is ongoing to understand COA and PRO sensitivity and meaningful change over time in participants with MSP1.

Author Contributions

L.N.A., M.A.I., N.F.R., L.P.L., N.P., and A.P. contributed to the conception and design of the study. L.N.A., M.A.I., N.F.R., L.P.L., L.P., K.A., L.H., A.B.K., C.L.S., M.A.S., Z.S., A.M.C., M.A., E.S.D., N.P., and A.P. contributed to the acquisition and analysis of data. L.N.A. drafted a significant portion of the manuscript and figures. All authors contributed to the review and approval of the final manuscript.

Acknowledgements

We would like to sincerely thank all the participants and their family, friends, and caregivers for their time and effort they provided to contribute to our study. This study was funded by Cure VCP Disease Inc. In addition, the team at Cure VCP Disease compiled and provided equipment kits, as well as an initial orientation call to prepare participants for the remote visits. Cure VCP Disease also provided travel support for participants traveling to Columbus, Ohio, for onsite visits. The authors would also like to acknowledge the contributions of Dr. Jerry R Mendell, who, when he served as a principal investigator at Nationwide Children's Hospital, initiated development efforts in MSP1 and participated in capturing physical exams across the study to contribute to the validation of clinical outcomes in this collaborative effort.

    Conflicts of Interest

    The authors declare no conflicts of interest.

    Data Availability Statement

    The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

      The full text of this article hosted at iucr.org is unavailable due to technical difficulties.