Online assessment of ALS functional rating scale compares well to in-clinic evaluation: A prospective trial

Self-assessment of symptom progression in chronic diseases is of increasing importance in clinical research, patient management and specialized outpatient care. Against this background, we developed a secure internet platform (ALShome.de) that allows online assessment of the revised ALS Functional Rating Scale (ALSFRS-R) and other established self-assessment questionnaires. We developed a secure and closed internet portal to assess patient reported outcomes. In a prospective, controlled and stratified study, patients conducted a web-based self-assessment of ALSFRS-R compared to on-site assessment. On-site and online assessments were compared at baseline (n = 127) and after 3.5 months (n = 81, 64%). Results showed that correlation between on-site evaluation and online testing of ALSFRS-R was highly significant (r = 0.96; p < 0.001). The agreement of both capturing methods (online vs. on-site) was excellent (mean interval, 8.8 days). The adherence to online rating was high; 75% of patients tested on-site completed a follow-up online visit (mean 3.5 months, SD 1.7). We conclude that online self-assessment of ALS severity complements the well-established face-to-face application of the ALSFRS-R during on-site visits. The results of our study support the use of online administration of ALSFRS-R within clinical trials and for managing the care of ALS patients.


Introduction
The ALS Functional Rating Scale (ALSFRS) is a validated, clinician-administered instrument for assessment in the domains of gross and fi ne motor function, bulbar symptoms and breathing ability in patients with ALS (1). The score refl ects deterioration of function in the natural course of ALS but may have lower sensitivity in advanced disease stages (2,3). The scale was developed primarily to assess outcomes in pharmaceutical clinical trials and does not rely upon physical examinations or instruments (1,4). An initial imbalance within the scale that minimized the importance of respiratory function was rectifi ed by a revision (ALS Functioning Rating Scale, revised (ALSFRS-R)) to incorporate respiratory symptoms and the need for ventilation (5). When administered as an interview, the ALS-FRS-R shows a high inter-rater and intra-rater reliability (6,7) and can be reliably administered over the telephone (7 -9).

ORIGINAL ARTICLE
Online assessment of ALS functional rating scale compares well to in-clinic evaluation: A prospective trial would not degrade the quality of the data. We also sought to explore the time and physical burden to patients of collecting data online to establish feasibility of online monitoring methods. Because regular assessment of inevitably declining physical function might have adverse psychological consequences for patients we also sought to explore the emotional burden of participating in the site.

Study protocol and IT infrastructure
We designed a prospective single-centre clinical study to evaluate the feasibility and reliability of online self-assessment. The protocol encouraged clinician-confi rmed patients with possible, probable or defi nitive ALS (El Escorial criteria) (15) to visit the website weekly over a 52-week period. Each patient determined a week day on which to perform his or her weekly self-assessment using patient reported outcomes (PROs). In order to measure the inherent appeal of returning to the site unprompted, patients did not receive a reminder, e.g. by telephone, e-mail or text message. PROs included the ALSFRS-R, the ALS Assessment Questionnaire (ALSAQ-40) (16), the Council on Nutrition Appetite Questionnaire (CNAQ) (17) and two self-reporting assessments of dyspnoea (Borg ' s scale CR10, CDS) (18,19); data on these instruments will be reported separately at a later date. Subsequently, the ethics review committee and Data Security Offi cer approval was obtained (Ethikkomission der Charit é -Universit ä tsmedizin Berlin, Charit é platz 1, 10117 Berlin). We created the web-based application www.ALShome.de for data capture, a related database for storage and a content management system (CMS) for the administration and analysis of PRO measurements including the ALSFRS-R. The internet application www. ALShome.de and the CMS were developed in the programming language C Sharp (C#). Data are stored on a Microsoft ® SQL database located on a secure internal server at the university data centre. The CMS allows for website management including the administration of PRO measurements, data export for further analysis and adjustment of web visit protocols at each time-interval.

Patients and data collection
Between 2 February 2010 and 2 February 2011, 162 of 443 ALS patients seen at the Department of Neurology at the Charit é University Hospital of Berlin gave written consent for the trial in accordance with the Declaration of Helsinki. Lack of internet access and the refusal to participate were the most common reasons for non-attendance. One hundred and fortyfour patients completed at least one online assessment (89%). We excluded 17 patients from the analysis due to protocol deviation as more than four weeks had elapsed from on-site to online assessment. In-clinic FRS assessment was collected via tablet PC administered by a nurse. Thus, in 127 patients (78%) we obtained a complete set of in-clinic assessment followed by online assessments, at baseline. The mean interval between online and on-site visits was 8.8 days (SD 6.3). The on-site follow-up data were captured from 108 patients (85% of baseline) at a visit that was, on average, 3.5 months later (SD 1.7). In 81 patients (50% of all patients included in the study) we obtained both forms of assessment at two time-points ( Figure 1).
In addition, the patient was asked questions about who fi lled in the questionnaire (patient or caregiver), the time burden on the patient, and the physical and emotional strain of periodic online selfassessment, rated with a single-item ' none ' , ' low ' , ' moderate ' , or ' high ' response scale.

Data analysis
Data were analysed with IBM SPSS Statistics version 19.0.0.1 for Macintosh. Results were expressed as means ( Ϯ SD) if normally distributed and medians (maximum/minimum) if distribution was non-Gaussian. Correlational analysis was performed with Spearman ' s rho because of the ordinal nature of the ALSFRS-R. A statistically signifi cant difference of paired samples was analysed with a t-test. The difference plot method (20) by Bland and Altman was used for analysing agreement. A value of p Ͻ 0.01 (two-tailed) was considered signifi cant.

Results
One hundred and twenty-seven patients (described in Table I) met the conditions for statistical analysis according to study protocol. The mean ALSFRS-R on-site at baseline was 33.6 (SD 9.1) compared to 33.8 (SD 9.1) online. The mean loss of the ALS-FRS-R value per month (modifi ed delta ALSFRS-R: 48 -ALSFRS-R at ' time of inclusion ' /duration from onset to inclusion) at baseline was 0.77 (SD 0.79), which is slightly slower progression than has been reported in other ALS studies (0.81, 48 -ALSFRS-R at ' time of diagnosis ' /duration from onset to diagnosis) (21). Despite the curvilinear progression of ALS (22), this calculation method is based on a simplifi ed linear progression model of ALS. Median time since symptom onset at baseline was 23 months. After 3.5 months the mean follow-up on-site ALSFRS-R was 31.9 (SD 8.7) compared to online 31.8 (SD 8.7). Correlation between baseline and fi rst online ALS-FRS-R was very high with a coeffi cient of 0.96 ( p Ͻ 0.001) (Figure 2). Agreement between both data-capture methods was very high (Figure 3). The distribution of differences was normal, suggesting there was no systematic directional bias to any differences. The mean difference (bias) was -0.18 and the upper and lower limits of agreement were 4.4 and -4.7, respectively. More than 95% of all pairs of measurement were within the limits of agreement.
A similarly high degree of correlation also existed between the in-clinic follow-up visit and the closest online ALSFRS-R to this visit (0.965; p Ͻ 0.001; Table I. Characteristics of patient population ( n ϭ 127) at baseline.   n ϭ 81). In the Bland-Altman plot for follow-up visits (Figure 4), the mean difference was 0.06 and upper and lower limits of agreement were 4.3 and -4.2. Again, more than 95% of the data were within the limits of agreement. These data also suggest very good agreement in follow-ups. Based on the additional question " who fi lled in the questionnaire " , 22.4% of online surveys were completed by a caregiver. There was a high correlation between the self-administered baseline ALS-FRS-R data and those of the two online groups, with values of 0.95 for the self-administered group ( n ϭ 86) and 0.92 for the caregiver-assisted group ( n ϭ 21) with p Ͻ 0.01. With the Bland-Altman method, no relevant bias was detected in the two groups (data not shown).
Eleven percent of the patients who gave written consent did not complete their online assessment at all. There was no statistically signifi cant bias according to age, disease duration or ALSFRS-R. Only a trend can be seen in gender, as there was a higher percentage of women in the group of non-completers compared to completers (50% vs. 29%, p ϭ 0.075). Table II shows the report on time burden, physical limitation and emotional strain of online selfassessment. When interviewed about the time burden and the emotional and physical strain, more than 95% of the patients who fi lled in the surveys felt that they were ' not at all ' or only ' slightly ' affected by online self-assessment. Three months on, no signifi cant change was observed (Table III: t -test, p Ͼ 0.1).

Discussion
We found very high correlations between ALSFRS-R scores at two time-points administered in-clinic compared to through the internet. There was no evidence of systematic bias towards higher or lower scores online. We also found that patients did not consider online ALSFRS-R entry to be physically or emotionally burdensome, or to be time-consuming. We propose for future development that a time-span between online assessments adapted to a given patient ' s rate of progression could be even more efficient. Completing PROs online could be a way to complement face-to-face visits and manage care in a more personalized and needs-based way, rather than relying upon regular time-intervals such as 3-or 6-month follow-up appointments. Online PROs could also be used to improve the convenience and thereby participation in clinical trials that use the ALSFRS-R as an endpoint.
Our fi ndings must be considered in the context of their limitations. The original ALSFRS-R was designed as a paper-based clinician interview rather than a patient self-report measure, let alone one measured through the internet. However, other studies have found no evidence that self-report (23), caregiver report, or telephone administration significantly degrades the quality of the scores. Furthermore, the nature of the measure (12 items comprising short, clear questions with well-defi ned anchor points for response options) means that there is very little difference in user experience between the paper-pencil method and computerized administration. The move from paper-based PROs (pPROs) to electronic PROs (ePROs) (24) has prompted a number of similar studies to establish equivalence between the two methods in various medical indications and psychometric tests (25 -28). On the whole, our population was relatively early in its disease course and it is unclear how its ability to respond online, or the perceived burden of the ALSFRS-R, would be felt at later stages of the disease -it is plausible that as physical function deteriorates the physical burden would increase to the point that data entry would be diffi cult or impossible. However, these patients also fi nd attendance at clinic diffi cult, and there are numerous adaptations available to operate a computer, such that online data collection may actually increase the representativeness of data that can be collected from clinical centres.
A further limitation was that 11% of the patients who gave written consent did not complete their online assessment at all. We could not show a systemic bias in this group, but it cannot be excluded that factors such as insuffi cient technical requirements or discomfort with submitting data online have infl uence. Finally, our population was predominantly urban, seen at a specialist centre, and located in a technologically advanced country; our fi ndings may not generalize readily to other populations and the issue of the ' digital divide ' should be kept in mind.
Outside clinical practice, online administration of PROs has been in use for ALS patients since 2006 at the social internet platform www.patientslikeme. com. Use of the ALSFRS-R has included an extension to the scale for more disabled patients (3) and even a clinical observational study to test the effect of lithium on ALS progression (29). However, such online assessment preceded validation that collecting outcome data was viable and valid from patients with a clinically confi rmed diagnosis; this study has fulfi lled this important foundational step.
An internet-based assessment of ALSFRS-R may facilitate patient follow-up in the home care environment. This methodical approach contributes to greater density and continuity in the collection of outcomes that otherwise would not be possible to achieve. At the same time, it may save time and reduce costs by integrating online data into the workfl ow of clinic visits. Hence, this online selfassessment is a welcome addition to an electronic Case Report Form (eCRF), the most common tool for data collection in clinical trials. Dividing the internet-submitted FRS score into its component subscores (e.g. speech, walking, arm/hand, respiratory, swallowing) could also support clinicians to estimate the progress of individual symptoms and time their interventions in a needs-based manner. For instance, the fall of the respiratory subscore beneath a critical threshold may indicate the need for timely consultation.
In general, the use of patient-reported outcomes (PRO) such as ALSFRS-R has become more widespread in recent years, particularly in trials dealing with chronic disabling diseases (30,31). In compliance with the FDA standards (32), PROs supplement objective measures or replace them, especially if there is a lack of biomarkers or surrogate endpoints as in ALS. Apart from clinical trials, PRO measurements are aimed at improving patient care by intensifying the patients ' involvement and considering their perspective on the disease. A study of Patient-sLikeMe ' s users suggested better health literacy and communication with their healthcare providers as a result of tracking their progress with PROs (33). Our study supports the notion that online self-assessment is a practicable way to integrate the patient in managing their care.
Future research should establish and quantify the potential for resource savings and improved patient outcomes in incorporating online data collection into clinical management.