Rasch analysis for patient-reported outcome measures in OSD

There has recently been a tremendous increase in the use of questionnaires, or patientreported outcome instruments, for the evaluation of the impact of various health conditions on people as well as the effectiveness of various treatments. Clearly, there has been an acknowledgment in the healthcare community that questionnaires, which account for the patient's perspective, complement other objective measures. Patientreported outcome measures can help us to quantify the impact of ocular surface disease (OSD) on an individual, providing information not necessarily obvious from clinical signs. They can be used as outcome measures in trials of new drugs or other treatments, or as indicators of how well new diagnostic tests correlate with patient symptoms.

Certainly, researchers and practitioners in OSD have joined in this movement, and patient-reported outcome measures are in wide use in the field. The report of the International Dry Eye Workshop¹ paid special attention to the properties of existing questionnaires and called for more psychometric work in the area to better understand the role that questionnaires may play in determining disease severity and sensitivity to change. Recently, the report of the International Workshop on Meibomian Gland Dysfunction called for a validated questionnaire for assessment of symptoms of meibomian gland dysfunction.²

Modern psychometric techniques for evaluation of questionnaires

An alternative method for the evaluation and scoring of patient-reported outcome instruments is the use of the Rasch model,⁴ which produces intervallevel data from the raw responses and offers a number of other advantages in the evaluation of individual questionnaires. These include the ability to assess how well a questionnaire measures a single construct, how well it discriminates between people with different amounts of the construct it is intended to measure, and whether the response category structure of the questionnaire is valid. Excellent reviews of the use of Rasch analysis in the evaluation of questionnaires have been published.⁵

Rasch analysis in OSD

Recently, several questionnaires for use in OSD have been evaluated using Rasch analysis. In an evaluation of the McMonnies Questionnaire⁶ Gothwal et al.⁷ found that it functioned as a useful screening tool. However, they found that the questionnaire did not serve as a useful measure of dry eye severity - that is it was not able to adequately discriminate between more than two levels of disease severity. This determination was made using the person separation index statistic from the Rasch analysis.

The Ocular Comfort Index⁸ is unique in that it was developed from the start using Rasch analysis. It contains twelve questions regarding the frequency and severity of ocular surface irritation. The authors began with a set of questions and used Rasch item fit statistics to guide selection of a subset for inclusion in the final questionnaire. The resulting questionnaire demonstrated good ability to separate between different levels of disease severity, good response category function, and evidence that only one construct was being measured.

Perhaps the most widelyused questionnaire in the field is the Ocular Surface Disease Index (OSDI).⁹ The OSDI consists of 12 questions arranged in three subscales dealing with symptoms, visual function, and environmental triggers. We recently published the results of an investigation of this questionnaire using Rasch analysis.¹⁰ We found that some of the response category options should be collapsed, resulting in a three category response structure rather than the original five category structure. All of the items fit the model adequately, but there was evidence of multidimensionality (more than one construct was being measured). This is problematic, in that the use of a summary score implies that only one construct is being measured, and the interpretability of a score is limited by multidimensionality.