Background: The purpose of this study is to use Rasch analysis to explore the validity of considering self-report scores from Wisconsin Upper Respiratory Symptom Survey (WURSS-21) as a single global illness severity domain. The WURSS-21 is a widely used questionnaire instrument that assesses symptom severity and functional impact of common cold and flu-like illness.

Methods: This study applies item response theory, specifically Rasch modeling, to investigate dimensional and measurement properties of the WURSS-21, and looks at invariance over time. The data assessed represents 1167 people, each scoring the WURSS-21 once daily for up to seven consecutive days of acute upper respiratory infection (URI) illness.

Results: Rasch analysis supports a single domain WURSS-21 global symptom score. Assessment of differential item functioning across seven days of illness provides evidence for measurement invariance. While individual items rating physical symptoms were somewhat variable, items rating functional impairment and quality of life impact appeared quite consistent across a single domain over seven days of illness.

Conclusion: Rasch analysis of WURSS-21 items provides evidential support for a single invariant domain. These findings support the practice of using a simply summed daily global illness severity score to represent the overall symptomatic and functional impairments arising from URI.

Keywords: common cold, patient reported outcomes, quality of life, URI, validation

WURSS, Wisconsin upper respiratory symptom survey; URI, acute upper respiratory infection; CTT, classical test theory; IRT, item response theory; RSM, rating scale model; MNSQ, mean square; PCA, principal components analysis; DIF, differential item functioning

Acute upper respiratory infection (URI) illness is a clinical syndrome produced from viral infection of the upper respiratory tract. A wide variety of etiological agents are involved, including rhinovirus, coronavirus, adenovirus, influenza, parainfluenza and respiratory syncytial virus.¹ Influenza is often classified separately, but usually causes an illness syndrome very similar to other URIs.² In the U.S., non-influenza URI has an estimated annual health cost of $40 billion, including 40million missed work and school days.³ In a single year, influenza can lead to 31million outpatient visits, 3million hospitalizations, 610,000 life-years lost, and an estimated economic impact as high as $87 billion.⁴

The WURSS-21 is a valid and reliable self-report research tool incorporating specific symptoms and functional impairments common to URI illness.^5–7 Ten specific symptoms assessed include runny nose, plugged nose, sneezing, sore throat, scratchy throat, cough, hoarseness, head congestion, chest congestion, and feeling tired. The instrument also includes nine functional items rating ability to think clearly, sleep well, breathe easily, exercise, work inside and outside the home, accomplish daily activities, interact with others and live one’s personal life. An introductory item rates overall illness severity, and a concluding item rates change-since-yesterday. All items are scored on an 8-point Likert scales from 0 (absent or no impairment) through 1(very mild), 3(mild), 5(moderate) and 7(severe) (Table 1).

The original WURSS was developed using individual face-to-face interviews and focus groups among people recruited from the community⁸ with Jackson-defined colds.⁹ Semi-structured interviews included open-ended questions aimed at eliciting terminology for assessing symptoms and quality of life values related to experienced cold illness. Of more than 150 terms used to define symptomatic or functional impairment, 42 were chosen for the original WURSS instrument. Adding an introductory global severity item (How sick do you feel today?) and a concluding daily change item (Compared to yesterday, I feel that my cold is…..) led to the WURSS-44.⁸ Subsequently, assessment of item-level assessment of responsiveness and importance to patients produced the WURSS-21.⁵ The WURSS-44 and WURSS-21 have been independently validated.^5,7 Desire to reduce the time and burden associated with WURSS-21 completion led to the development of the WURSS-11 with similar dimensional construct.⁶

While the initial validation of the WURSS-44 using factor analysis based on classical test theory suggested a 10-dimensional structure,⁷ and the WURSS-21 to WURSS-11 derivation work indicated a 3-dimensional structure,⁶ we have generally recommended using a simple sum of the 19 items as the most appropriate daily global severity score. Numerous studies have used this simple sum global severity WURSS-21 score, including several NIH-sponsored randomized clinical trials.^10–15 To date, investigators at more than 250 institutions in more than 50 countries have registered to use one or more versions of WURSS. For non-profit and educational purposes, use of WURSS is free. License fees for commercial use go through the Wisconsin Alumni Research Foundation.

This current study uses Rasch analysis to explore the validity of the common practice of using a global score, and assesses invariance of the measure across the 7-day timeline of a typical URI illness. Our purpose is therefore to use the Rasch item response theory method to assess the validity of treating the WURSS-21 as a single global measurement domain.¹⁶

Data sources

Data for this paper come from four studies using the WURSS-21 instrument. These include the WURSS-44 validation (WURSS-21 development) study,⁷ the WURSS-21 validation study,⁵ and two clinical trials that used the WURSS-21.^11,12 Together these total a cohort of n=1167 people with URI illnesses self-reported daily on the WURSS-21, with sample size decreasing over time as people recover from illness: day1=1167, day2=1157, day3=1153, day4=1144, day5=1112, day6=1048, and day7=945. Inclusion of a viable URI illness was defined as having self-identified common cold, at least one nasal or throat symptom, and a score ≥2 points on the Jackson scale.⁹ Histories of allergy and asthma were reasons for exclusion if active symptoms were observed at enrollment. Use of antibiotics or immune related medication was also reason for exclusion. The end of the illness was confirmed by at least two subsequent days of not having symptoms. All study protocols were approved and monitored by the University of Wisconsin-Madison Institutional Review Board.

Statistical analysis approach

While Classical test Theory (CTT) was originally used in assessing psychometric properties of the WURSS-21, this method is limited by its focus on whole test assessments of multi-dimensional structure, and by the assumption that items are equally difficult for participants to understand and respond to. The Item Response Theory (IRT) approach may be more appropriate, as it assesses properties of the individual items across populations and within individuals over time, allowing for differential item difficulties.^16–19 IRT offers important advantages over CTT, especially when employing the Rasch Model.¹⁶ Rasch analysis, in contrast to the CTT approach, allows for the assumption that the set of symptom items is intended to measure a single domain. This fits with assessment of URI illness episodes, usually considered discrete events, and is consistent with the operational basis of the WURSS-21, which is used to provide a single score for each day of illness. It is not uncommon to first use a CTT factor analysis model to explore item-domain structure, followed by a Rasch-based IRT analysis to assess the quality of the items in the larger unitary domain.²⁰

Rasch model

A Rasch Model^16,21,22 was used to assess person and item reliability, item statistics and ordering of response categories, using the WINSTEP software Version 3.80.²³ A special case of the Rasch Model for use with polytomously scored items, known as the Rating Scale Model (RSM), was employed. The RSM assumes all items are equally discriminating and have the same number of response categories and estimates a person’s probability of responding to a certain item category.¹⁹ The model may be written as:

$Prob\left\{X_{vi}=x|{\theta }_v,{\beta }_i,{\omega }_h\right\}=\frac{\exp \left[{\displaystyle {\sum }_{h=0}^x\left({\theta }_v-\left({\beta }_i-{\omega }_h\right)\right)}\right]}{{\displaystyle {\sum }_{x=0}^m\exp \left[{\displaystyle {\sum }_{h=0}^x\left({\theta }_v-\left({\beta }_i-{\omega }_h\right)\right)}\right]}}$ (1)

Where $\upsilon =1, \dots , n$ are the respondents, i=1, …., k are the items, h=0,…,m-1 are the number of thresholds, x=0,…,m are the response categories, and Xυi is the response vector for each respondent. The parameters of θ and β represent the respondent and item parameters respectively, with the parameter ωh the common set of thresholds applied to all items.

Two fit indices were used to investigate item concurrence with the overall symptom domain, which were the mean square (MNSQ) outfit and in fit. The outfit MNSQ measures the average mismatch between the Rasch model and the data, and is sensitive to extreme values. The infit MNSQ is more sensitive to patterns of responses to items targeted for the subject matter. The expected value for both outfit and infit MNSQ is 1, with a range of values from 0 to infinity. Values near 1 indicate little distortion of the measurement system, and values greater than 2 indicates that the item fails to define the same construct as the other items do in a domain, and degrades the measurement. MNSQ values lower than 0.5 may be an indication of item redundancy with values 0.5 to 1.5 considered satisfactory.^24,25 In addition to assessing item-domain integrity, these measures may help detect problematic symptom items.

An underlying assumption of the RSM approach is uni-dimensionality,¹⁶ in that the symptom items measure only a single domain. To test this assumption, we used a post-hoc approach based on a principal component analysis (PCA) conducted on the standardized residuals produced from the RSM.²⁶ When there is the presence of a dominant factor with over 20% of variance explained, Reckase²⁷ suggests that the unidimensional assumption may be considered acceptable. In conjunction with RSM, confirmatory factor analysis was also employed to assess the assumption of uni-dimensionality.

Invariance over time

A major challenge in conducting longitudinal assessments is the possibility that measures developed for a given domain at one particular time may not be assessing the same domain at other points in time or differential item functioning (DIF). DIF refers to the condition in which an item displays different properties at different time periods after controlling for the abilities of the groups.²⁸ Investigating invariance over time assesses whether the WURSS-21 measures the same underlying symptom severity domain across the duration of URI illness. This allows both comparability and a meaningful interpretation of respondents’ symptom severity scores in longitudinal studies. Assessment of measurement invariance in the RSM context can be conceptualized as asking whether item parameters are applicable to the multiple assessments over time, and whether individual items have stable relationships to the domain of interest across longitudinal time measures.²⁹ When self-reports occur over multiple time periods, response dependency may impact underlying RSM assumptions.³⁰ In order to construct a “repetition-bias-free” RSM for multi-item longitudinal instruments, the first time measurement can be considered the benchmark, with subsequent time points randomized. Since the measurement framework is anchored, this controls for within-person over-time dependency, allowing all time-points to be analyzed together.

The WURSS-21 data were obtained from four studies (n=1167 total) spanning the years 2002-2010, as outlined above. The characteristics of the participants are presented in (Table 2).

Rasch model results

Rasch model principal components analysis using standardized residuals for data across all seven days showed that a single dominant factor explained 57% of total variance. When a dominant factor explains over 20% of variance, Reckase²⁷ suggests the use of uni dimensional RSM model. The single domain was also supported by confirmatory factor analysis, providing fit indices of 0.924 for the confirmatory factor index, and 0.915 for the Tucker Lewis Index.

IRT analysis was performed to estimate the goodness-of-fit (infit and outfit) of the observed data to the model-expected data and the item symptom rarity of the 19 items from WURSS-21 (Table 3). The fit indices determine how well each item contributes to a single common construct. The infit MNSQ index is more sensitive to unexpected responses to an item near a person’s ability level, and the outfit MNSQ index is sensitive to unexpected responses to more distant items.³¹ According to Wright and Linacre,²⁵ item fit indices of 1.0 are ideal, and values between 0.5 and 1.5 considered satisfactory indications of model-data fit.^24,32 While typical Rasch modeling uses the terminology of difficult/easy items in the assessment of an instrument, Linacre³³ suggests adjusting the terminology for symptom measures as rarely observed (=difficult item) and often observed (=easy item), which is appropriate here, as the WURSS-21 is a symptom instrument.

Strongly supporting unidimensional integrity, all of the infit MNSQ over time (19 items over seven days) were in the productive range of 0.5 to 1.5 (Table 3). Outfit statistics were also strongly supportive of this model. The exception was the symptom item of “Runny nose,” item I1, indicated rarity later on in the progression of the cold, as indicated by higher outfit MNSQ values. Although rare (difficult) in that specific time frame, it was still considered a productive (useful) item in the assessment of the overall domain.

The log odds of the probability (item rarity) are shown for each of the seven days, reflecting the rarity for an individual to assess the symptom (Figure 1). Higher scores indicate rarer observations with a symptom, and lower scores indicate a more prevalent occurrence of the symptom. The symptom that respondents indicated most problematic (rare) in the early stages of the cold (days 1-3) was chest congestion (I9), with the most highly observed (prevalent, easy, useful) symptom being feeling tired (I10). The pattern of symptom observation (rare vs prevalent) across each of the seven days of the illness was strikingly similar (Figure 1).

Figure 1 Log odds of the probability (Item rarity) for each day.

Time invariance

Rasch modeling was used to assess item equivalence or DIF over the span of seven days of the URI.²⁸ The requirement that an instrument works invariantly across time ensures that changes in symptom measures reflect real improvements in experienced symptoms and not just differences in the measurement of the symptoms. Figure 2 show the DIF size (difference between the individual day log odds and an overall log odds), indicating that more item difference across the seven days of the URI was noticed for the physical symptoms of sneezing (I3), cough (I6), chest congestion (I9), etc. and least different across time for the more social-type of symptoms, (e.g., feeling tired (I10), interacting with others (I18). While more symptom differences were encountered with the reporting of physical symptoms (I1-I9) rather than functional impairments (I10-I19), all differences were considered reasonable for an assessment of item invariance across the seven day time period (Figure 2). It was noteworthy that participants were variable relative to the symptom of cough (I6), with the symptom reported as rare initially in the progression of the illness (days 1, 2, and 3), but then becoming more prevalent latter on (days 4-7).

Figure 2 URL Item symptom difference across days of the cold.

Participant and symptom map

The map of items and individual responders (participants) is illustrated in Figures 3A-3C (maps per day). Ideally, location values of each symptom item in the WURSS-21 should cover the continuum, with their distribution sufficiently wide to collect the variability of the URI symptoms. These figures show that both participant and item distributions do provide sufficient variability. The participant-symptom map here shows the distribution of participant scores (left side), and the symptom item level of observation (right side) for the single domain WURSS-21. Participants with higher WURSS-21 score and items with “rarely observed symptoms” are located on the positive side of the graphic, at the top of the map. One may observe the decline in symptoms as the cold progresses from day 1 to day 7, as shown in changes of WURSS-21 score distributions (left side of the figure).

Figure 3a Patient and Item Symptom Maps.

Figure 3b Patient and Item Symptom Maps.

Figure 3c Patient and Item Symptom Maps.

From an item standpoint, one may see that symptom item I9 (chest congestion) was rarely observed early on in the cold, especially the first three days. Whereas, item I10 (feeling tired), was reported more frequently, and was consistent throughout the seven day period.

The use of the item response theory Rasch model provided useful detailed insight as well as support for a single domain WURSS-21 symptom score. Assessment of DIF scores provided evidence for measurement invariance across the first seven days of the illness. While physical symptoms (items I1-I9) were slightly more variable relative to occurrence in the unitary domain structure, the functional and quality of life responses appeared very consistent in the domain over the seven days of illness. In general, analysis of individual items provided strong evidential support for an invariant domain measure. This supports the use of a simply-summed global severity score for the WURSS-21, consistent with its common use.

The authors would like to thank their colleagues, departments, institutions, and especially the research participants who provided the data used here by filling out the WURSS-21 questionnaire every day during a time of illness. Throughout the writing of this paper, Bruce Barrett was supported by a midcareer investigator award (K24AT006543) from the National Center for Complementary and Alternative Medicine (NCCAM). Data from two trials sponsored by NCCAM contributed to the data sets analyzed here (R01AT001428; R01AT004313). The data set was compiled by Chidi Obasi and initial analyses were conducted while he was supported by a National Research Service Award from the Health Resources and Services Administration (T32HP10010).

All authors contributed substantially to the development of this manuscript and approved the final version of this work.

All authors contributed substantially to the development of this manuscript and approved the final version of this work.

1. Monto AS. Epidemiology of viral respiratory infections. Am J Med. 2002;22(112 Suppl 6A):4S–12S.
2. Eccles R. Understanding the symptoms of the common cold and influenza. Lancet Infect Dis. 2005;5(11):718–725.
3. Fendrick A, Monto A, Nightengale B, et al. The economic burden of non–influenza–related viral respiratory tract infection in the United States. Arch Intern Med. 2003;163(4):487–494.
4. Molinari NA, Ortega SIR, Messonnier ML, et al. The annual impact of seasonal influenza in the US: measuring disease burden and costs. 2007;25(27):5086–5096.
5. Barrett B, Brown RL, Mundt MP, et al. Validation of a short form Wisconsin upper respiratory symptom survey (WURSS–21). Health Qual Life Outcomes. 2009;12(7):76.
6. Obasi CN, Brown RL, Barrett BP. Item reduction of the Wisconsin upper respiratory symptom survey (WURSS–21) leads to the WURSS–11. Qual Life Res. 2013;23(4):1293–1298.
7. Barrett B, Brown R, Mundt M, et al. The Wisconsin Upper Respiratory Symptom Survey is responsive, reliable, and valid. J Clin Epidemiol. 2005;58(6):609–617.
8. Barrett B, Locken K, Maberry R, et al. The Wisconsin upper respiratory symptom survey (WURSS): a new research instrument for assessing the common cold. J Fam Pract. 2002;51(3):265.
9. Jackson GG, Dowling HF, Muldoon RL. Acute respiratory diseases of viral etiology. VII. Present concepts of the common cold. Am J Public Health Nations Health. 1962;52:940–945.
10. Heinz SA, Henson DA, Austin MD, et al. Quercetin supplementation and upper respiratory tract infection: A randomized community clinical trial. Pharmacol Res. 2010;62(3):237–242.
11. Barrett B, Brown R, Rakel D, et al. Placebo effects and the common cold:a randomized controlled trial. Ann Fam Med. 2011;9(4):312–322.
12. Barrett B, Hayney MS, Muller D, et al. Meditation or exercise for preventing acute respiratory infection: a randomized controlled trial. Ann Fam Med. 2012;10(4):337–346.
13. Murdoch DR, Slow S, Chambers ST, et al. Effect of vitamin D3 supplementation on upper respiratory tract infections in healthy adults:the VIDARIS randomized controlled trial. JAMA. 2012;308(13):1333–1339.
14. Byun JS, Yang SY, Jeong IC, et al. Effects of So–cheong–ryong–tang and Yeon–gyo–pae–dok–san on the common cold:randomized, double blind, placebo controlled trial. J Ethnopharmacol. 2011;133(2):642–646.
15. Tiralongo E, Lea RA, Wee SS, et al. Randomised, double blind, placebo–controlled trial of echinacea supplementation in air travellers. Evid Based Complement Alternat Med. 2012;2012:1–9.
16. Georg Rasch. Probabilistic Models for Some Intelligence and Attainment Tests. 1992.
17. De Boeck P, Wilson M. Explanatory item response models: a generalized linear and nonlinear approach. Journal of Educational Measurement. 2005;42(3):303–307.
18. Lord FM. Applications of item response theory to practical testing problems. Journal of Educational Measurement. 1981;18(3):178–180.
19. Embretson SE, Reise SP. Item Response Theory for Psychologists. Lawrence Erlbaum Associates. 2000. 384 p.
20. Randal J, Engelhard G. Using confirmatory factor analysis and the Rasch model to assess measurement invariance in a high stakes reading assessment. Applied Measurement in Education. 2010;23(3):286–306.
21. Andrich D. Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement. 1978;2(4):581–594.
22. Andrich D. Rating formulation for ordered response categories. Psychometrika. 1978;43(4):561–573.
23. Linacre JM. Winsteps® Rasch measurement computer program. Beaverton; 2013.
24. Linacre JM. What do Infit and Outfit, Mean–square and Standardized mean? Rasch Measurement Transactions. 2002;16(2):878.
25. Wright BD, Linacre JM. Reasonable mean–square fit values. Rasch Measurement Transactions. 1994;8(3):370.
26. Brentari E, Golia S. Unidimensionality in the Rasch Model: How to detect and interpret. Statistica. 2007;67(3):253–261.
27. Reckase MD. Unifactor latent trait models applied to multi–factor tests: results and implications. Journal of Educational Statistics. 1979;4(3):207–230.
28. Angoff WH. Perspectives on differential item functioning methodology. 1992.
29. Reise S, Widman K, Pugh R. Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychol Bull. 1993;114(3):552–566.
30. Chien TW. Repeated Measure Designs (Time Series) and Rasch. Rasch Measurement Transactions. 2008;22(3):1171.
31. Piquero NL, MacIntosh R, Hickman M. Applying Rasch modeling to the validity of a control balance scale. Journal of Criminal Justice. 2001;29(6):493–505.
32. Smith RM, Schumacker, Bush MJ. Using item mean squares to evaluate fit to the Rasch model. J Outcome Meas. 1998;2(1):66–78.
33. Linacre JM. Rasch Measurment Forum; 2013.