Aim: To examine the validity and reliability of HSCL-25 through using CFA. 

Methods: A non-experimental survey design was used to examine the anxiety and depression of doctors (n=547) using online and paper-and-pencil methods with the use of N-HSCL-25. Confirmatory factor analysis (CFA) and other applicable psychometric tests were carried out. 

Results: A positive and strong correlation between anxiety and depression were measured with a correlation coefficient value of .88. The reliability scores of Cronbach alpha for anxiety and depression were 0.890 and 0.926 respectively. Three items, including a suicide screening item, of the depression sub-scale, and two items of the anxiety sub-scale had to extricate due to poor factor loadings. Post-hoc CFA verified Nepali version of HSCL-20, bi-factor model, with model fit scores for SMRs, RMR, and RMSEA less than 0.08. The model fit scores of NFI, TLI, AGFI, and CFI were above .90. 

Conclusions: CFA confirmed N-HSCL-20 with validity and reliability indicators for further use in the Nepali population. The study recommends using a separate suicide screening tool in the Nepali context.

Keywords: HSCL-25, Nepali version, anxiety, depression, suicide, confirmatory factor analysis

Hopkins Symptoms Checklist (HSCL-25), a measure for anxiety and depression, is widely used as a screening tool for clinical and non-clinical settings. HSCL-25 is a short self-reporting measure derived from HSCL-90.¹ Thapa and Hauff² applied HSCL-25 to the Nepali population with translation, psychometric analysis, and cultural adaptations. The content analysis and cultural adaptations HSCL-25 was further explored with the Nepali-speaking Bhutanese refugees resettled in the USA.³ The first author used the Nepali version of HSCL-25 to understand the prevalence and address the distress of the conflict-affected population and corporate staff in Nepal.^4,5 There were no studies with the use of Confirmatory Factor Analysis (CFA) applied to the HSCL-25 Nepali version. This study explores the reliability, validity and unimodality of HSCL-25 and discusses with proper recommendations for future use of the tool in the Nepali population.

A cross-sectional design was applied to measure the anxiety and depression of Nepali doctors. The data was collected through online and paper-pencil methods. The inclusion criteria were NMC registration, and ongoing practice as a doctor, whereas doctors who discontinued their practice due to mental health, health, or retirement reasons were excluded from the study. Out of the Nepal Medical Council (NMC) registered doctors, 557 doctors, 415 males (75.90 %) and 129 females (23.60 %) respectively, responded the survey with 62.80% response rate. Three respondents (0.5%) responded to no gender option. The author used the Nepali version of HSCL-25 to understand the prevalence of distress in the Nepali doctor’s population and carried out CFA to test the reliability, validity, and unimodality of the tool. Ethical approval from two institutions, the Austrian Academy of Psychology (AAP) and the Nepal Health Research Council (NHRC), were acquired before collecting the data. The data was coded, cleaned, and analyzed through using SPSS-25 and SPSS AMOS 25 versions. Normality, homogeneity of the variances, linearity and multicollinearity, and co-variances were tested before a detailed analysis. Apart from the correlations among the variables, Levene’s test for the homogeneity of variances was also conducted. Descriptive statistical analysis was applied to measure the means and standard deviations of continuous variables. The analysis was carried out using SPSS-25 version. The significant testing for normality followed the distribution checks via scatterplots, box plots, kurtosis, and skewness. A Pearson's product-moment correlation was analyzed after testing the relevant assumptions for the correlation.

The Cronbach alpha for anxiety and depression disorders were .890 and .926 respectively. The mean scores of the population for anxiety and depression distress were 16.15 (SD = 5.16), 13.16 (SD = 7.93), and 6.92 (SD = 5.18). The HSCL-25 cut-off scores were 1.75 for each item of anxiety and depression.² After applying the cut-off criteria of 17.50 for anxiety and 26.25 for depression, it was found that the prevalence of anxiety and depression among medical doctors were 30.89 % and 25.41 % respectively. 19.56% medical doctors scored, above the cut-offs, for both anxiety and depression. 5.7 % doctors expressed their rigorous intention of suicide. Of the latter, 3.7% claimed to have suicidal thoughts ‘a lot’ and 2%, ‘extremely high’ in the last two weeks before responding. The prevalence and impact of depression were higher on female doctors, doctors below the age of 25 and undergraduate doctors than on other groups. Equally, doctors below 25 years of age, undergraduate doctors, and doctors working outside of Kathmandu valley were highly affected with anxiety.

Confirmatory factor analysis of HSCL-25 Nepali version

The proposed model of the Nepali version of HSCL-25 (N-HSCL-25) was tested by applying the CFA. Two measures, anxiety and depression, with 10 and 15 items respectively, were considered for the assessment purpose. The hypothetical model was expected to fit with the set of collected data. As done in previous studies, a two-factor model of HSCL-25 was applied.^6–8 A measurement model with all the collected data for each item of the two measures was taken from the 25th version of SPSS AMOS software.

CFA and outcomes

Descriptive statistics of N-HSCL-25 were calculated. The reliability scores of Cronbach alpha for anxiety and depression were 0.890 and 0.926 respectively. These are very good. The mean scores of anxiety and depression for all the data were M = 16.15 (SD = 5.15) and M = 23.16 (SD = 7.93). Field⁹ analyzed and suggested that Kolmogorov Smirnov and Shapiro-Wilk test results are only accurate for the smaller sample size studies. In large samples, the researcher has to analyze the normality based on the visual inspection of histograms, Q-Q plots, box plots, and interpretation of skewness and kurtosis scores. Histograms, normal Q-Q plots, box plots, scatter plots, Mahalanobis’ distances for the DVs and residuals were reviewed. The skewness and kurtosis were tested for all the items of both the anxiety and depression measures. The kurtosis values of depression were found slightly positively deviated. The quadratic Mahalanobis’ distance (MD²) was used to outline possible outliers. Some outliers were identified, but there were no major differences even after they were deleted. Therefore, the normality of the data was ensured. CFA for N-HSL-25 was run for a two-factor model as was done in previous studies.^6,7 The two-factor model of HSCL-25 includes 10 items for anxiety and 15 items for depression. Details on the coding and items for each factor are presented below in Table 1.

Gerbing and Anderson¹⁰ demonstrate that an exploratory analysis with item correlation and factor analysis is not sufficient to evaluate the unimodality of any measure. CFA, however, can assess both the model fit of a measurement model and unimodality. Hair et al.¹¹ emphasize that the model identification process in a SEM technique provides adequate pieces of information to offer a solution for a set of structural equations. Byrne¹² proposes checking the number of degree of freedom for any proposed measurement model. The hypothesized CFA of HSCL-25 output showed a total of 325 distinct sample moments identified from a sample covariance matrix. A of total of 78 parameters were found in the model. Of them, 27 were fixed and 51 free to be estimated. The degree of freedom was 274 with chi-square value of 1219.17. The model was over-identified, as the probability level of the data was non-significant. However, according to Heir at al. (2005), having an adequately large sample size influences the degree of freedom, and the proposed model can still be tested to determine the reliability of its results through fit indices.

While doing the analysis, the loading for one of the indicators in each measure is fixed to 1.0 so that the scale for Latent variables can be generated automatically in AMOS. The initial measurement model of HSCL-25 based on a two-factor combination, anxiety and depression, for CFA is presented above (Figure 1). The absolute goodness-of -fit measures of the HSCL-25 initial measurement model were acquired through analysis. They are presented in Table 2.

Figure 1 Initial Measurement Model for CFA of N-HSCL-25.

The fit indices for the measurement model should meet the minimum acceptable criteria based on the table prescribed by Ahmad, Zulkurnain, and Khairushalimi.¹³ For a proper fit of the model, fit indices scores for SMRs, RMR, and RMSEA should be less than 0.08, and the scores of NFI, TLI, AGFI, and CFI should be above .90. The summary of fit indices for the initial measurement model of HSCL-25 of the current study, including CMIN/df, NFI, TLI, GFI, and CFI, did not yield an adequate model fit for the given empirical data. The p-value for the chi-square score was 1219.17, which is non-significant for the given empirical data to justify the observed covariance matrix with the estimated covariance matrix. The model fit indices presented above also have to be investigated based on the sensitivity of the sample size to the chi-square statistical test (Bergh, 2015).

Besides the sample size issues, the factor loading of each item has to be examined to understand the best fit of the model with the given data. The factor loadings for all the items of each construct were above .63 except for two items measuring anxiety (i.e. ANX_6, and ANX_8) and three measuring depression (DEP_4, DEP_10, and DEP_15). The values of squared multiple correlations (R²) for the items were above 0.4 except for two items measuring anxiety (ANX_6, and ANX_8) and three items measuring depression (DEP_4, DEP_10, and DEP_15). Awang (2012) recommends omitting any item having a factor loading value less than 0.6 and a R² value less than 0.4 from the measurement model.

Post-Hoc analysis of CFA for N-HSCL-25

The post-hoc analysis of the measurement model of HSCL-25 with two items of anxiety and three items of depression omitted was carried out. From the parameter summary obtained from AMOS results for the default model, 210 sample moments of parameters were drawn. Of them, 44 distinct parameters were free to be estimated. The degree of freedom was measured as 166. The chi-square value was 499.20 with p<.001. Again, the given empirical data could not justify the observed covariance matrix with the estimated covariance matrix, a result common in studies with large sample sizes. Therefore, the proposed model can still be tested to observe the reliability of the results through fit indices.¹² The measurement model of the HSCL-25 with the post-hoc CFA test is presented in Figure 2.

Figure 2 Post-Hoc Test of Measurement Model for CFA of N-HSCL-25.

Goodness-of-Fit results for the CFA of N-HSCL-20

The post-hoc analysis of CFA for HSCL-25 requires that all the items of both the anxiety and depression measures have factor loading values over .63. The value of R² from the measurement model was over .40 for each item. The correlation between anxiety and depression was measured as strong and positive, with a correlation coefficient value of .88. Three items of depression, DEP_2, DEP_3, and DEP_7, were redundant, as were two items of anxiety, ANX_4 and ANX_5. There were three redundant values for interrelationships for redundant items, which are presented in Figure 2. After these adjustments, the goodness-of-fit indices for the post-hoc CFA test for N-HSCL-20 Nepali were calculated and presented in Table 3.

The fit indices of CFA for the measurement model of HSCL-25 were good fit scores for absolute fit indices (RMSEA, GFI, etc.), incremental fit indices (NFI, TLI, CFI, RMR, SMRs, etc.) and parsimonious fit indices (CMIN/df). Fit indices scores for SMRs, RMR, and RMSEA were less than 0.08, and the scores of NFI, TLI, AGFI, and CFI were above .90. The fit indices scores presented are statistically justifiable to accept the model fit.^13–15

Unimodality, validity and reliability of N-HSCL-20

Unimodality: To confirm the unimodality of any scale, factor loading values must be above 0.5 and positive for each item. As Table 3 shows, all items of each construct met these criteria. Thus, the unimodality of the anxiety and depression measures of N-HSCL-20 Nepali is confirmed, and further exploration of their reliability and validity is recommended.

Reliability: A summary of each construct and item with its factor loading, reliability coefficient (Cronbach alpha), CR, and AVE is presented below (Table 3). To achieve construct reliability, the values of CR and AVE should be greater than 0.6 and 0.5 respectively. The AVE scores for anxiety and depression measures were 0.479 and 0.488 respectively.

Fornell and Larcker¹⁶ advised considering scores slightly below 0.5 AVE when the CR vales are above 0.60. In this case, the CR values for anxiety and depression measures were far better than 0.60: they were 0.880 and .919 respectively. The internal consistency of HSCL-20 was measured through a reliability coefficient, Cronbach alpha (α), whose value for the overall construct was .944. Similarly, the reliability score for each construct was measured; each should be above 0.7. The Cronbach alpha (α) score of HSCL-25 for anxiety was 0.880 (p = 0.05) and that for depression was .919 (p = 0.05). The internal consistency for the anxiety and depression sub-scales of HSCL-20 is in line with the previous studies done in Nepal.² After evaluating all reliability measures, the reliability of the HSCL-20 scale with two subscales, anxiety and depression, was confirmed. Validity: As explained in Section 5.3.4, convergent validity is achieved when all the items of a scale are statistically significant. The analysis results did show statistically significant results for all the items of both subscales, i.e. anxiety and depression, but the AVE scores were slightly below the prescribed threshold of 0.5 (Table 4).

Some researchers, such as Fornell and Larcker¹⁶ and Ping¹⁷ recommend accepting AVE scores slightly below 0.5 when CR is above 0.60 or when a study is done for the first time in the studied population in a given context. Construct validity is established when the fitness level of indices meets the acceptable criteria. The model fit indices of the measurement model of HSCL-20 met all the recommended fit indices which are, a factor loading value less than 0.6 and a R² value less than 0.4 for each item.¹⁸ As recommended by Hair, Black, Babin, Anderson, and Tatham,¹⁹ the convergent validity can be explained and verified through computing and analysing the values of CR and AVE of each measure. The values of CR of each construct should be above 0.7 and the CR score for each measure should be greater than its AVE score.¹⁹ Discriminant validity is confirmed when both MSV and ASV are greater than AVE.^11,19 The scores of AVE, ASV and MSV presented in Table 5 show that, for HSCL-20, MSV<AVE and ASV<AVE; thus, the discriminant validity of the scale is confirmed.

N-HSCL-25 was applied to measure anxiety and depression. N-HSCL-25 was contextualised in the Nepali speaking population earlier by Thapa and Hauff² and Baird³ In both studies, the original 10 items for anxiety and 15 items for depression were retained by their statistical analysis results for which an exploratory factor analysis was applied. The present study was probably the first study in the Nepali population in which CFA was used to examine and test the factors of the measures of HSCL-25.

As per the qualitative evaluation of the translation and cultural adaptation of HSCL-25, Baird, LeMaster, and Harding³ explain that ANX_6 (trembling) item was very difficult to translate exactly into Nepal. Often the meaning of the word ‘trembling’ was difficult to understand. Wind and Knipscheer⁸ reviewed the available literatures on the perception of headache as a part of anxiety cluster or not. Their study concluded, headache is considered as a component of anxiety or depression in the western contexts. In their study with the non-western refugee population, headache is explored neither the part of a depression nor an anxiety scale. There is no in-depth study on this topic in Nepal. The headache (ANX_8) could not be retained and had to take out due to lower factor loading in this study.

Similarly, DEP_4, which asks respondents to rate ‘interest in sex/sexual desire’ was not well responded by participants, probably due to sex-related taboos in Nepali context. DEP_10, which reads ‘thoughts of ending your life,’ is also culturally difficult to feel. Many people have no hope but are afraid of ending their lives and do not express such suicidal thoughts. DEP_15 item is also difficult to translate: the word ‘worthlessness’ has been also misunderstood in the Nepali context. People internalize it as ‘I did nothing’ rather than ‘every effort I made was meaningless or had no positive results.’ These interpretations, which were provided by the Nepali language-speaking Bhutanese refugee population in Eastern Nepal, may also be reflected among the population studied in this study. The study of Baird, LeMaster, and Harding³ suggests that the low factor loading values of these items need further explorations. Thus, a post-hoc CFA was conducted after deleting the problematic items (i.e. two from the anxiety measure and three from the depression measure) for the final testing.

The prescribed HSCL-20 scale includes 12 items for depression and 8 items for anxiety for future use. The psychometric properties of HSCL-20 obtained through CFA are presented in the results section. The researcher suggests using an alternative measure for the screening of suicidal ideation. The studied population is highly educated and represents the higher strata of socio-economic groups in Nepali society. The researcher clearly outlined in the inclusion criteria that the respondent should be fluent in English and Nepali language. Since most of the doctors had probably studied in English language medium institutions since high school, some of the terminology translated into Nepali on the data collection tools may have been difficult for them to fully understand. Thus, the results based on the collected data and its analysis might have some bias in this regard. The study findings must be analyzed with such caveats in mind.²⁰

The study concluded that two items of anxiety, for trembling and headaches were not probably not well understood or did not consider as a component of anxiety, were omitted from the anxiety scale of HSCL-25 in Nepali. Three items of depression, concerning sex, suicidal ideation, and hopelessness; are taken out due to low factor loading. The items related to sexual desire is often considered as a taboo, and hopelessness is often misinterpreted in Nepali society. Lastly, the desire for self-harm is often not easily expressed in Nepali society. The culturally sensitive items (such as sexual desire, suicidal ideation, and hopelessness) should be reworded in future research. A separate measure for screening suicide is equally recommended. Despite the lack of primary information on the qualitative aspects of above-mentioned issues, N-HSCL-20, a two-factor model, was found to be reliable and valid to use for the screening of anxiety and depression in the Nepali population.

This research refers to the Ph.D. degree in Psychology of the first author under the supervision of the second author at the University of Nicosia, Cyprus. The authors are grateful to the support of Dr. Martin Nechtelberger, Adjunct lecturer, and Dr. Nuno Ferreira, coordinator of the Ph.D. program in Psychology at the University of Nicosia. We acknowledge the contributions of all the medical doctors who participated in and contributed to this study. The management of two institutions, Rhythm Neuro-Psychiatry Hospital and Research Centre and Transcultural Psychosocial Organisation (TPO Nepal), are credited for their institutionalized support in providing mental health and psychological support to respondents in need. Deepest gratitude goes to senior Nepali psychologist Mr. Bhava Poudyal, psychologist Mr. Sujen Man Maharjan, and Nepali language expert Hem Raj Adhikari for their contribution to the contextualization process of the tools. Thanks to two social workers, Mr. Raaz Chhetri and Ms. Sumina Machamasi, for the support in collecting paper-and-pencil based responses from the doctors.

This research contributes to the researchers and clinicians to use HSCL-20 Nepali context with validity and reliability of the tool.

None.

1. Derogatis LR, Lipman RS, Rickels K, et al. The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory. Behavioral Science. 1974;19(1):1–15.
2. Thapa SB, Hauff E. Psychological distress among displaced persons during an armed conflict in Nepal. Social Psychiatry and Psychiatric Epidemiology. 2005;40(8):672–679.
3. Baird MB, LeMaster JW, Harding A. Translation and cultural adaptation of the Hopkins Symptom Checklist–25 (HSCL–25) into Nepalese for use with Bhutanese refugees. Online Journal of Cultural Competence in Nursing and Healthcare. 2016;6(1):14–30.
4. Hateymalo accompaniment programme: ICRC's comprehensive psychosocial support programme to the families of missing persons in Nepal (2010-2016). ICRC. 2016.
5. Adhikari Y. Effects of Gorkha earthquake: corporate staff’s psychological well–being and perceived organizational support at Kathmandu valley on the aftermath of earthquake in Nepal. MOJ Public Health. 2018;2(2):75–81.
6. Skogen JC, Øverland S, Smith OR, et al. The factor structure of the Hopkins Symptoms Checklist (HSCL-25) in a student population: A cautionary tale. Scandinavian Journal of Public Health. 2017;45(4):357–365.
7. Ventevogel P, De Vries G, Scholte WF, et al. Properties of the Hopkins Symptom Checklist–25 (HSCL–25) and the Self-Reporting Questionnaire (SRQ–20) as screening instruments used in primary care in Afghanistan. Social Psychiatry and Psychiatric Epidemiology. 2007;42(4):328–335.
8. Wind TR, Knipscheer J, et al. The assessment of psychopathology among traumatized refugees: measurement invariance of the Harvard Trauma Questionnaire and the Hopkins Symptom Checklist-25 across five linguistic groups. European Journal of Psychotraumatology. 2017;8(sup2):1321357–1321357.
9. Field A. Discovering statistics using SPSS. 5th edn. Thousand Oaks, California: SAGE Publications; 2016:1–21.
10. Gerbing DW, Anderson JC. An updated paradigm for scale development incorporating unimodality and its assessment. Journal of Marketing Research. 1988;25(2):186–192.
11. Hair JF, Black B, Babin BJ. Multivariate data analysis. Saddle River, NJ: Prentice Hall. 2005.
12. Byrne BM. Structural equation modeling with AMOS, EQS, and LISREL: Comparative approaches to testing for the factorial validity of a measuring instrument. International Journal of Testing. 2001;1(1):55–86.
13. Ahmad S, Zulkurnain NNA, Khairushalimi FI. Assessing the validity and reliability of a measurement model in structural equation modeling (SEM). Journal of Advances in Mathematics and Computer Science. 2016;15(3):1–8.
14. Ockey GJ, Choi I. Structural equation modeling reporting practices for language assessment. Language Assessment Quarterly. 2015;12(3):305–319.
15. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal.1999;6(1):1–55.
16. Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research. 1981;18(1):39–50.
17. Ping RA. Is there any way to improve Average Variance Extracted (AVE) in a Latent Variable (LV) X (Revised)? 2009.
18. Awang Z. A handbook on structural equation modeling. 5th edn. Shah Alam: Universiti Teknologi MARA Publication; 2012.
19. Hair JF, Black WC, Babin BJ. Multivariate data analysis: Pearson new international edition. New York, USA: Pearson Higher Education. 2013.
20. Glaesmer H, Braehler E, Grande G, et al. The German version of the Hopkins Symptom Checklist (HSCL–25) – factorial structure, psychometric properties, and population–based norms. Comprehensive Psychiatry. 2014;55(2): 396–403.