Pregnancy is a moment of vulnerability when physical and emotional changes can result in symptoms of anxiety, stress, or depression. Although negative emotional states represent risks for adverse outcomes for mother and baby, instruments for their early identification and prevention are scarce. The study aims to examine the validity of the DASS-21 scale among pregnant Brazilian women and determine an optimal cut-off point for screening anxiety, depression, and stress during pregnancy. Receiver Operating Characteristic curve analysis was conducted with data collected from 378 women that answered the DASS-21, and the BDI, BAI, and LSSI as gold standard measurements. Area Under the Curve, sensitivity and specificity were analyzed for each trimester of gestation to suggest clinical cut-off points. All AUC values were higher than 0.80, and the sensitivity and specificity indexes were between 74.36 and 51.65, respectively. These findings suggest that the DASS-21 is a viable measure of screening among pregnant women.

Keywords: pregnancy, DASS21, depression, anxiety, stress

According to the World Health Organization (WHO), psychosocial circumstances such as anxiety, depression, psychosocial stressors, and stressful living circumstances represent important determinants of health.¹ Considering pregnancy as a period of vulnerability in a woman’s life,² many pregnant women can show symptoms of emotional distress due to the physical and psychological changes typical for this period.

Anxiety, depression, and stress during pregnancy have an important negative impact on maternal mental health, resulting in adverse outcomes for mothers and children. Depression during pregnancy is considered a Specifier for Depressive Disorders,³ and even though, 19.2% of women in low/middle-income countries show prenatal depression symptoms, they are still under-detected and under-treated.⁴ Furthermore, a Brazilian study found that 26.8% of pregnant women were affected by anxiety disorder.⁵ Also, stress has a high prevalence during pregnancy, affecting approximately 25% of the pregnant population and contributing to negative outcomes for fetal development.⁶

According to Schetter and Tanner,⁷ anxiety is associated with prematurity and adverse outcomes for fetal neurodevelopment, as well as depression symptoms are related to deficits in postnatal growth.⁸ Depression also has negative impacts on women’s personal adjustment, marital relationships, and mother-infant interaction.⁹ In addition, stress can be related to many predictors, like low material resources, heavy household responsibilities, and complications during pregnancy,⁷ besides anxiety symptoms during pregnancy.⁹ Despite its negative outcomes for pregnant women, studies about gestational stress are scarce.

In general, assessment scales to identify common mental disorders and psychological distress symptoms are not frequently validated for use in perinatal populations,¹⁰ especially in Brazil. According to Gelaye et al., instruments for early identification, treatment, and prevention of perinatal emotional syndromes and their negative outcomes in low/middle-income countries are required.

The Depression Anxiety Stress Scale (DASS-21) is a self-report scale that measures anxiety, depression, and stress.¹¹ It has good psychometric properties within the Brazilian population, with Cronbach's alpha of 0.92 for the depression subscale, 0.90 for the stress subscale, and 0.86 for the anxiety subscale.¹² Furthermore, the DASS-21 is considered an excellent tool for discriminating diagnoses of anxiety, depression, and stress because it is free of somatic items.¹⁰

Considering the adverse obstetric outcomes of anxiety, depression, and stress during pregnancy and their consequences for maternal and fetal health, it’s important to have screening tools to identify women in psychological suffering in order to plan preventive interventions for this population.⁴ Based on this, the objective of this study was to (1) examine the validity of the DASS-21 scale among Brazilian pregnant women, and (2) establish an optimal cut-off point for the DASS-21 scale to improve the detection of those symptomatologies in the pregnant population in Brazil.

Design and participants

This is a psychometric study based on a descriptive and exploratory design conducted with 378 pregnant women (27.2%; n=103 in the first trimester; 32.1%; n=121 in the second trimester, and 40.7%; n=154 in the third trimester). All of them attended the prenatal service of the Maternity school hospital of Universidade Federal do Rio de Janeiro (ME-UFRJ), a tertiary hospital that attends women with high-risk pregnancies. The inclusion criteria were a) confirmed pregnancy; b) older than 18 years old and c) voluntary participation in the research. Exclusion criteria included a previous history of psychiatric disorders. 

Measures

The following instruments were used: a) the DASS-21 scale; b) the Lipp's Inventory of Symptoms of Stress for Adults (LISS); and c) the Beck Scales: Beck Depression Inventory (BDI) and Beck Anxiety Inventory (BAI). Additionally, a General data protocol was used to collect socio-demographic data to analyze the sample profile.

DASS-21

The DASS-21 is a 21-item self-report scale that evaluates anxiety, depression, and stress.¹¹ Items are scored on a 4-point Likert scale from 0 (“did not apply to me at all”) to 3 (“applied to me very much or most of the time”). The three scores range from 1–7, 8–14, and 15–21 in each subscale indicating mild, moderate, and severe levels of the measured negative emotional state.⁶

LISS

The LISS is a Brazilian scale with strong psychometric properties (Cronbach’s Alpha, 0.91) that evaluate manifestations and symptoms of stress. It classifies stress into four stages (alarm, resistance, almost-exhaustion, and exhaustion) based on the identification of physical, psychological, and mixed symptoms observed in each stage of stress.¹³ Concerning its psychometric properties, LISS items have shown a strong correlation and discriminating power, where all of the items have great variance between them, contributing to the scale's Reliability.¹³

Beck scales-beck depression (BDI) and anxiety (BAI) inventories

The BDI and BAI are 21-item self-report questionnaires to measure depression and anxiety, respectively. In both, the items are scored on a 4-point Likert scale from 0 to 3. Regarding the past week, the respondent chooses his answer for each item. The scores for all 21 items generate a total score for depression (0-11=minimum; 12-19=mild; 20-35=moderate, and 36-63=severe) and anxiety (0-10=minimum; mild=11-19; moderate=20-30, and severe =31-63). A validated Brazilian version was used.¹⁴

To determine the gold standard, the scales must be correlated with the instrument to be validated. Also, they should be widely used and have a shared method of application.¹⁵ All three scales (BDI, BAI, and LISS) use the same procedure adopted to apply the DASS-21. Furthermore, they all show strong correlations with DASS-21: 0.86 for the BDI, 0.80 for the BAI, and 0.74 for the LISS.¹² Considering that they are also widely used by professionals and researchers,¹² these scales represent valid gold standards for ROC analysis.

Data were collected (April/2016 to March/2019) after patients signed the Free and Informed Consent Form approved by the Ethical Committee of ME-UFRJ. The DASS-21, LISS, BAI, and BDI questionnaires, as well as the General Data protocol, were applied while pregnant women were waiting for prenatal care.

Data analysis

Sociodemographic data were analyzed descriptively using the SPSS 20 version. The DASS-21 validity was analyzed via MedCalc statistical software using Receiver Operating Curve (ROC) analysis with LISS, BDI, and BAI adopted as gold standards and a Confidence Interval (CI) of 95%.

ROC analysis can determine the accuracy of the test by calculating the Area Under the Curve (AUC),¹⁵ which expresses the competence of the DASS-21 to truly detect those pregnant women with depression, anxiety, or stress. This would indicate how accurately DASS-21 classifies true positives and true negatives.¹⁶ The AUC value varies from 0.5 to 1, where 0.5 indicates that the predictive value is equivalent to chance and 1 indicates a perfect predictive value. Most of the best-performing assessment scales show AUC values in the 0.7–0.8 range under clinically realistic conditions and with a valid reference standard diagnoses.¹⁵

Besides the AUC value, ROC analysis also estimates sensitivity (Sens) and specificity (Spec) statistics to establish the accuracy of DASS-21. Sensitivity, otherwise known as the true positive rate, is the probability that the psychological disorder (depression, anxiety, or stress) is in fact present when detected by the DASS-21. Specificity, otherwise known as the true negative rate, is the probability that the psychological disorder is not present when undetected by the DASS-21. Furthermore, positive likelihood ratios (+LR), negative likelihood ratios (-LR), positive predictive values (PPV), and negative predictive values (NPV) were calculated for the DASS-21 for each trimester and symptomatology.

The optimal cut-off point is where the AUC has a greater proportion of patients correctly identified. Three factors must be considered when trying to determine an optimal cut-off point: (1) the intended application of the test, (2) the prevalence of the disorder in the period before the test application, and (3) the implications of correct classification and erroneous classification.¹⁵ Considering the DASS-21 as a screening test, high sensitivity is more important than specificity, since its proposal is to avoid not detecting cases that truly fit into a confirmed diagnosis of depression, anxiety, and stress. Based on this, the analysis to determine the optimal cut-off point was carried out to search for the highest sensitivity in balance with the specificity and good positive and negative predictive and likelihood ratios values.

Sample characteristics

The sample was composed of the pregnant population of Rio de Janeiro, Brazil, who attended the public prenatal service of the Brazilian Unified Health System. They were 18 to 47 years old (Mean=30.3 years), predominantly single (55%), and employed (56.6%). Sociodemographic data were summarized in Table 1.

Main analyses

The area under the curve (AUC) with standard error and confidence intervals was presented in Table 2.

The sensitivity (Sens), specificity (Spec), positive likelihood ratio (+LR), negative likelihood ratios (-LR), positive predictive values (PPV), and negative predictive values (NPV) for cut-off points for the depression, the anxiety and the stress scales of DASS-21 were presented in Tables 3 – 5, respectively.

Considering the ROC analysis as a good method to evaluate the validity of clinical measures, the AUC results confirmed that the DASS-21 can be considered a viable screening tool to identify prenatal general affective syndromes and to measure the symptomatology of depression, anxiety, and stress in the pregnant population, at any gestational trimester. Furthermore, likelihood ratios and predictive values results were considered to corroborate the diagnostic accuracy of the DASS-21 and to determine the optimal cut-off points.

Studies suggest that depression, anxiety, and stress levels change during pregnancy, according to the gestational trimester, and only one screening of psychological changes during the antenatal period is not enough.^5,17,18 According to this, the ROC analyses were performed for each gestational trimester to search for the optimal cut-off point to guarantee a sensitive detection of anxiety, depression, and stress in the pregnant population using the DASS-21.

Based on the sensitivity and specificity findings, our results suggest the necessity to adopt different, and mostly lower, DASS-21 cut-off points for the perinatal population. The psychological vulnerability during pregnancy is higher than in other moments in a woman's life and specific tests to screen for anxiety, depression, and stress among pregnant women should be considered.² Also, it highlights that different cut-off points for screening mood and anxiety disorders and psychological distress in each gestational trimester must be used to achieve reliable information. Certainly, this information can support a timely and efficient treatment based on the psychological evaluation findings throughout the nine months of pregnancy.

All the cut-off points to screen depression by DASS-21 during pregnancy were different from the general population (≥10). Considering the high sensitivity and specificity (>70) as good indexes, different cut-off points were indicated for screening for depression per gestational trimester:  ≥9 for 1^st trimester; ≥5 for the 2^nd trimester, and ≥7 for the 3^rd trimester. Differently from the first trimester, the cut-off points for the 2nd and 3rd trimesters had higher sensitivity than specificity, following Youngstrom's¹⁵ study. For this author, it is important a high sensitivity for the screening tests in order to avoid not detecting false positive cases. The cut-off point of ≥9 for the 1^st trimester showed good positive likelihood ratios (4.13) and positive predictive values (48.3), while the cut-off points for the 2^nd trimester (≥5) and the 3^rd (≥7) trimester showed satisfactory values of sensitivity (84.85 and 86.67, respectively). Moreover, the predictive values of all these cut-off points indicated that more than 48% can show depression. All those findings indicated that the DASS-21 is a valid and reliable tool for screening for depression among the pregnant population. Considering the DASS-21 as a brief scale that can be included in prenatal routine, those findings indicate that it is a valid tool for screening for depression among the pregnant population, similar to the general population.^19,20

Studies suggest that screening for depression during pregnancy is difficult because of the lack of valid scales for the perinatal population.¹⁰ Although the relation between depression and its negative impact on mothers and babies is confirmed,⁸ this mental disorder has been under-detected and pregnant women have not been attended by specialized professionals, sometimes because of the stigma of being a depressive mother and/or the limited resources for mental health services in Brazil.²¹

The cut-off point of the DASS-21 to screen for anxiety in the general population is ≥8 which is different from the cut-off points for the pregnant population: ≥7 for 1^st trimester; ≥9 for the 2^nd trimester; and ≥5 for the 3^rd trimester. Cut-off points with higher sensitivity (>74) than the specificity were found for all gestational trimesters. Furthermore, the cut-off point of ≥9 for the 2^nd trimester showed good positive (4.29) and negative (0.18) likelihood ratios and positive predictive value (47.4). Although positive likelihood ratios were not high for the 1st and 3rd trimesters (2.61 and 2.25, respectively), those cut-off points of DASS-21 can diagnose more than double those of pregnant women with anxiety symptomatologies.

Pregnant women show a higher level of anxiety in the 3^rd gestational trimester than in the 1^st and the 2^nd trimesters.^5,17 According to this, a lower cut-off point was suggested to screen for anxiety symptoms in the 3^rd trimester. Women's worries related to the delivery and concerns about their maternity⁹ are common at the end of pregnancy and can increase prenatal anxiety. Considering its negative outcomes for the mother-baby binomial, prenatal anxiety needs to be identified timely in order to offer integral prenatal assistance for anxious women. Based on this, our results indicate that the DASS-21 is a viable and reliable screening measure for detecting anxiety among pregnant women that should be used with caution, especially for positive results.

Related to the stress, none of the gestational trimesters showed cut-off points like the one suggested by DASS-21 (≥15) for the general population. The sensitives cut-off points were lower for the beginning (≥7 for 1st trimester; and ≥13 for the 2nd trimester) and higher at the end (≥17 for the 3rd trimester) of the pregnancy. In the first trimester, the physical changes are still not present, so a lower cut-off point would be enough to identify the perinatal stress. On the opposite, a higher cut-off point was suggested for the third trimester, to avoid the detection of false-positive cases due to the stress symptoms that are similar to physical symptoms common in pregnant women.

The point of ≥7 was considered an optimal cut-off point with high sensibility (100.00) to screen stress in the first trimester. Moreover, the negative likelihood (0.00) ratio was also highly satisfactory. In addition, the 2nd and 3rd trimesters showed good positive likelihood ratios (3.07 and 3.25, respectively), indicating that are three chances more to detect a positive case of stress. However, positive predictive (≥21.4) values indicate that the DASS-21 needs to be applied with caution, considering that only 21.4% of those who tested positive for stress in the first trimester actually have the diagnosis. Taking that into account, there should always be a follow-up interview with pregnant women that tested positive for stress.

Although positive predictive values are clinically relevant and helpful rates, they vary according to the prevalence of the diagnosis. So, a lower prevalence results in a lower positive predictive value, which was found in this study. Despite this, all AUC values were higher than 0.80, as well as the most of sensitivity values were higher than 0.80, supporting that the DASS-21 is efficient for screening for anxiety, depression, and stress in the pregnant population. However, the potential for measurement bias should be taken into consideration. It is important to highlight that women who score above those cut-off points should be followed up in a diagnostic interview to further assess symptoms and determine the necessity of psychological or psychiatric intervention.

Nevertheless, some limitations of the study must be considered. Firstly, participants were recruited only at the prenatal service of the Maternity School Hospital of UFRJ, which is a referral service for at-risk pregnancies in Rio de Janeiro city which is part of the Unified Health System of Brazil. Even though Rio de Janeiro is a big and metropolitan city in Brazil, our sample was partially representative of the Brazilian pregnant women population. Secondly, the lack of a pregnancy-specific stress measurement tool to use as a gold standard measurement can be considered. Physical changes related to general stress symptoms can be typical for pregnant women. So, LSSI items can represent some manifestations of typical changes from the pregnancy regardless of the presence or not of the symptomatology. For example, items like “appetite changes”, “dizziness” or "constant fatigue",¹³ are common among the pregnant population due to hormonal and physical changes during pregnancy. Summarizing, future studies with a large sample of pregnant populations from different regions of Brazil are important to support those cut-off points of DASS-21 and generalize our findings. Finally, studies focused on reliable clinical measures to discriminate physiological from psychological symptomatology of women during pregnancy are also recommended, considering that subjective particularities are usually ignored by most of the screening tests.

Despite those limitations, all findings showed that the Brazilian version of DASS-21 can be considered a reliable tool to distinguish positive from negative cases of depression, anxiety, and stress among the Brazilian pregnant population and other Portuguese-speaking countries. Considering the number of scales validated to identify stress, anxiety, and depression symptoms in pregnant women are scarce, our study contributes to future investigations focusing on the evaluation and intervention during pregnancy. Considering this moment as a period of extreme psychological vulnerability for women, it is important to highlight that the DASS-21 is a reliable assessment scale to use during pregnancy because it does not have somatic items, such as sleep disturbance or lack of energy and/or concentration, which are not valid anxiety, depression, and stress markers for pregnant or postnatal women.¹⁰ Furthermore, the application of the DASS-21 scale is simple and quick, allowing its use in perinatal care services for early detection of maternal changes and stress responses and to prevent their negative outcomes for mother-child binomial. Different screenings conducted in different trimesters of the same pregnant woman can enhance the effectiveness of the detection and treatment of depression, anxiety, and stress symptoms. Finally, the cut-off points suggested for different gestational trimesters can improve the development of efficient interventions for pregnant populations at risk for mental disorders and acute stress responses.

We are grateful for the Laboratório de Estudos, Pesquisa e Intervenção em Desenvolvimento e Saúde (Laboratory of Study, Research and Intervention in Development and Health), Maternity school hospital of Universidade Federal do Rio de Janeiro. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors’ contributions

Conceptualization: Cassia Patricia Barroso Perry and Ana Cristina Barros da Cunha; Methodology: Cassia Patricia Barroso Perry and Ana Cristina Barros da Cunha; Formal analysis and investigation: Cassia Patricia Barroso Perry; Writing – original draft and preparation: Cassia Patricia Barroso Perry; Writing – review and editing: Ana Cristina Barros da Cunha. All authors contributed to and have approved the final manuscript.

Availability of data and material

The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

The authors declare that they have no conflict of interest.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

1. WHO. World Health Organization. A conceptual framework for action on the social determinants of health. Geneva. Paper 2 (Policy and Practice). 2010.
2. Dubber S, Reck C, Müller M, et al.  Postpartum bonding: the role of perinatal depression, anxiety and maternal–fetal bonding during pregnancy. Arch Women Ment Health. 2015;18(2):187–195.
3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA; 2013.
4. Woody CA, Ferrari AJ, Siskind DJ, et al. A systematic review and meta–regression of the prevalence and incidence of perinatal depression. Journal of Affective Disorders. 2017;219:86–92.
5. Silva MMJ, Nogueira DA, Clapis MJ, et al. Anxiety in pregnancy: prevalence and associated factors. Revista da Escola de Enfermagem da USP. 2017;51:1–8.
6. Dolatian M, Mahmoodi Z, Alavi‐Majd H, et al. Psychosocial factors in pregnancy and birthweight: Path analysis. J Obstet Gynaecol Res. 2016;42(7):822–830.
7. Schetter CD, Tanner L. Anxiety, depression and stress in pregnancy: implications for mothers, children, research, and practice. Curr Opin Psychiatry. 2012;25(2):141–148.
8. Gelaye B, Rondon MB, Araya R, et al. Epidemiology of maternal depression, risk factors, and child outcomes in low–income and middle–income countries. The lancet Psychiatry. 2016;3(10):973–982.
9. Rubertsson C, Hellström J, Cross M, et al. Anxiety in early pregnancy: prevalence and contributing factors. Arch Womens Ment Health. 2014;17(3):221–228.
10. Meades R, Ayers S. Anxiety measures validated in perinatal populations: a systematic review. Journal of Affective Disorders. 2011;133(1–2):1–15.
11. Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. 2nd ed. Psychology Foundation of Australia, Sydney; 1995.
12. Vignola RCB, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. Journal of Affective Disorders. 2013;155:104–109.
13. Lipp MEN. Lipp's inventory of symptoms os stress for adults (ISSL) in federal civil servants of São Paulo: Casa do Psicólogo. 2000.
14. Cunha JA. Manual for the Portuguese version of the Beck Scales. São Paulo: Casa do Psicólogo; 2001.
15. Youngstrom EA.  A Primer on Receiver Operating Characteristics Analysis and Diagnostic Efficiency Statistics for Pediatric Psychology: We Are Ready to ROC. Journal of Pediatric Psychology. 2014;39(2):204–221.
16. Henry SK, Grant MM, Cropsey KL. Determining the optimal clinical cutoff on the CES–D for depression in a community corrections sample. Journal of Affective Disorders. 2018;234:270–275.
17. Priyambada K, Pattojoshi A, Bakhla AK. A study of antenatal anxiety: comparison across trimesters. Int J Reprod Contracept ObstetGynecol. 2017;6:1810–1013.
18. Lee MA, Lam KS, Sze ML, et al. Antenatal anxiety in the first trimester: Risk factors and effects on anxiety and depression in the third trimester and 6–week postpartum. Obstetrics & Gynecology. 2007;110(5):1102–1112.
19. Oei TP, Sawang S, Goh YW, et al.  Using the Depression Anxiety Stress Scale 21 (DASS‐21) across cultures. International Journal of Psychology. 2013;48:1018–1029.
20. Andrew TG, Howard MR, Diane N, Jens K, et al. Psychometric properties of the Depression Anxiety and Stress Scale–21 in older primary care patients. Journal of Affective Disorders. 2008;110(3):248–259.
21. Pereira AT, Soares MJ, Bos S, et al. Why should we screen for perinatal depression? Ten reasons to do it. International Journal of Clinical Neurosciences and Mental Health. 2014;1:1–7.