Background: The aim for this study was to assess the construct validity and reliability of the Health Dialogue Questionnaire (HDQ), 7th grade in compulsory school through comparison of the HDQ, with Paediatric Quality Of Life Inventory (PedsQLTM), Local monitoring of youth policy questionnaire (LUPP®) and Health behaviour in Swedish school-aged children (HBSC).

Design and methods: A sample was created from HDQ (n= 2008), PedsQLTM (n=477), LUPP (n=2648), HBSC, (n=1500) andan exploratory factor analysis was performed in order to evaluate the construct validity of HDQ of the school children´s health in school settings.

Results: The results supported the HDQ as a valid 24 items factorial model, for girls a five factorial model and for boys a four factorial model for school children aged 13years old (grade 7). The girls’ model explained 63 % of the variance, while the boys’ model explained 58 % of the variance. A majority of the questions showed an agreeable concurrent and discriminant validity.

Conclusions: The HDQ questionnaire is a valid instrument for measuring 13-year-old school children’s self-reported-health.

Keywords: adolescents, construct validity, factor analysis, health dialogue questionnaire, health promotion

HDQ, health dialogue questionnaire; PedsQLTM, paediatric quality of life inventory; LUPP®, local monitoring of youth policy questionnaire; HBSC, health behaviour in Swedish school-aged children; RMSEA, root mean square error of approximation; TLI, tucker-lewis index; CFI, comparative-fit-index; GFI, goodness-of-fit index; ITC, item-total-correlation; M, mean value; SD, standard deviation; SMC, squared multiple correlation

In a WHO study including 40 countries, Sweden was identified as one of the countries where the majority of children live healthy and safe lives, eat healthy foods regularly, are physically active and play and go to school in a safe environment.¹ Schoolchildren’s Health in Sweden has improved for many years, with increasing life expectancy and decreasing health problems. This increased health can be linked to increased financial resources, enhanced level of education and better health care and knowledge of how to promote health and prevent illness.² Children’s health is a broad concept, incorporating much more than a simplistic freedom from medical illness. In this study the World Health Organizations³ definition is used. Time trends from 1985/86 to 2005/06 among Swedish boys and girls; show that the risk of mental health problems such as nervousness, feeling low and sleeping difficulties has increased steadily.^1,4,5 Childhood health and development is affected by the environment and the child’s genetic heritage, but first and foremost by the family, living conditions, lifestyles, school and friends.⁶

Thirteen year olds

School-children in 7th grade are in the middle of the greatest development process in their life; puberty. There is a major puberty gender difference, in that girls are approximately two years ahead of boys. Both genders develop their sexuality and they go through a great cognitive ability development. They are striving to answer important questions: ‘Who am I?’ and ‘What will become of me?’. During this period of time there is also an increasing workload with homework and higher academic demands from school. The social activities often include sports clubs etc. and a closer friendship within and across gender often develop. Parallel to this the school environment changes in different ways, for instance, with larger school units. All together this period of time is very demanding period for most teens.^7–9

The health dialogue questionnaire(HDQ)

The HDQ has been tested for 10-y-olds in grade 4and proved to be reasonably valid.¹⁰The HDQ specific for the 13-y olds in 7th grade is not yet tested for validity and reliability.

Aim

The aim was to assess the construct validity and reliability of the HDQ 7th grade in compulsory school through comparison of the HDQ with Paediatric Quality Of Life Inventory (PedsQLTM), Local monitoring of youth policy questionnaire (LUPP®) and Health behaviour in Swedish school-aged children (HBSC).

Research context

The data originate from the school nurses HDQ in 7th in compulsory school in the county of Vasternorrland. Each year registers about 8,000health consultations in the county. The county contains of seven municipals, and the region is characterized by large rural areas and few cities approximately 250,000 inhabitants located in the middle/north of Sweden. Furthermore the county has lower middle income, education level and proportion of residents with foreign background compared with the national average. The county’s school students have slightly lower average level of grades compared with the national average.¹¹

Sample

The validity and reliability calculation were investigated based on the data from the HDQ in 7th grade in compulsory school (n= 2008), the HBSC questionnaires (n=1500), the “LUPP®”questionnaires (n=2648) and the PedsQLTM (n=477), in the academic year 2009/2010. The sample size for HDQ corresponded to 77 % girls and 79 % boys of all 7thgrades school girls and boys in the county.¹¹

Data collected

The text regarding the used instruments was presented as follows:

1. Name
2. History, Background and Responsibility for the instrument
3. Which dimensions and numbers of dimensions included
4. Numbers of items and how these are sorted in to dimensions
5. How the questionnaire was distributed and handled

The health dialogue questionnaire, the HDQ: The County Council and the seven municipalities reached consensus on a systematically structured approach to the collection of school children’s health data. The HD represents a cross-sectional image of the child’s SRH regarding physical, mental and social health. The HDQ was offered to all school children at four occasions: in preschool, in fourth grade, in seventh grade, the first year of high school, and follows the child’s development and growth from 6 to 16years old and therefore repeat the content with age appropriate questions.^12–14 The HDQ consisted of a structured questionnaire covering three dimensions of health physical, mental and social. The physical dimension included six questions/items. The mental dimension included five questions/items, while, the third dimension was social and included four questions/items. The school nurses who conduct the HDQ was situated at the school and shared the school environment with the school children at a daily base and were responsible for the data collection at each separate school i.e. distribution of written as well as oral information to all children and parents regarding voluntary participation and the registration of each HDQ. The school nurses were also responsible for the individual feedback to the students.

Health behavior in Swedish school-aged children (HBSC): The HBSC was a part of the WHO global survey and since 1985/1986 this study has taken place around the world every four years, most recently in over 40 different countries. The international standard questionnaire enables the collection of common data across participating countries and thus enabled the quantification of patterns of key health behaviours, health indicators and contextual variables. School children aged 11, 13 and 15 were asked to answer different health questions during their ordinary school classes and give a cross-sectional image of school children’s health.¹⁵ The HBSC consisted of a structured questionnaire with 82 questions covering four dimensions of health behaviour among school aged children. The dimensions were ‘Self-rated health and general well-being’, ‘Lifestyle’, ‘Social relationships’ and ‘School’.

The sampling method for participating school children was carried out in a two-step cluster design. First, a national representative cluster of schools was randomly selected. Second, a selection of schools or classes in each grade was included in the study, with at least 1500 school children in each grade; data consisted of school children 13years old from the academic year 2009/2010. The cluster of schools differs for each occasion the HBSC is conducted. Furthermore the HBSC was a school-based survey with data collected through self-completion questionnaires administered in the classroom by teachers. The HBSC did not include individual contact, dialogue or feedback concerning the questionnaires with the school children. The teacher is responsible for distribution of information concerning the study, such as parental agreement and school children´s voluntary participation.

Pediatric quality of life inventory (PedsQLTM): The PedsQLTM was a modular approach to measuring health-related quality of life in children and adolescents. The PedsQLTM consisted of brief, practical, generic core scales suitable for use with healthy school and community populations. Accordingly, the PedsQLTM consisted of developmentally appropriate forms for children ages 2-4, 5-7, 8-12 and 13-18years. Paediatric self-report was measured in children and adolescents ages 5-18years. The PedsQLTM was not used on a regular base in school environment, but mostly used in the research context.¹⁶ The PedsQLTM encompassed the essential core domains for paediatric health related quality of life measurement:

1. ‘Physical Functioning’ (8 items)
2. ‘Emotional Functioning’ (5 items)
3. ‘Social Functioning’ (5 items)
4. ‘School Functioning’ (5 items)¹⁷

The questionnaires were distributed by the school nurses in one of the municipalities in the county, in addition to the HDQ to the same individuals.

Knowledge about young people-LUPP®: The LUPP® (a local follow-up of youth policy) was a cross-sectional survey that enabled municipalities to gather knowledge on the living situation of young people, as well as information on their experiences and opinions, since 2001. The Swedish Agency for Youth and Civil Society has developed the LUPP® survey in consultation with municipal representatives and researchers. The survey was adapted for different age groups: young people at compulsory school aged 13-15; young people at upper secondary school aged 16-18; and young adults 19-25years old. The LUPP® survey contains approximately 80 questions was divided into different modules with questions concerning the perception of influence and democracy, wellbeing school, leisure activities, work, health and future plans. The survey was mainly intended to be conducted electronically in school. In order to reach young people outside school it could be supplemented with a mail survey, in either electronic or paper form.

The LUPP® survey could be carried out electronically or on paper, by the latter age group, a web-based survey, while the younger age group typically use questionnaires implemented during a lesson hours in school by teachers. Distribution was like the HBSC. LUPP® survey was analysed and reported at a municipal and national level and used in the research context.

Statistical analyses

Assessing the usefulness of school health records demands a structured approach to validity. A schematic picture of the construct validity1 model was used as the analysing structure could be seen in Figure 1. A variety of ways exists to evaluate the construct validity and in general the order of analysis follows chronologically 1 to 3 (Figure 1). Construct validity consisted of Translational validity4, though in the model not analysed and henceforward not mentioned, while Criterion validity2 consisted of concurrent3 (Table 1), predictive3, convergent3 and/or discriminant3 (Table 2) validity.

Figure 1 Schematic overview of the construct validity model used when working with the HDQ [20]. (The numbers 1, 2, 3 and 4 in the model refers to the ‘Statistical analysis’ section in the text).

Related items defined a part of a construct and were grouped and modelled together as a factor/s or a latent construct/s. A structural equation model approach was used to develop and test the factor models. In the first structure analyses models, all the items were modelled as indicators of the intended underlying latent constructs. A two-step structural equation model procedure was applied. In the first step, an exploratory factor analysis was conducted to determine a model where the constructs measured the underlying latent variables. The factor analysis assumed each manifest variable/item to be a distinct indicator of an underlying latent construct, whereby different constructs were permitted to be inter correlated. In the second step a Mann Whitney U test was carried out to determine if the latent variables were acceptable as independent constructs that could be sufficiently distinguished from each other. Significant Mann-Whitney test values indicate discriminate validity. The appropriateness of a specific factor analysis model was assessed by measures of global and local fit. Global fit indicated whether the empirical associations among the manifest variables were appropriately reproduced by the model. For a variety of these global fit measures, certain criteria have to be met in order to accept the model under study as plausible and parsimonious. For the root mean square error of approximation (RMSEA), the model can be classified as good if the RMSEA is <= 0.05, and is acceptable if it is <=0.08. Furthermore, measures of incremental fit were used namely the comparative-fit-index (CFI) and the Tucker-Lewis index (TLI). A rule of thumb for incremental fit measures is that values >=0.97 are indicative of good fit, while values >=0.95 may be interpreted as acceptable. The goodness-of-fit index (GFI) is the measure of the relative amount of variance in the sample that was explained by the model. The index range from zero to one, and values close to one represented good fit. Measures of local fit evaluated whether each construct can be reliably estimated from its indicators and whether the constructs within the model are sufficiently distinguishable. The item-total-correlation (ITC) was a correlation between the question and the overall assessment, and in this cases the manifest items and their latent variables. Values for an item-total correlation between 0 and 0.19 may indicate that the question was not discriminating well, values between 0.2 and 0.39 indicate good discrimination and values 0.4 and above indicate very good discrimination.^18–20

Ethics

This study followed the ethical principles recommended by the Research Council and was approved by the Ethics Committee at the Medical Faculty, Umea University, to be evaluated the (Dnr 2008-122M).

The HDQ was salutogenic and the percentages of positive answers are presented in Table 1. Measuring the concurrent validity are done by comparing the percentages emanating from the HDQ with similar questions from different but resembling questionnaires HBSC, PedsQLTM and LUPP®. Describing and analysing the percentages in Table 1, the percentages were averagely underestimated 9 percentages (Table 1); the items “daily physical activity”, “eating breakfast every school day”, and “sleep” differ mostly between HDQ and the population proxies.

Items in HDQ exploratory factor analysis models with insufficient model compatibility were sequentially eliminated from the model until criteria for good model fit were reached. The resulting models each comprised of 24 items and exhibited an all in all acceptable fits (Tables 3 & 4), acceptable fit threshold (actual model fit index value) χ2/df <3 (2.86); RMSEA<=0.08 (0.031); CFI>=0.95 (0.926); TLI>=0.95 (0.914); GFI >=0.90 (0.94). RMSEA even reached the good fit threshold (<=0.05) while the CFI and TLI levels just reached suggested less restrictive fit thresholds (>=0.90).^18,19

In the girl’s model (Figure 2), items 1 (psych 1) and item 2 (psych 2) showed content overlap. Item 4 (psych 4) and item 6 (psych 6) also appeared to overlap as well as item 1 (psych 1) and item 5 (psych 5). Each manifest item was mutually correlated >0.50. Error covariance parameters were included in the model to manage the item content overlaps in the girl model. In the boy’s model (Figure 3), items 1 (psych1) and item 2 (psych 2) also showed content overlap. Item 16 (phys 7), item 19 (phys 9) and item 13 (phys 1) and item 15 (phys 4) correlated >0.40. Alas to modify the model, error covariance parameters were added in the boy model. Turning to the unstandardized model estimates, all were significant in both models as could be seen in Table 3 & 4. In the girls model the standardized parameter estimates were generally high (lowest psych 4-0.31 vs. highest behave 1 0.90) as well as in the boy model (lowest psych 3 0.18 vs. highest behav 2 0.84) (Table 3 & 4).

Figure 2 Factor model girls.

Figure 3 Factor model boys.

The item total correlations between the manifest item and their latent variable (factor) were generally high for both girls and boys. One item (psych 3) though, in the boy model only managed to reach 0.14 (Table 5). All the other values, at least indicated good discrimination (0.2>=ITC<=0.39), with a majority of the values (90%) 0.4 and above indicating a very good discrimination (Tables 5 & 6).^18,19 Furthermore, indices of local-fit proved that each latent construct was agreeably reliably measured by its indicators/items 0.42 % of the manifest items in the girl model and 17% in the boy model was measured acceptably by its underlying construct (indicator reliability, >=0.40) (SMC R2in Tables 5 & 6).^18,19

A confirmatory factor analysis confirmed a five factor model for the girls and a four factor model for the boys in school (7th grade) using the items from HDQ (construct validity). The analyses resulted in two finals Table structures (girls and boys) that were tested and confirmed on another HDQ seven graders sample (academic year 2010/2011, n=1979). The five factor girl’s model resulted in a 63% cumulative variance explained and the four factor boy’s model in 58% cumulative variance explained (principal axis factoring with varimax rotation). In Table 2, the bivariate correlations of all scales from the exploratory factor analyses models (Figures 2 & 3) and the results of the convergent and discriminant validity test is shown. The tests were done on a new sample of seven graders from another year (academic year 2010/2011, n=1979) to test stability of the scales. The internal consistency measured by Cronbach’s alpha, were generally high (behaviour 0.47 girls and 0.75 boys; environment 0.76 girls and 0.79 boys; psychology 0.78 girls and 0.67 boys; work 0.82 girls; physical 0.74 girls and 0.73 boys).

The aim was to psychometrically test HDQ for measuring the self-reported health in 7thgraders in compulsory school. The major finding was that it was possible to adequately model five (girls) respectively four (boys) hypothetical dimensions of the health In girls health these were psychological, environmental, physical, behavioral, and working environment (work) and the four hypothetical dimensions in boys health were psychological, environmental, physical and behavioral (Figures 2 & 3). A confirmatory factor analysis confirmed that it was possible to adequately model girls and boys health (construct validity) in 7th grade using the items from HDQ. The analyses resulted in two finals Table structures (girls and boys) that were tested and confirmed on another HDQ seven graders sample (academic year 2010/2011, n=1979). The five factor girl’s model resulted in 63% cumulative variance explained and the four factor boy’s model in 58% cumulative variance explained (principal axis factoring with varimax rotation). The concurrent criterion related validity was analysed when the HDQ was compared with answers from three other questionnaires similar questions and similar sample (for HDQ and PedsQLTM actually identical). Ambiguous questions showed less criterion related validity, but most of the questions in the HDQ related highly with their proxy counterparts.

Convergent and discriminant validity was examined using the newly produced scales from the structural equation modelling factor analysis and correlating the scales resulted in understandable and expected associations. To further strengthen the validity of the HDQ would be to test the questionnaire in a highly different sample. The fact that we analyzed a regional cohort composed of mostly white children, living predominantly rural, with low or middle class parents mean that the population was well-defined. The questionnaire results might not be generalizable, for instance, to a more heterogenic urban area.

Methodological consideration

Since all results were assessed using self-reports, results may have been biased by common method variance. The common method variance may state a real bias in survey-type of studies and enhance the observed correlation between variables.²¹ Despite the above mentioned, the overall findings suggests that structured validation studies helps developing sound and usable questionnaires.

Children “learn health” through role models (mainly parents but also other adults close to the children).²² Therefore, the school is an important arena for health promotion, as all children attend school and all adults within a school have an opportunity to serve as role models.^23,24 A health promoting school strengthens children’s self-esteem and school performance²⁵ and learning and health are known to be generated by the same factors.^26,27 The questionnaire may be used in health promoting practices in school, serving as a feedback instrument for assessing the health of school children in 7th grade.

The authors would like to acknowledge all the school children and school nurses.

The author declares no conflict of interest.

1. Hagquist C. Discrepant trends in mental health complaints among younger and older adolescents in Sweden: an analysis of WHO data 1985–2005. J Adolesc Health. 2010;46(3):258–264.
2. Kohler L. A Child Health Index reveals poorer health in Gothenburg’s northern districts. Gothenburg: NHV; 2010.
3. World Health Organization (WHO). WHO definition of Health. Geneva: World Health Organization; 1948.
4. Kolip P, Schmidt B. Gender and Health in Adolescence. Europe: WHO Regional Office for Europe; 1999.
5. Currie C, Zanotti C, Morgan A, et al. Social determinants of health and well–being among young people. Health Behaviour in School–aged Children (HBSC) study: international report from the 2009/2010 survey. Copenhagen: WHO Regional Office for Europe; 2012.
6. Eriksson U. Children’s self–reported health– the importance of social capital in the school and in the community. Sweden: Mid–Sweden University; 2012.
7. Holmstrom MR, Olofsson N, Asplund K, et al. Transitions in the Swedish school system and the impact on student's positive self–reported–health. BMC Public Health. 2014;14:1045.
8. Kelly KB. Transition: bridge over troubled water? Acta Paediatr. 2010;99(12):1782–1784.
9. Kelly KB. Adolescent health, school health activities, community contexts, and health surveys in Sweden. J Adolesc Health. 2003;33(4):226–230.
10. Holmstrom MR, Olofsson N, Kristiansen L. Assessing the validity and reliability of the 'Health Dialogue' in 10–year–olds. British Journal of School Nursing. 2013;8(8):9.
11. Statistics Sweden (SCB). Statistical Database. Sweden: Statistics Sweden; 2012.
12. Holmstrom MR, Olofsson N, Kristiansen L, et al. Health among 6–year–old children in a Swedish county: based on the Health dialogue. Vulnerable Groups & Inclusion. 2012;3: 8416.
13. Holmstrom MR, Olofsson N, Asplund K, et al. Exploring the development of school children's health. British Journal of School Nursing. 2013;7(4):183–190.
14. Jonsson M, Wickberg C, Bjor O. The Health Dialogue–an opportunity to gain knowledge of children and young peoples' health. Harnosand: A pilot project in Harnosand; 2001.
15. King A, Wold B, Smith CT, et al. The Health of Youth: A cross–national survey. WHO Reg Publ Eur Ser. 1996;69:1–222.
16. Varni JW, Seid M, Rode CA. The PedsQL (TM): measurement model for the Pediatric Quality of Life Inventory. Med Care. 1999;37(2):126–139.
17. Varni JW, Seid M, Kurtin PS. PedsQL (TM) 4.0: Reliability and validity of the Pediatric Quality of life Inventory Veriosn 4.0 Generic Core Scales in healthy and patient populations. Med Care. 2001;39(8):800–812.
18. Ommen O, Wirtz M, Janssen C, et al. Psychometric evaluation of an instrument to assess patient–reported 'psychosocial care by physicians': a structural equation modeling approach. Int J Qual Health Care. 2009;21(3):190–197.
19. Byrne BM. Structural Equation Modeling with AMOS Basic concepts, Applications, and programming. USA: Routledge Taylor & Francis group; 2012.
20. DeVon HA, Block ME, Wright PM, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155–164.
21. Podsakoff PM, MacKenzie SB, Lee JY, et al. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. 2003;88(5):879–903.
22. Pugliese J, Tinsley B. Parental socialization of child and adolescent physical activity: a meta–analysis. J Fam Psychol. 2007;21(3):331–343.
23. Gadin KG, Weiner G, Ahlgren C. Young students as participants in school health promotion: an intervention study in a Swedish elementary school. Int J Circumpolar Health. 2009;68(5):498–507.
24. Gadin KG, Weiner G, Ahlgren C. School Health Promotion to Increase Empowerment, gender Equality and Pupil Participation: A Focus group Study of a Swedish elementary School Initiative. Scandinavian Journal of Educational Research. 2013;57(1):54–70.
25. Gustafsson LH. The health of student's begins in the classroom. Lund: Student litteratur; 2009.
26. Bremberg S, Wennerholm JP. When children and adolescents get to decide it affects the health: A systematic review. Sweden: Swedish national Institute for Public Health; 2004.
27. Wennerholm JP, Danielsson M. Schoolchildren's health and welfare 1985–2001: A reliability and validity report from schoolchildren health habits. Sweden: Swedish national institute for Public Health; 2005.