Objective: Concept inventories are assessment tools used in pre-health disciplines, but lack in nutrition. We describe the process of design and validation for a life cycle nutrition (LCN) concept inventory and implement it to: 1) modify teaching for effective learning and 2) assess students’ knowledge.

Methods: A LCN instructor developed inventory items followed by expert validation. The inventory was administered as a pre- and post-test in the LCN course. The scores informed teaching and allowed for learning gain measurements across demographics.

Results: Validators consistently mapped LCN competencies onto inventory items (Cronbach’s alpha 0.69 to 1.0) and post-test and in-class assessments correlated (Pearson correlations >0.42; all P-values <0.005). Students displayed learning gains (average 22.1%; P<0.001) with significant differences across concentration and ethnicity.

Conclusions and implications: Nutrition programs across the United States can adapt the development process or inventory tool itself. Its implementation is especially advantageous as an assessment tool for accreditation.

Keywords: concept inventory, life cycle nutrition, learning gain, validation

LCN, lifecycle nutrition; ACEND, accreditation council for education in nutrition and dietetics

Students being trained as dietitians or public health nutritionists will work with an assortment of populations that differ by demographics such as age, gender, and socioeconomic status. As such, these students must gain a wide scope of knowledge to develop their understanding of the nutritional needs across populations and the life cycle. Accredited nutrition programs offer courses to students that encompass a variety of topics such as nutritional biochemistry, community nutrition, medical nutrition therapy, and nutrition policy, and which integrate and prepare students to deliver informed nutritional care to diverse populations.

Nutrition programs typically offer a course on life cycle nutrition (LCN) because LCN competencies are core to the curriculum (Appendix 1). This is particularly true of those programs accredited by the Accreditation Council for Education in Nutrition and Dietetics (ACEND) due to the required fulfillment of the competencies such as; “Perform the Nutrition Care Process and use standardized nutrition language for individuals, groups and populations of differing ages and health status, in a variety of settings”.¹ This study utilized a representative LCN course, within an ACEND accredited nutrition program, that was designed to help students understand the components of proper nutrition that support growth and development during critical stages of the human life cycle (i.e., from pregnancy to older adulthood). The course also integrates discussion of lifestyle factors and demographics that play a role in meeting nutritional requirements. Because LCN is part of the required curriculum specified by ACEND, it is crucial that instructors evaluate the background knowledge of students entering the course to ensure that content is taught and delivered effectively, and it is important that instructors measure student-learning gains to assess successful LCN competency gains.

A concept inventory is an educational assessment tool that was first developed and implemented in 1985 to measure 1) student placement (i.e., likelihood of student to have difficulty in the course when concept inventory is given at the start of the semester), 2) effectiveness of instruction, and 3) students’ misconceptions in introductory physics.^2,3 The usefulness of this physics tool has led instructors to adapt existing or develop new concept inventories suitable to their discipline, especially in the health and natural sciences.⁴

This project focuses on the process of design, validation, and implementation of a concept inventory for a LCN course. This paper also offers examples of ways to use the validated tool for instructional and assessment purposes, which can be applied in any discipline with a concept inventory. In particular we used it 1) for instructors to a) modify how they teach course content according to student background knowledge and b) assess the effectiveness of their teaching techniques in relation to student learning outcomes⁵ and 2) to assess students’ attainment of LCN fundamental concepts. This LCN course is open to students who are outside of the accredited nutrition program, thus consists of a diverse student body. This tool presents the opportunity to capture students’ understanding of key LCN concepts at the beginning and end of the course. LCN knowledge is critical as students develop skills as a nutritional professional. To our knowledge there is not an existing concept inventory tool that is specific to the nutrition discipline. Other nutrition programs and instructors can adapt this tool. In addition, the process of development this tool can be adapted to create concept inventories for assessment of other key nutrition concepts.

Tool development

The core instructor of the LCN course designed the concept inventory. The instructor is a registered dietitian and holds a doctorate degree in nutritional sciences. The items were written in multiple-choice format.⁶ In the first phase of development, the instructor laid out learning outcomes for each life-stage unit in the course in order to identify the key overarching course outcomes.⁶ The list of life-stage concepts covered in the course were summarized as the following: 1) pre-conception, 2) pregnancy, 3) infancy, 4) toddler/preschool, 5) pre-adolescence, 6) adolescence, 7) emerging adulthood, 8) adulthood, and 9) older adulthood. Additional key concepts embedded throughout the LCN course include life-course requirements for 10) Vitamin D, 11) iron, and 12) folate as well as 13) basic nutrition concepts (that can be applicable at every life stage). As the inventory items were developed, these concepts were mapped onto the items to ensure each key concept was represented by at least one multiple-choice question. The final tool contained 12 multiple-choice questions (Supplemental Material).

Validation by experts

Five doctoral-level nutritional sciences graduate students aided in the validation process.^2,6 They were chosen as the expert validators because they possess one or more of the following criteria: 1) completed all didactic work required of ACEND accreditation for placement in a dietetic internship, 2) are licensed registered dietitians, 3) was previously a graduate student instructor for the LCN course and/or 4) completed the LCN course. The validators were instructed to take the concept inventory as if it had been administered as an in-class exam. Following the completion of the inventory, they were asked to map each life-stage concept (listed in previous Tool Development section) onto each question (some questions may have multiple concepts). Next, the expert validators were asked to make thorough notes on questions that came across as unclear to aide in clarification and editing purposes. Lastly, they were asked to answer specific questions: 1) did any of the questions because confusion and 2) is there a major topic/concept relevant to nutrition in the life cycle that was missing?

Inter-rater reliability was calculated to measure agreement across the validators of aligning the concepts to each question (n=12 reliability measurements). Cronbach’s alpha was used as the measurement of internal consistency. The agreement was considered “acceptable” for >0.7 and “good” for >0.8.^7,8

Reliability measurement by student assessments

The concept inventory was administered as a pre-test on the first day of class, before any course content was covered and as a post-test, on the last day of class after all course material was covered. To measure the reliability of the concept inventory, specifically whether the content and difficulty were comparable to other course assessments, the average score of the post-test was measured against the first exam and the average of all three course exams using Pearson correlation. Course exams consist of three-quarter multiple-choice and one-quarter short answer. Identification of questions commonly marked incorrectly by students on the pre- and post-tests were looked at to ensure normal distribution.

Instructor use to modify teaching methods

The instructor used the results of the pre-test to identify specific concepts that students had less knowledge about upon starting the course. After identifying commonly missed concepts the instructor modified the lecture material for that concept and used small stake assessments, like quiz and case study questions, before assessing the students on the concept during an exam. On an individual basis, instructors could identify students who scored low overall and the specific concepts those students may need more help with.

Measurements of student learning

The pre-test was used as an indicator for placement of the students, or in other words, what level of LCN knowledge they have upon starting the course. The inventory was administered again at the end of the course as a post-test. A total of 47 students took both the pre- and post-tests. Students who took only one of the tests were removed from the analyses. The student learning gains were calculated as the post-test minus the pre-test. Using a paired t-test, the degree of gains across student learning was measured given the scores of the pre- and post-tests. The students’ demographic information including concentration, ethnicity, and gender were used separately in an ANOVA model with Bonferroni post hoc comparison to elucidate the differences in the scores on the pre-test, post-test and the learning gains across each of these demographics. For purposes of this study we combined student concentrations into biological sciences (Biology, Neuroscience, Kinesiology), TEM (Technology, Engineering and Mathematics), and undeclared/non-sciences (n= 26, 4, and 17, respectively).

Institutional review board

Student demographic data was obtained for each student from the University’s registrar records. The study protocol was approved by the Institutional Review Board at the University as exempt. All analyses were done in SPSS version 24 (Armonk, NY).

Results

Validation: The inter-rater reliability across the expert validators was overall “good” when aligning concepts to the multiple-choice concept inventory questions (Cronbach’s alpha for each question ranging from 0.70 to 1.0) (Table 1). The validators had constructive suggestions about the content of the questions and grammar, which were addressed before administration of the pre-test. The average score for the post-test was correlated to the average of the three course exams and the first exam (Pearson correlations=0.484 and 0.417, respectively; P=0.001 and 0.005, respectively).

Instructor use for teaching methods: Following the pre-test, the instructor for the course was able to identify concepts that students did not understand correctly. The instructor focused on the content in pre-test questions that 50% or more of the students marked incorrectly, specifically, inventory items number 1, 3, 6, 10 and 12 (Figure 1). All questions, except for number 9, displayed an improvement in the number of students correctly answering them when comparing the pre-test and post-test (Figure 1).

Figure 1 The percent of incorrect responses marked by life cycle nutrition (LCN) students on the pre- and post-test LCN concept inventory (n=47 students).

Learning gains: The average of the pre-test was 58.0%. The average of the post-test was 80.1%. Across all students this showcases significant learning gains (average gain 22.1%) (P<0.001). There were few differences after looking at the average pre-test score, average post-test score and the average gains in students by concentration, ethnicity, and gender. There were differences in the pre-test score (P=0.05) and post-test scores (P=0.04) across concentrations. Specifically, the undeclared/non-science students had an average lower score by 10.6% on both the pre- and post-tests when compared to the biological sciences major (Figure 2A). The post-test scores were different by ethnicity (P=0.04). Students of all other indicated ethnicities versus white scored lower on the post-tests by 10% or more when compared to white students (Figure 2B). Males did better on the pre-test by an average of 6.0%, but the females had larger learning gains than males (23.0% versus 15.0%, respectively) (Figure 2C).

Figure 2 Average pre- and post-test life cycle nutrition (LCN) concept inventory scores of students in a LCN course by A) Concentration, B) Ethnicity C) Gender as measured by ANOVA. Bonferroni post hoc comparisons are indicated by the connector bars if significant. *P<0.05.

This 12-item multiple-choice concept inventory tool underwent intensive validation evidencing its use as a reliable measurement for student background knowledge and learning gains of core LCN concepts. Multiple-choice is the preferred assessment format for concept inventories, because misconception items are embedded in the choices as distractor questions and it provides a reliable and consistent opportunity for scoring.⁶

The high agreement across the five validators for aligning each LCN concept to each inventory item ensured that the questions aligned with all major concepts of LCN (Table 1). The in-class student assessments were used to further validate the tool by measuring its correlation against the course exam scores. This is an indication of the reliability of the concept inventory, such as whether the content and difficulty were comparable to the major course assessments. The correlation was measured specifically against the first exam, because it is the most comparable for students’ familiarity with the instructor’s exam format.

Administering a concept inventory as a pre-test before any course content has been covered helps instructors identify the placement and acuity of their students in regard to key course concepts. Using this tool to identify concepts that students have difficulty understanding allows for the creation of a robust teacher- and student-centered approach.⁹ The instructor can translate commonly misunderstood concepts into intentional lecture or assignment components to address specific gaps as well as to ensure that students are making learning gains in all key concepts of the course.¹⁰ For example, the scores on the pre-test from this LCN concept inventory revealed that students had difficulty on a basic nutrition concept of differentiating between the functional significance of macro- and micronutrients across the life cycle. This question was mismarked by 52% of the students on the pre-test (Figure 1). Therefore, this concept was embedded throughout the lectures when discussing specific needs for nutrients in each life stage to emphasize that this is a fundamental concept when learning about or applying LCN. The concept was also given as a question on the first quiz (i.e., a low-stake assessment). On the post-test, this question was mismarked by only 12% of the students, a 40% improvement for this particular concept (Figure 1). Comparison of the frequency of mismarked questions by the students for the pre- and post-test indicate an improvement in knowledge of all LCN concepts across all students (Figure 1).

Altogether, students had a significant increase in the average scores from the pre- to the post-tests. This is a frank indication that the major LCN concepts were effectively covered throughout the duration of the course, and furthermore, that the students retained this knowledge upon completion of the course. After comparing scores of the pre- and post-tests as broken down by concentration, ethnicity and gender, there were some differences in the average scores of the pre-tests, post-tests and average learning gains. Students who were undeclared/non-science majors had significantly lower scores on their pre- and post-tests compared to the students majoring in the biological sciences (Figure 2A).

The undeclared students may be lacking pre-requisite courses, like biochemistry and physiology, which students enrolled in the accredited nutritional sciences program or biological sciences major are required to complete. Students self-identified as white had significantly higher scores on the post-test than all other ethnicities (Figure 2B). Although, males did better on the pre-test, the females had larger learning gains than males from the pre- to post-test. Because of the larger learning gain in females, they also exceeded males on the average score of the post-test (Figure 2C). This is an indication that students who score lower on the pre-test have a larger learning gap thus more room to develop LCN knowledge. The field of nutritional sciences, specifically dietetics, is dominated by white females with only 3.8% of the workforce made up of males and 18% of the dietetic workforce made up of non-white ethnicities.¹¹ The lack of diversity in STEM fields may also play into this, as biological science courses are pre-requisites for nutritional sciences programs. Continuing efforts to increase diversity across the STEM fields in undergraduate courses could aid in increased diverse student bodies in nutritional sciences courses and accredited nutritional sciences programs.

A limitation of the measurements conducted following implementation, especially the demographic measurements, is small sample size due to the nature of the course. This limitation does not take away for the development process of a reliable concept inventory or the value the LCN inventory holds for proving learning gains. Additionally, the results from the student demographics are comparable to the dietetic workforce being dominated by white females (68%) and females regardless of ethnicity (89%), and 11% males and 28% non-white ethnicities.

This project provides a direct process in developing a reliable concept inventory tool specifically for a core nutrition competency. Concept inventories are lacking in the discipline of nutrition, so this tool allows for direct adoption or modified adoption for specific content to a program’s own LCN course for its use of student assessment directly related to accreditation. The process to develop this inventory also paves the way for nutrition experts to develop other concept inventories for the remaining nutrition competencies. This work demonstrates a variety of analyses using the data from the pre- and post-tests to measure and ensure students are gaining the correct concepts in a particular course. With the advancement of course-related management system technologies, course instructors will have the ability to integrate additional variables for student learning analyses like online quizzes, discussion forums, and increased out-of-class student-teacher interaction. Making use of educational assessment tools like this LCN concept inventory will create effective learning environments and provide a means for efficient competency assessment to ensure nutrition programs across the United States are meeting accreditation standards.

We would like to acknowledge Jennifer LaBarre, Erica Cooper, Mary Connolly, and Amanda Imperiale for their help as expert validators.

The authors have no conflicts of interest and declare no competing financial interests.