Submit manuscript...
Journal of
eISSN: 2373-4345

Dental Health, Oral Disorders & Therapy

Research Article Volume 10 Issue 3

Team-based learning student assessment instrument in Brazilian dental education: a validation study

Fábio Barbosa De Souza,1 Elaine Judite De Amorim Carvalho,2 Bruna Leal De Castro,3 Emmanoel Matheus De Oliveira Matos,4 Ingrid Melo Schüler Arreguy,3 Altamir Oliveira De Figueiredo Filho,4 Jessica Silva Peixoto Bem4

1Department of Prosthodontics and Oral Facial Surgery, Federal University of Pernambuco, Brazil
2Department of Clinics and Preventive Dentistry, Federal University of Pernambuco, Brazil
3Undergraduate Student, Dental School, Federal University of Pernambuco, Brazil
4Dental School, Federal University of Pernambuco, Brazil

Correspondence: Fábio Barbosa de Souza, Department of Prosthodontics and Oral Facial Surgery, Federal University of Pernambuco, Av. Prof. Moraes Rego s/n, Cidade Universitária, Recife-PE, CEP: 50670-901, Brazil, Tel +55-81-2126-8830, +55-81-9974-8237, Fax +55-81-2126-8344

Received: April 18, 2019 | Published: May 1, 2019

Citation: Souza FBD, Carvalho EJDA, Castro BLD, et al. Team-based learning student assessment instrument in Brazilian dental education: a validation study. J Dent Health Oral Disord Ther. 2019;10(3):182-186. DOI: 10.15406/jdhodt.2019.10.00481

Download PDF

Abstract

The aim of this study was to validate the Team-Based Learning Student Assessment Instrument in students of a Dental School of a Brazilian University. The validation process was performed progressively in the steps: translation and back-translation; consensus version and cross-cultural adaptation; face validation; teste version; test-retest. The reliability assessment was performed by calculating the internal consistency by obtaining the Cronbach Alpha coefficient. The statistical analysis of the test-retest was the calculation of the intraclass correlation coefficient. The margin of error used in the tests was 5.0%. After application of TBL-SAI to a population of 80 students, the Cronbach's Alpha coefficient was 0.670. After removing questions 12 and 19, the result would change to 0.702 and 0.701, respectively. The validation process was carried out in a satisfactory way. It was verified the need to reduce the quantity of items, in order to make the evaluation of perception in relation to the method of teaching more reliable.

Keywords: team-based learning, active methodologies, dentistry, validation

Introduction

Education of the current generations (millenials) poses a challenge for teaching. These students’ value communication and teamwork.1 Thus, traditional education, in which the teacher makes the deposit of contents, while the student must memorize them2 does not fit the parameters necessary for an effective contemporary teaching. In this sense, active learning methodologies appear as a strong allied to meaningful teaching. They are educational methods capable of engaging students in the learning process, putting them in control of their own path of knowledge development.3 Team-based learning (TBL) represents one of these.4 This instructional strategy has been shown to be viable and effective in the professions of health and medical education.5−8 However, these investigations were applied in North American and Asian students, and the application of the perception assessment method in Brazilian students is non-existent. One of the instruments capable of analyzing students' perceptions about TBL is the Team-Based Learning Student Assessment Instrument (TBL-SAI), developed by Mennenga, in 2012.8 However, to apply it in a Brazilian population, whose primary language is Portuguese, a validation study is necessary. Validating an instrument means obtaining a new version similar to the original, making cultural adaptations, as well as confirming properties that show the ability to measure it. This study had the objective of validating the TBL-SAI in students of a Dental School of a Brazilian University.

Materials and methods

Study design and population

This was a validation study of the TBL-SAI questionnaire,9 originally written in the English language. The population was composed of students over 18 years of age, of both sexes, who attended the Dental School of Federal University of Pernambuco (Brazil) in 2018, which used TBL as teaching methodology. This research was approved by the Institutional Research Ethics Committee, under the number of process: 81177017.5.0000.5208. Subjects who agreed to participate were included, by reading and signing the Free and Informed Consent Form.

TBL-SAI questionnaire

The questionnaire presents 33 short and direct multiple-choice questions whose answers are on a Likert scale, in which all questions should be answered based on agreement with the sentences that are stated in each question. For each sentence (question) a number is assigned according to the degree of agreement:

  1. Strongly Disagree.
  2. Disagree.
  3. Neither disagrees nor agrees.
  4. I agree.
  5. Strongly Agree.

The TBL-SAI consists of three subscales:

  1. Responsibility, consisting of eight items.
  2. Preference for expositive/traditional class or ABE, containing 16 items.
  3. Student satisfaction, comprising nine items. In this study, a printed version was used to validate the questionnaire.

Validation

Translation and back-translation

The TBL-SAI was translated into Portuguese by a bilingual professional native to the Portuguese language fluent in English, who knew the subject and questionnaire. The back-translation, whose purpose is to check whether the translated version reflects the original, was performed by another bilingual native English-speaking professional with no knowledge of the original instrument. It aids in the identification of possible inconsistencies.

Consensus version and cross-cultural adaptation

A multidisciplinary team consisting of two professionals, one graduate in dentistry and the other a graduate in business administration compared the two versions (translated and back-translated) with the original to obtain the version used in face validation. In this process, there was a need to adapt the term "lecture" whose consensus resulted in the translation: "traditional class" or expositive lecture”. It is common to change, create or delete terms considered inappropriate or ambiguous, as long as the concept is similar to the original.

Face validation

The face validation of TBL-SAI was performed with a group of 9 individuals. At this stage, the students answered the questionnaire individually and commented on the issues of the applied version, pointing out difficulties and suggesting more understandable terms. Respondents were asked to verbally paraphrase each item and the evaluators wrote down, to verify whether or not the item was understood. Subsequently, modifications were made necessary for a better understanding and adaptation of the questionnaire to the target audience. Finally, it was also recorded the average time each participant took to answer it in full, besides registering the acceptability of the TBL-SAI format.

Test version

Based on the performance of the questionnaire in face validation, in relation to the understanding of the questions and also in the suggestions of the respondents, the examiners met to make the necessary changes, which led to the writing of the test version of the instrument (Table 1). The elaborated version was later tested for its psychometric properties and forwarded to the authors of the original questionnaire for the conference.

1. I spend my time studying before class in order to be more prepared

2. I feel that I need to prepare for class to get good income

3. I contribute to the learning of my team members

4. My contribution to the team is not important

5. My team members expect me to help them in their learning

6. I am responsible for learning from my group

7. I am proud of my ability to help my group in their learning

8. I have a need to contribute to the learning of my team

9. During a traditional lesson, I almost always find myself thinking about things not related to class

10. I am easily distracted during a traditional class

11. And it distracts me easily during team-based learning activities

12. It is easier for me to fall asleep during an expository class than during classes using team-based learning activities

13. I get bored during team-based learning activities

14. I talk about things not related to class during team-based learning activities

15. I easily remember what I learn when I work in a team

16. I remember the subject better when the teacher gives an expositive lecture about it

17. Team-based learning activities help me remember past information

18. It is easier to study for the tests when the teacher has already lectured on the subject

19. I remember the information for a longer time when I participate in the activities

20. I best remember the subject after classroom practice in team-based learning

21. I can easily remember the subjects taught in the expositive lectures

22. After discussing with the members of my team, I find it difficult to remember what we discussed during class

23. I do better in exams when we use team-based learning of the subjects to be studied

24. After attending the lecture, I have difficulty remembering what the teacher said during class

25. I like the activities of team-based learning

26. I learn best when I'm in a group

27. I think team-based learning represents an effective approach to learning

28. I do not like working in teams

29. The activities of a team-based class are fun

30. The activities of a team-based class are a waste of time

31. I think team-based learning has helped me improve my school performance

32. I have a positive attitude towards team-based learning activities

33. I had good experience with team-based learning

Table 1 Brazilian version of TBL-SAI questions

Test-retest

At this stage the questionnaire was applied to a sample composed of 30 individuals (test) at the dental school of UFPE. After 7 days, the questionnaire was re-applied to the same individuals to verify agreement, aiming to verify the reliability of the questionnaire and its consistency.

Statistical planning

The data were compiled in Excel and the statistical analysis was performed using MEDCALC software in version 14.8.1. The margin of error used in the tests was 5.0%. The reliability assessment was performed by calculating the internal consistency by obtaining the Cronbach Alpha coefficient, which is one of the most used psychometric indicators to verify reliability or internal consistency of an instrument. The statistical analysis of the test-retest was the calculation of the intra class correlation coefficient (ICC), adequate parameter to define the stability of the instrument in question. The ICC can range from 0 to 1 and is considered a reflection of the agreement or correlation between the scores obtained in the test and the retest by a group of evaluated individuals. The reference values are: ICC less than or equal to 0.40=weak correlation; ICC between 0.41 and 0.60=moderate correlation; ICC between 0.61 and 0.80=good correlation; KIC greater than or equal to 0.8=excellent correlation.10

Results

The validation and adaptation process for the Portuguese language was linear. Only one modification was made in order to promote understanding regarding the questions of the form. This was done by replacing the term "lecture" with "traditional class" or “expositive lecture”. In the face validation, all participants reported to fully understand the questionnaire. There was no report on the need for modifications of the evaluation items. The Cronbach's Alpha coefficient was applied after the application of the Test to a population of 80 students. The result for the total of questions was 0.670. After removing questions 12 and 19, the result would change to 0.702 and 0.701, respectively. This would make the result acceptable. The questionnaire was applied again with 30 students (retest). The degree of agreement between the two evaluations was obtained from the ICC and respective confidence interval between the two evaluations for each of the statements or item is expressed in Table 2.

Question

Intraclass correlation coefficient

 

Value

IC 95%

1

0,55

0,26 a 0,76

2

0,82

0,66 a 0,91

3

0,49

0,19 a 0,72

4

0,64

0,37 a 0,81

5

0,46

0,14 a 0,69

6

0,77

0,56 a 0,88

7

0,33

-0,02 a 0,61

8

0,23

-0,11 a 0,54

9

0,41

0,05 a 0,66

10

0,55

0,25 a 0,76

11

0,29

-0,08 a 0,59

12

0,43

0,09 a 0,68

13

0,54

0,23 a 0,75

14

0,39

0,04 a 0,66

15

0,46

0,15 a 0,71

16

0,69

0,45 a 0,84

17

0,24

-0,14 a 0,55

18

0,59

0,30 a 0,78

19

0,25

-0,08 a 0,54

20

0,39

0,06 a 0,65

21

0,53

0,17 a 0,75

22

0,58

0,28 a 0,78

23

0,36

0,01 a 0,63

24

0,41

0,05 a 0,66

25

0,78

0,59 a 0,89

26

0,62

0,35 a 0,79

27

0,67

0,41 a 0,83

28

0,48

0,14 a 0,71

29

0,61

0,31 a 0,79

30

0,14

-0,22 a 0,47

31

0,63

0,35 a 0,80

32

0,49

0,16 a 0,72

33

0,45

0,12 a 0,69

Table 2 Coefficient of intra class correlation between the two evaluations by question

Discussion

Dental education increasingly requires the use of active methodologies suitable for the training of future professionals. In this sense, the ABE method has a much greater potential to prepare students for the professional challenges of the 21st century when compared to the traditional teaching model. In this traditional model, there is a unidirectional knowledge distribution of a teacher to the students.2 However, to verify the appropriateness of the method to its actual application, it is necessary to evaluate the perception regarding the use of the method, as proposed by Mennenga.9 This research was a validation of the TBL-SAI research instrument for a Brazilian population, whose results revealed a good degree of understanding by the respondents. Few adaptations were performed to the questionnaire for Portuguese-speaking students.

In this sense, an analysis was performed to ascertain the intensity of correlation between the items of the questionnaire-Cronbach's Alpha coefficient. This coefficient may increase after the deletion of an item from the measurement scale (questionnaire). If this occurs, it is assumed that this item is not highly correlated with the other items on the scale. Therefore, Cronbach's alpha determines whether the scale is actually reliable because it evaluates how each item reflects its reliability. Thus, the removal of items 12 and 19 provided this change in the application of TBL-SAI in this investigation. It is important to note that the alpha value is affected by the number of items that make up a scale. As the number of items increases, the variance, systematically placed in the numerator, is increased such that an overestimated value of the consistency of the scale is obtained.11 Similarly, the value of Cronbach's alpha can be overestimated if the size of the sample is not taken into account: the larger the number of individuals who fill a scale, the greater the expected variance.12 The ICC calculation revealed a weak correlation between test and retest for questions 7, 8, 11, 14, 17, 19, 20, 23 and 30. This result can be attributed to the fact that TBL-SAI is an extensive form of questions, which would make it difficult to obtain similar results after a period of time. Moreover, since the Likert scale has very close answers, this divergence could be expected to some extent. These data point to the need to adapt the questionnaire, with the possibility of reducing the number of questions, or even reformulation, in order to identify items that could be aggregated into unique questions. From this perspective, we suggest that a simplified TBL-SAI, with 27 questions, be applied to future students of Brazilian dental schools. In this sense, items 7, 8, 11, 12, 19 and 30 would be withdrawn. In addition, future work is needed in order to establish this proposal of evaluation of students' perceptions about TBL, which has shown to be stimulating for the students present a more active position in the search for knowledge, obtaining a greater involvement with the studied subject.6,13,14

Conclusion

The validation process of the Team-Based Learning Student Assessment Instrument for a group of Brazilian students of a dental school was carried out in a satisfactory way. It was verified the need to reduce the quantity of items, in order to make the evaluation of perception in relation to the method of teaching more reliable.

Acknowledgements

None.

Conflict of interest

The authors declare that there are no conflicts of interest.

References

  1. Boysen PG, Daste L, Northern T. Multigenerational challenges and the future of graduate medical education. Ochsner J. 2016;16(1):101−107.
  2. Mitre SM. Metodologias ativas de ensino-aprendizagem na formação profissional em saúde: debates atuais. Ciência & saúde coletiva. 2008;13(2):2133−2144.
  3. Prince M. Does active learning work? A review of the research. JEE. 2004;93(3):223−231.
  4. Bollela VR. Aprendizagem baseada em equipes: da teoria à prática. Medicina. 2014;47(3):293−300.
  5. Abdelkhalek N. Using team-based learning to prepare medical students for future problem-based learning. Med Teach. 2010;32(2):123−129.
  6. Livingston B, Lundy M, Harrington S. Physical therapy students’ perceptions of team-based learning in gross anatomy using the Team-Based Learning Student Assessment Instrument. J Educ Eval Health Prof. 2014;11:1.
  7. Takeuchi H. Effects of team-based learning on fixed prosthodontic education in a Japanese School of Dentistry. J Dent Educ. 2015;79(4):417−423.
  8. Addo-Atuah J. Performance and Perceptions of Pharmacy Students using Team-based Learning (TBL) within a Global Health Course. Innovations in Pharmacy. 2011;2(2):1−15.
  9. Mennenga HA. Development and Psychometric Testing of the Team-Based Learning Student Assessment Instrument. Nurse Educ. 2012;37(4):168−172.
  10. Li L. Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging. J Biomed Opt. 2015;20(5):50801.
  11. Krus DJ, Helmstadter GC. The Problem of Negative Reliabilities. Educational and Psychological Measurement. 1993;53(3):643–650.
  12. Bland J, Altman D. Statistics Notes: Cronbach’s Alpha. BMJ. 1997;314:572.
  13. Haj-Ali R, Al Quran F. Team-based learning in a preclinical removable denture prosthesis module in a United Arab Emirates dental school. J Dent Educ. 2013;77(3):351−357.
  14. Simonson SR. Making students do the thinking: team-based learning in a laboratory course. Adv physiol Educ. 2014;38(1):49−55.
Creative Commons Attribution License

©2019 Souza, et al. This is an open access article distributed under the terms of the, which permits unrestricted use, distribution, and build upon your work non-commercially.