LOGIN
REGISTER.png)
Perceptions and Clinical Applications of Feedback Type
Elmer E. Gonzalez1*
Jay Parkes2
1Assistant Professor and Dental Hygiene Program, Director at New Mexico State University, University of New Mexico US
2Professor of Educational Psychology at the University of New Mexico, US
*Corresponding author: Gonzalez EE, Dental Hygiene Program, Director at New Mexico State University, 3400 South Espina Street, Las Cruces, NM 88011, US, Tel: 575-528-7016, E-mail: Elmer6@unm.edu.
Citation: Gonzalez EE, Parkes J (2019) Perceptions and Clinical Applications of Feedback Type. J Dent Sci Res Ther 2019: 1-9. doi: https://doi.org/10.29199/2637-7055/DSRT-301026
Abstract
Purpose/Objectives: The aim of the study was to evaluate students’ perceptions and clinical applications of feedback in Clinical Dental Hygiene Education in 2019. The study explores Ego-Stroking Sandwich feedback and Non-Sandwich feedback as the feedback types and Instructors feedback vs. Scoring guide feedback as the feedback delivery method.
Methods: Eleven participants participated in the study. An initial questionnaire assessing perceptions of feedback was administered. Participants performed a maxillary and a mandibular impression while receiving a type of feedback; the impressions were evaluated using a scoring guide. A post impression questionnaire was administered to gather participants’ perceptions on preferred feedback delivery method.
Results: The results show that Ego Stroking Sandwich and Motivation (M=3.511, SD=.488) was slightly less effective than Non-Sandwich and Encouragement (M=3.597, SD=.336) and the Instructor Feedback subscale (M =3.28 , SD =.443) had a statistically significant higher mean for students in the Ego Stroking Sandwich condition than the Scoring Guide subscale did (M= 2.67 , SD = .242). The instructor feedback subscale had a statistically significant higher mean (M= 3.70, SD = .253) for students in the Non-Sandwich condition than the Scoring Guide subscale did (M= 2.71, SD = .149). These results indicate that students prefer instructor feedback to scoring guide feedback.
Conclusion: In allied dental education, feedback is the basis for clinical teaching and student skill development. In order to deliver feedback effectively, instructors must understand different types of feedback available and different ways of delivering feedback in the clinical setting to foster self-efficacy.
Keywords: Feedback, Dental Hygiene, Feedback in Dental Hygiene, Feedback in Allied Education, Feedback in Clinical Education, Formative Feedback, Clinical Teaching, Educational Assessment, Education, Dental.
Operational Definitions
Feedback: Verbal delivery of information to the student in order to fill the gap between instruction and learning, address faulty interpretations of performance and/or to improve performance and skills that are in the developmental novice stage.
Ego Stroking Sandwich Type Feedback: In this study it is defined as feedback type given to the student using a combination of praise at the beginning, substantive feedback in the middle, and praise at the end. This type of feedback is many times confused with sandwich type feedback when delivering feedback.
Non-Sandwich Type Feedback: Feedback type given to the student using direct, task oriented, positive or negative feedback without praise at the beginning and end of the statement.
Sandwich Feedback Type: The sandwich type feedback is considered good when it provides positive feedback at the beginning, more feedback in the middle and ends with more positive feedback. This should not be confused with Ego Stroking Sandwich Type feedback that uses ego-stroking statements at the beginning and end.
Introduction
Giving and receiving feedback is important and significant when teaching and learning. When feedback is directed at the task and within the appropriate learning level, it assists students to comprehend, to engage, and to develop effective strategies to process the information intended to be learned. Effective feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and must provide logical. Feedback is a tool that should be used in dental hygiene and other clinical fields for teaching to improve learning and enhance students’ self-efficacy and clinical skill development. Feedback is one way to provide students with valuable information to assess their performance and improve their clinical skills. It is crucial to understand that feedback is the consequence of teaching/learning. In this study, feedback is framed utilizing concepts from Educational Psychology with an emphasis in clinical teaching and learning, specifically within the areas of allied dental education as an example.
Effective feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and must provide logical connections [1]. Feedback is a tool that should be used in dental hygiene and other clinical fields for teaching to improve learning and enhance students’ self-efficacy and clinical skill development. Feedback is one way to provide students with valuable information to assess their performance and improve their clinical skills. In this study, feedback is framed within deliberate [2]; Ericsson, specifically within the areas of clinical dental hygiene education [3].
Deliberate practice is a more effective and efficient framework to guide clinical educators as they grow in expertise from novice students to expert professionals than any of the framework components individually. The deliberate practice framework consists of learners first being motivated to improve and willing to exert effort toward improvement.3 They need to practice skills and tasks that draw on their prior knowledge. They need to receive timely and effective feedback and they need practice - to repeat the performance. Practice without feedback does not develop expertise. Thus feedback is an indispensable part of learning.
Enhancing feedback delivery needs to be fostered in order to encourage effort, self-driven behaviors that promote student motivation and foster self-efficacy. Feedback needs to be given carefully and appropriately, to ensure that self-efficacy is cherished rather than destroyed [4]. Suitable feedback is the one that emphasizes progress, “enhances perceived self-efficacy, aspirations, and efficient analytic thinking, self-satisfaction, and performance accomplishments. Highlighting deficiencies undermines self-regulative influences with resulting deterioration performance” (p 125) [4].
The Feedback Sandwich
The good “feedback sandwich” technique consists of responding to a student’s performance by making positive statements about the task performed at the beginning and the end, and providing criticism in between [5]. Advocates claim that this way of giving feedback helps to build trust, decreases negative feelings about negative comments, and boosts comfort level of the receiver. The feedback sandwich is ineffective when it provides ego boosting praise, criticism in between and ends with praise because this feedback strategy does not necessarily improve learning outcomes.5 Instructors sometimes try to soften the impact of critical feedback by offering positive comments at the beginning and/or end of their comments.
Non-Sandwich Feedback
Teachers can improve instruction and feedback delivery by having clear goals, which can increase performance and goal attainment. Having clear written goals is more effective than those that are vague [6]. Clear goals provide students with tools to become motivated and teachers are then able to provide more specific feedback directed to the task and goal in discussion [6-8].
Feedback is one way to provide students with valuable information to assess their performance and improve their skills based on the feedback they received. “If feedback is directed at the right level, it can assist students to comprehend, engage, or develop effective strategies to process the information intended to be learned. To be effective, feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and to provide logical connections. It also needs to prompt active information processing on the part of learners, have low task complexity, relate to specific and clear goals, and provide little threat to the person at the self-level” (p. 104) [9]. It is crucial to understand that feedback is the consequence of teaching/learning, instruction happens first and feedback second. Without prior instruction or learning, feedback has no use, however, feedback that happens after learning or teaching has a powerful influence on learning [9].
Feedback in the Clinical Settings
Several clinical studies have suggested techniques to improve feedback delivery and methods to improve the use of feedback in the clinical setting during teaching and learning [10-13]. Ramany and Krackov, present a summary of twelve tips for effective implementation of feedback techniques in the clinical setting. Some of the discussion on feedback here, involves the establishing of a respectful environment, goal setting and communicating goals in a clear and direct way so that objectives for feedback are set, focusing feedback on performance, and the use of neutral, specific language to focus on performance. In addition, the success of feedback orientation depends on setting professional development opportunities for faculty and staff and creating an institutional atmosphere where feedback is valued [13]. The use of feedback, concurrent and summative was explored in conjunction with computer-based video instruction when teaching knot-tying skills to first year medical students [14]. Utilizing expert concurrent and summative feedback is beneficial for novice students in the clinical setting [12]. When medical students were exposed to knot-tying, those students who were exposed to expert feedback and computer-based video instruction performed better than those who only received the computer-based video instruction alone.14 In addition, vicarious modeling plays an important role in successful skill development [4,15]. Students are able to model their performance on that of the expert and compare the product [4]. Feedback provides students not only with information regarding their skill performance, but equally important, it provides the opportunity to detect mistakes and correct faulty interpretations.14
Methods
The Institutional Review Board reviewed this study and exempt approval was obtained from The University of New Mexico IRB, reference #24717 and expedite approval was obtained from the New Mexico State University IRB, reference #16097. All participants were English speakers. Participants were not compensated for participation in the study; however, in gratitude for their participation, the investigator donated $150.00 dollars to the Dental Hygiene student club and $150.00 dollars to the Dental Assisting club.
Materials
The Dental Hygiene Procedures Videos- eCommerce Version, 1st edition was used for initial instruction of the procedures to be performed. DXTTR Dental Mannequin was utilized to carry on the impressions using alginate material. All participants received the same type and amount of pre measured alginate and water for each impression according to manufacturer instructions. Spatulation time and speed were not measured as this was something they had to learn from the instructional video and type of feedback received.
Baseline impression
Participants were randomly assigned to a procedure, maxillary X1 or mandibular X2 impression for baseline purposes by selecting either a number one or two in sequence as 1, 2, 1, 2. After the procedures for obtaining baseline were completed, participants were randomly assigned to experimental Group A (Non-Sandwich type) or Group B (Ego Stroking Sandwich type) by choosing either a letter A or a letter B in sequential order as A, B, A, B.
All participants performed a baseline impression, maxillary or mandibular. No feedback and no rubric was given to perform the baseline impression. Participants received an initial questionnaire about the general understanding of feedback and their perceived effectiveness. The initial 20 item questionnaire was adopted and modified from “The Instructional Feedback Orientation Scale: Conceptualizing and Validating a New Measure for Assessing Perceptions of Instructional Feedback” [15].
At baseline, all participants watched the 3-minute video titled “Mixing Alginate” from start to finish. This video was not shown to participants at any other time during the study. A second video was shown with instructions on performing the impression. Participants who landed on the maxillary baseline impression were shown the video titled “Making a Maxillary Preliminary Impression” that lasted 6.29 minutes and participants assigned to mandibular impression were shown the video titled “Making a Mandibular Preliminary Impression” that lasted 6.25 minutes. Video instruction was used at baseline to control for confounding variables and attempt to isolate the effects of feedback on performance rather than arguing that video instruction alone had an effect on performance.
Group A and Group B
Group A- Non Sandwich Feedback, consisted of (n=6, repeated measures) and Group B-Ego Stroking Sandwich Feedback, consisted of (n=5, repeated measures). Each group were first shown the instructional video corresponding to “Making a Maxillary Preliminary Impression”. Each group performed their first impression on the maxillary arch and received the corresponding feedback type, then, participants completed a post impression questionnaire. After completion of the first post impression questionnaire, participants were shown the procedural video “Making a Mandibular Preliminary Impression” and were instructed to proceed to take the impression on the mandibular arch. Then, participants were given a scoring guide to have handy and were instructed to proceed and complete their mandibular impression receiving feedback according to the group assigned. At the conclusion of the mandibular impression, participants turned in the product to the instructors for evaluation. Participants then completed the second post impression questionnaire.
Results
The assumptions were tested and upheld for all analyses. An alpha level of α=0.05 was established. Information in this section is presented in two sections, Part one presents results on instrument analyses and Part two provides results that address the research questions.
Part One-Instrument Analyses
Questionnaires: An exploratory factor analysis [16,17] was attempted on both questionnaires to provide internal structure evidence of validity [18]. Principal Components Analyses (PCA) were substituted for the exploratory factor analyses. The extracted components were rotated using varimax rotation. The decision to define the two components was based on the post-rotation eigenvalues and a visual examination of the scree plot. Both, the initial and after feedback questionnaires yield two components each. The component scales were determined as follows: Initial questionnaire: Ego Stroking Sandwich Feedback and Motivation and Non-Sandwich Feedback and Encouragement. The scales for the after-feedback questionnaire are Instructor Feedback and Scoring Guide Feedback.
Ego Stroking Sandwich Feedback and Motivation. These items had in common Ego Stroking Sandwich type feedback and motivation to learn and or improve the task, therefore, a scale titled Ego Stroking Sandwich and motivation was created. An example of one of the items is “Great job, When mixing alginate material, remember to utilize an adequate powder to water ratio; good job overall” (Table 1).
Table 1: Initial perceptions questionnaire. |
Scoring Guide Feedback. The second component had in common scoring guide feedback (Table 2). Some examples of the items here are “Using a rubric helps me to perform a clinical skill in more detail” and “Using a scoring guide in addition to feedback helps me improve the outcome of my clinical performance”
Table 2: After feedback type questionnaire items. |
Instructor Feedback. PCA Analyses yield two components (Table 2). The similarities between these items were instructor feedback. An example of the items is “The feedback I just received from the instructor was useful to improve my understanding of the technique I was performing”.
Non-Sandwich Feedback and Encouragement. Upon Analysis of the items, the general commonalities between them were Non-Sandwich type feedback and Encouragement. An example of one of the items is “Your instrumentation skills can improve by rolling your instrument from the mesial to the distal in an exploratory motion and small strokes. In order to prevent tissue trauma, it is important to adapt the instrument to the tooth at all times. Start at the distobuccal line angle engaging the blade on the side towards you and keeping your instrument touching the tooth at all times, then, overlap at the distobuccal line angle and engage the opposite blade on the same side of the instrument and keeping the instrument adapted to the tooth, proceed mesially, following the exploratory strokes we talked about earlier.” Participants’ perceptions on this type of feedback and encouragement showed to be relatively equal to sandwich and motivation (Table 1).
A reliability analysis was conducted on the scores obtained from each of the components produced by the PCA analyses.
Inter-Rater Reliability Study Prior to the study, both instructors were calibrated by the investigator using 4 impressions, two maxillary and two mandibular. Inter-Rater Reliability analyses were conducted on the four scores received by the instructors. For all participant’s impressions, the instructors used the scoring guide and pre-determined criteria. Instructors also used a consensus method and photographs to determine a final score on each impression. The inter-rater reliability was based on ratings obtained from training session prior to study and was calculated as Cronbach’s Alpha (.94).
Part Two-Analyses to Answer Research Questions
Research Question #1:
Which type of feedback, Non-Sandwich or Ego Stroking Sandwich do students perceive to be superior to improve their clinical skills and performance?
Ego Stroking Sandwich and Motivation (M=3.511, SD=.488) was perceived to be slightly less effective compared to Non-Sandwich and Encouragement (M=3.597, SD=.336) on the initial questionnaire by the 11 participants. This difference was not statistically significant when tested with a one-way repeated measures ANOVA (F(1,10) .209, MSe=0.40, p=.657).
Research Question #2:
Is the use of a scoring guide, paired with instructor feedback better than instructor feedback alone for student’s perceptions of their clinical performance?
This research question was analyzed using a mixed two-factor analysis of variance where subscale was the within subjects factor and feedback delivery method was the between subjects factor.
The interaction between feedback delivery method and subscale was statistically significant (Wilks’ Lambda= 0.568, F (1.9) = 6.859, p = .028.) The interaction was followed up with simple main effects tests. Subscale differences were not statistically significant for Instructor feedback subscale (F(1.9)= 3.908, MSe= .481, p=0.79 ) or Scoring Guide subscale (F(1.9)= 0.90, MSe= .003, p=.771).
The two subscales were statistically significantly different within each feedback type condition. The instructor Feedback subscale (M=3.28, SD=.443) had a statistically significant higher mean for students in the Ego Stroking Sandwich condition than the Scoring Guide subscale did (M=2.67, SD=.242) (Wilks’ Lambda=.225, F (1.9) =31.057, p=.000). Subscale and feedback simple main effects are shown in figure 1. Similarly, the instructor feedback subscale also had a statistically significant higher mean (M=3.70, SD=.253) for students in the Non-Sandwich condition than the Scoring Guide subscale did (M= 2.71, SD = .149) (Wilks’ Lambda=.083, F (1.9) =99.786, p=.000).
|
Figure 1: Simple main effects subscales and feedback type. |
Research Question #3:
Which type of feedback, (No Feedback, Non-Sandwich or Ego Stroking Sandwich type) shows to be superior on students’ clinical performance?
An ANCOVA was conducted to assess the effects of Non-Sandwich versus Ego Stroking Sandwich feedback on student clinical performance using No Feedback as a covariate. The analyses showed no statistically significant differences between Non-Sandwich and Ego Stroking Sandwich feedback between groups, (F (1.8)=2.852, MSe=13.239, p = .130).
Although there are no statistical significant results on feedback type effectiveness between Non-Sandwich and Ego Stroking Sandwich feedback on student performance, based on mean differences, we can observe that Non-Sandwich Feedback condition is better (M=16.083, SD=2.222) in comparison to Ego Stroking Sandwich Feedback (M=13.900, SD=1.781).
Figure 2 shows a breakdown of marginal means of impression depending on the feedback type received.
|
Figure 2: Estimated means of impression by feedback type. |
Discussion on Participant’s Perceptions of Feedback
Research Question #1
The initial questionnaire was administered to gain a general sense of how students perceived feedback and which type of feedback (ego stroking sandwich or non-sandwich) they perceived to be most effective for learning and performance in the clinical setting.
Based on students’ perceptions we can ascertain that there is no difference on feedback type and students’ perceptions of its effectiveness in learning and performance. We can see a very small mean difference between the two where Non-Sandwich type feedback was perceived slightly better but because the mean difference is so small, we conclude that participants did not perceive Ego Stroking Sandwich feedback to be different than Non-Sandwich feedback or vice versa.
Feedback is crucial for student learning and skill development. Instructors of dental assisting and dental hygiene clinical courses ought to be familiar with feedback delivery methods and what strategies have shown to be most effective. The bulk of the literature show that feedback specificity is important for student outcomes and performance [4,5,8-10,14]. In order to look into feedback type and task performance; it is important to recognize what perceptions students bring into the teaching and learning environment so that our approach to deliver feedback is based on the idea of improving student learning and performance. Instructors ought to provide students with tools that improve their ability to focus on the task and either improve what they are doing or address areas that are in need of additional instruction [12,19-24].
Research Question #2
The two factor ANOVA with simple main effects yield results indicating that participants perceived instructor feedback more valuable than scoring guide feedback paired with instructor feedback within groups and across groups. Another observation was that participants perceived instructor feedback to be most effective while receiving Non-Sandwich type feedback. The video recordings were used as a fidelity check only to ensure that each group received feedback pertaining to the group they were in. Example, the Non-Sandwich group received Non-Sandwich “Based on the feedback I gave you before you had to load the tray very fast, now on this one, instead of loading a big chunk you want to split the tray in half. You want to load this half first, then this other half of the tray and then smooth it, okay…” Similarly, the Ego Stroking group received Ego Stroking feedback “Do you have any questions. I think you guys are fabulous; you did a fantastic job with maxillary impressions. When placing the mandibular tray, you want to make sure that you retract the lips of DEXTTR with force because the lips are tight, then you place the tray as instructed in the video.”
During the mandibular impression, participants were able to have the scoring guide used by the instructors handy. One of the assumptions in this situation was that the scoring guide served more as a distractor than a useful tool for students to improve clinical performance. Because participants lacked experience taking impressions on a DEXTTR and using the fast set alginate, adding additional factors such as the scoring guide, could have caused some cognitive overload, thus, their perception of using a rubric did not seem to be superior to feedback alone.
Clinical Skill Evaluation (CSE) Discussion:
When looking at the descriptive statistics, there is mean difference between feedback type and clinical performance point in a positive direction, thus, a possible bigger sample size may yield results that are more powerful. Figure 2 shows the estimated means of impression by feedback type. Looking at the average means for feedback type, Non-Sandwich feedback (M= 16.08, SD 2.222) is greater than Ego Stroking Sandwich feedback (M= 13.9, SD=1.781) indicating that those who received Non-Sandwich feedback performed better in the clinical procedure of taking impressions.
Baseline CSE:
The baseline CSE was administered to participants to obtain baseline data in order to compare any changes seen after feedback was delivered. Participants who had prior experience as dental assistants or were in the dental assisting program obtained better scores than those who had no prior experience at the time of the study.
No assumptions were made as to whether or not prior experience would play a role due to the logical explanation that prior experience usually does have an effect on performance. What is important to note is the fact that even those with prior experience at taking impressions showed improvements after feedback type was provided compared to baseline scores.
Research Question #3
The simple main effects Analysis of covariance performed on the clinical skill evaluations using baseline scores as a covariate shed some light on the effects of feedback type between groups.
Although no statistical significance was found on the ANCOVA, the mean differences on the descriptive statistics show that those individuals who received Non-Sandwich feedback performed better impressions compared to those who received Ego Stroking Sandwich type feedback regardless of prior experience and impression type.
Mean differences also show a trend that indicates that Non-Sandwich feedback was most effective during the maxillary impression for both groups compared to the mandibular impression. It is important to note, that during the mandibular impression, participants had access to the scoring guide used to evaluate the overall quality of the impressions. When students are in the novice stages, their focus is devoted to the one task they are performing. In this case, the participants’ cognitive and psychomotor skills were all being utilized by taking an impression. Their cognitive capacity at the time was fully loaded allowing little to no space for a rubric to be read while performing the procedure and trying to attend to feedback. This observation aligns with other research findings [15].
The findings on this study are also similar to those reported by Xeroulis and colleagues, 2006 where medical students were exposed to knot-tying. In that study, those students who were exposed to expert feedback and computer-based video instruction performed better than those who only received the computer-based video instruction alone [14] which is similar to the results seen on this study. Students at baseline were only exposed to video instruction and no expert feedback, thus, their scores were lower compared on their clinical skill evaluations compared to the scores obtained after video instruction and receiving feedback type. This study did not analyze the differences between expert feedback and video instruction but went beyond and looked into the differences of feedback type on students’ performance keeping video instruction as a controlled variable that everyone received.
Practical Implications
The results of this study should be considered as an approach to understand feedback type in clinical applications with novice students as well as a method to determine the feedback type each clinical instructor wants to use while teaching clinical procedures. It is also crucial to understand that allied and dental students are performance oriented, thus, providing feedback that improves performance and self-efficacy is essential for student success. Furthermore, it is possible that the question is not only in relation to the type of feedback offered but how the feedback is offered. Immediate feedback may be more useful than delayed feedback in the clinical setting as well as feedback from the instructor is more effective than feedback from a scoring guide [9].
Based on the results and observations of this study; the study shows promising paths to better understand the effects of feedback type in clinical performance as well as the ability for instructors to recognize that utilizing too many methods for feedback delivery at once could be counterproductive for novice students. It is best to focus on one method of delivering feedback and the feedback ought to be aimed at the task such as the one delivered by the instructor, non-Sandwich type, in this study.
Conclusion
This study explored student perceptions of feedback type as well as clinical applications in dental hygiene and dental assisting while using feedback type as an independent variable to assess the outcome of the impressions taken. This study has not only looked at perceptions of feedback but also included clinical applications while measuring the outcome after feedback type had been provided.
The results on this study have shown that although students learn with feedback regardless of feedback type; the non-sandwich type shows trends to be better to improve clinical performance based on the descriptive statistics. Feedback delivery method also showed to be important in this study.
References
1. Akinbobola O, Adeleke A (2012) Educator's Self-Efficacy and Collective Educators' Self-Efficacy among University Academic Staff: An Ethical Issue. Ife Psychologia 20: 57-69.
2. Ericsson KA, Krampe RT, Tesch-Romer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychological review 100: 363.
3. Ericsson KA (2015) Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med 90: 1471-1486.
4. Bandura A (1993) Perceived Self-Efficacy in Congnitive Development and Function. Educational Psychologist 28: 117-148.
5.Parkes J, Abercrombie S, McCarty T (2013) Feedback sandwiches affect perceptions but not performance. Adv in Health Sci Educ 18: 397-407.
6.Dweck C (2008) The Perils and Promises of Praise. Educational Leadership 65: 34-39.
7.Dweck CS (2010) Mind-Sets and Equitable Education. Principal Leadership PP: 26-29.
8.Dweck C (2016) Praise the effort, not the outcome? Think again. The Times Educational Supplement.
9.Hattie J, Timperley H (2007) The Power of Feedback. Review of Educational Research 77: 81-112.
10.Cantillon P, Sargeant J (2008) Giving feedback in clinical settings. BMJ 1292-1294.
11.Fugill M (2005) Teaching and learning in dental student clinical practice. Eur J Dent Educ 9: 131-136.
12.Hauser AM., Bowen DM (2009) Primer on Preclinical Instruction and Evaluation. J Dent Educ 73: 390-398.
13.Ramani S, Krackov SK (2012) Twelve tips for giving feedback effectively in the clinical environment. Med Teach 34: 787-791.
14.Xeroulis GJ, Park J, Moulton CA, Reznick RK, Leblanc V, et al. (2006) Taching suturing and know-tying skills to medical students: A randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 141: 442-449.
15.King PE, Schrodt P, Weisel JJ (2009) The Instructional Feedback Orientation Scale: Conceptualizing and Validating a New Measure for Assessing Perceptions of Instructional Feedback. Communitcation Education 58: 235-261.
16.Costello AB, Osborne JW (2005) Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most From Your Analysis. Practical Assessment, Research & Evaluation 10: 1-9.
17.Yong AG, Pearce S (2013) A Beginner's Guide to Factor Analysis: Focusing on Exploratory Factor Analysis. Tutorials in Quantitative Methods for Psychology 9: 79-94.
18.Messick S (1993) Foundations of Validity: Meaning and Consequences in Psychological Assessment. Princeton, New Jersey: Educational Testing Service.
19.Boehler ML, Rogers DA, Schwind CJ, Mayforth R, Quin J, et al. (2006) An investigation of medical student reactions to feedback: a randomised controlled trial. Med Educ 40: 746-749.
20.Busser D (2012) Delivering Effective Performance Feedback. T+D 66: 32-35.
21.Hattie J, Fisher D, Frey N (2016) Do They Hear You? Educational Leadership. EDL 73: 16-21.
22. Kluger AN, DeNisi A (1996) The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory. Psychological Bulletin 119: 254-284.
23. Kluger AN, DeNisi A (1998) Feedback Interventions: Toward the Understanding of a Double-Edged Sword. Current Directions in Psychological Science 7: 67-72.
24.Shute VJ (2008) Focus on Formative Feedback. Review of Educational Research 78: 153-189.