In UME, we often discuss not only how to improve the quantity of responses we receive on student evaluations but also how to improve the quality of the responses students provide. In examining data quality, we have recently begun exploring satisficing in the student evaluation process. When one engages in satisficing while completing a survey, evaluation, etc., they are simply using enough energy to produce a suitable response rather than the best possible response (Kaminska, McCutcheon, & Billiet, 2010; Krosnick, Narayan, & Smith, 1996). Straight-lining items can be an example of strong satisficing with respondents failing to put much thought in their responses and simply selecting the same response for an item set (Cole, McCormick, & Gonyea). Various factors can contribute to a respondent engaging in satisficing while completing student evaluations of courses and instructors such as motivation.
Recently, we explored responses to student evaluations of clerkships during the 2012-2013 academic year looking at the percentage of straight-lined evaluations as well as the average number of comments provided. The thinking here was that simply responding to each item with the same number does not necessarily confirm that satisficing took place; however, if one also failed to provide comments, that may speak more to a lack of effort put into the responses provided. Reviewing student evaluations of clerkships for straight-lining and comments provided produced some interesting results. Specifically, the results for the OBGYN clerkship compared to all clerkships stood out.
The percentage of straight-lined evaluations was much lower for the OBGYN clerkship when compared to all clerkships. Less than 10% (9.70%) of student evaluations for OBGYN were straight-lined which was much less than the percentage for all clerkships (17.76%). Additionally, the average number of comments provided on student evaluations of OBGYN was 5.34 which was much greater than the average number of comments for all clerkships (2.97). To better understand these findings, we reached out to the Vice Chair of Education for OBGYN, Dr. Mark Di Corcia, and found that the OBGYN clerkship has built student evaluation of the clerkship and preceptors into the clerkship with students having time to complete their evaluations. Also, the clerkship shares a presentation that stresses the value placed on student feedback, specifically constructive qualitative feedback, and how data have been used to improve the clerkship.
Before the close of administration, UME will begin contacting course and clerkship directors with updates on response rates so that they can further encourage students to provide feedback. However, all instructors and preceptors can promote student motivation and encourage student feedback by talking with your students about how you have used data to improve teaching and better promote student learning. Examples could be a brief presentation similar to OBGYN or dedicating a section of your syllabus/handouts to highlighting how you have used student feedback to improve. Stressing the importance of student feedback may not only lead to greater data quantity but also quality.
REFERENCES
Cole, J. S., McCormick, A. C., & Gonyea, R. M. (2012). Respondent use of straight-lining as a response strategy in education survey research: Prevalence and implications. A paper presented at the annual meeting of the American Educational Research Association, Vancouver, BC, Canada.
Kaminska, O., McCutcheon, A. L., & Billiet, J. (2010). Satisficing among reluctant respondents in a cross-national context. Public Opinion Quarterly, 74(5), 956-984.
Krosnick, J. A., Narayan, S., & Smith, W. R. (1996). Satisficing in surveys: Initial evidence. In M. T. Braverman and J. K. Slater (Eds.), New Directions for Evaluation (Advances in Survey Research), 70, 29-44.
