Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1 -- 2700 2024-01-24 23:22:01 |
2 Reference format revised. -5 word(s) 2695 2024-01-25 01:51:45 |

Video Upload Options

Do you have a full video?

Confirm

Are you sure to Delete?
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Bordere, J.; Carter, F.; Caudill, S.; Mixon, F. Student Evaluations of Teaching. Encyclopedia. Available online: https://encyclopedia.pub/entry/54313 (accessed on 21 May 2024).
Bordere J, Carter F, Caudill S, Mixon F. Student Evaluations of Teaching. Encyclopedia. Available at: https://encyclopedia.pub/entry/54313. Accessed May 21, 2024.
Bordere, Jasmine, Fonda Carter, Steven Caudill, Franklin Mixon. "Student Evaluations of Teaching" Encyclopedia, https://encyclopedia.pub/entry/54313 (accessed May 21, 2024).
Bordere, J., Carter, F., Caudill, S., & Mixon, F. (2024, January 24). Student Evaluations of Teaching. In Encyclopedia. https://encyclopedia.pub/entry/54313
Bordere, Jasmine, et al. "Student Evaluations of Teaching." Encyclopedia. Web. 24 January, 2024.
Student Evaluations of Teaching
Edit

Student evaluations of teaching (SETs) reflect a broad range of objective and subjective qualities of instructors. These relate to academic discipline, gender and other demographics, and teaching experience. Student evaluations are a standard component of the way colleges and universities assess the quality of an instructor’s teaching for purposes of promotion and tenure, merit raise allocations, and reappointment. 

economics education teaching quality student evaluations of teaching

1. Academic Discipline Effects in SETs

Ongeri [1] takes a deep dive into the SET literature published over the prior 20 years, which consistently puts the quality of instruction in economics below that in most other academic disciplines. The relatively lower quality of instruction in economics tends to be the result of its math orientation, the lack of organization and presentation skills of its instructors, the utilization of multiple-choice tests in assessing economics literacy, and employment of “chalk and talk” teaching approaches that require little, if any, active participation by students. As Asarta et al. [2] point out, this intensive use of lecturing is a sustained practice in the economics discipline. Ongeri [1] concludes that low SETs in economics courses likely reflect a real, underlying problem with the adequacy of university economics education. Relatedly, econometric analysis of a panel of SET data from undergraduate economics courses presented in other studies finds that SETs are a function of class size [3][4][5] and instructor age [4]. Not only does rapport between instructors and students deteriorate with increasing class size, but large lecture enrollments require the use of the types of standardized tests that Ongeri [1] explains reduce SET scores. Additionally, the finding in McPherson et al. [4] that economics students prefer younger instructors to older instructors is consistent with the finding in Ongeri [1] of the low regard held by students of antiquated teaching methodologies.

2. Gender and Other Demographic Effects in SETs

The potential for gender discrimination in SETs has been the subject of a number of studies. The study of a potential gender effect in SETs by Wagner et al. [6] is perhaps the most comprehensive of its type. It analyzes a unique dataset featuring mixed teaching teams and a diverse group of students and teachers. The blended co-teaching approach allows for the examination of the link between SETs and instructor gender (and ethnicity) in a way that encompasses within-course variations. The analysis finds a negative effect of being a female instructor on SETs equal to about 25% of the sample standard deviation of SETs [6]. More specifically, the results suggest that female instructors are 11 percentage points less likely to meet the SET threshold for promotion to associate professor than are their male counterparts [6]. Boring’s [7] application of SET data from a French university to both fixed effects and generalized ordered logit regression analyses finds that male students express a bias in favor of male professors. Among the individual teaching dimensions, Boring [7] finds that students’ SETs match gender stereotypes. For example, male professors are perceived by both male and female students as being more knowledgeable in the subject and exhibiting superior class management. Interestingly, the analyses also suggest that students appear to learn as much from female professors as they do from male professors [7].
A recent study by Mengel et al. [8] employs a quasi-experimental dataset of 19,952 student evaluations of university faculty in a context where students are randomly allocated to female or male instructors. Even though students’ grades and effort are unaffected by the instructor’s gender, the results suggest that female instructors receive systematically lower SETs than their male colleagues. This bias is driven by male students’ evaluations and is particularly pronounced for junior female faculty [8]. Mengel et al. [8] add that gender bias in teaching evaluations may alter the career progression of women by affecting junior women’s confidence, as well as through the reallocation of instructor resources away from research and toward teaching. Keng [9] examines SET data from a public university in Taiwan in order to test for statistical discrimination against female instructors. In doing so, the study relies upon a learning model wherein the instructors’ value added to grades is used to measure teaching effectiveness. Empirical results support the presence of gender bias in SETs, particularly by male students and in science- and math-oriented academic disciplines that employ relatively few female instructors [9]. As one moves toward academic disciplines wherein female instructors are more common, the ratings of female instructors by female students rise. Lastly, in light of earlier findings of a gender effect in SETs, Buser et al. [10] investigate whether gender differences in SETs vary over the course of a semester. Their study examines the application of SETs in principles of economics courses at multiple institutions on three separate occasions during the semester in order to determine whether the evaluations of male and female instructors change throughout an academic term, specifically after the first exam is returned. Tests presented by Buser et al. [10] point toward a negative effect on evaluations for female instructors relative to male instructors associated with returning grades, thus highlighting the importance of temporal effects (related to gender) in the application of SETs.
A number of studies have found that race/ethnicity bias is also present in SETs (e.g., [11][12][13][14][15]). One example is the experimental approach employed by Chisadza et al. [11] that randomly assigned South African students to various course lecturers who all used the same instructional materials. They report that black lecturers received lower SETs than white lecturers, and that this result held even for black students [11]. Relatedly, Chávez and Mitchell [13] utilize a quasi-experimental design wherein instructors recorded welcome videos that were presented to students at the beginning of an online course, thus revealing instructors’ race/ethnicity. Examination of post-course SETs revealed that non-white instructors received lower SETs than their white counterparts, holding constant course content, assignments, schedules and communications [13]. A similar study by Basow et al. [16] indicates that brief lectures presented by computer-animated instructors who vary by race are associated with biased SETs, where the white animated actors receive higher SETs than their black counterparts.
Lastly, a recent strand of the literature focuses on the abusive nature of SETs, particularly with regard to anonymized open-ended comments from students (e.g., [17]). Extensive studies by Jones et al. [18], Tucker [19] and Uttl and Smibert [20] document that abusive comments are often present in SETs, and that most are directed towards women and other minority groups. New research by Heffernan [17] examines results from a survey of 674 academics about abusive comments as well as the anonymized student comments attached to SETs at the 16,000 higher education institutions that collect this information each semester. According to the survey, 59% of academics report having been the target of abusive language in open-ended comments attached to SETs. As a result, two-thirds of this group reported mental health declines, while about one-sixth of this group sought professional medical help [17]. Heffernan’s [17] results support prior work by Jones et al. [18], Tucker [19] and Uttl and Smibert [20] by indicating that the brunt of abusive language in students’ open-ended comments is aimed at female and other minority instructors. Survey evidence indicates that about 52% of males report having received abusive comments, compared to 63% of women and 60% of non-binary individuals [17]. Additionally, while 55% (60%) of straight men (straight women) report having received abusive comments, 64% (83%) of gay (lesbian) instructors report having been on the receiving end of such abuse [17]. Lastly, as Heffernan [17] also points out, exploration by DiPietro and Faye [21] and Hamermesh and Parker [22] finds at least limited evidence of traditional SET discrimination against instructors who are visibly disabled, or who are viewed by students as not being heterosexual or a binary gender.

3. Grading Effect in SETs

Krautmann and Sander [23] expertly qualify the problems with SETs in relation to students’ grades in economics. As they point out, if SETs can be improved through the assignment of higher grades, then they are a flawed instrument for the evaluation of teaching. This flaw may be contributing to grade inflation, unmeritorious decisions regarding tenure and promotion, and a dilution of the signaling role of educational credentials in screening workers for the labor market [23]. Regression results presented in both Krautmann and Sander [23] and the aforementioned studies by McPherson [3] and McPherson et al. [4] support the latter’s explanations in finding a positive and significant relationship between SETs and students’ grade expectations. In fact, all three studies conclude that instructors can “buy” better SET scores by inflating students’ grade expectations. Lastly, a more recent study by Matos-Díaz and Ragan [24] asserts that, because of risk aversion, SETs are dependent upon the characteristics of the distribution of class grades. Matos-Díaz and Ragan [24] find support for this assertion in their own analysis of SETs from the University of Puerto Rico, which indicates that SETs are significantly and negatively related to the variance of expected grades, implying that faculty may be able to boost their SETs by narrowing the grade distribution, particularly in the case of the weakest students.

4. Experience Effect in SETs

Although prior research reports that SET scores are not impacted by an instructor’s experience or academic rank [5], these potential determinants have remained a focus of subsequent studies. For example, McPherson [3] employs a longitudinal approach in examining 607 economics classes over 17 semesters in order to account for unobserved heterogeneity. In the case of economics principles classes, McPherson [3] and McPherson et al. [4] find that the level of experience of the instructor is a significant determinant of SET ratings. A more recent study by Alauddin and Kifle [25] applies SET data from an elite Australian university to a partial proportional odds model to investigate the influence of students’ perceptions of instructional attributes on SETs. Among its many findings, the study reports that instructors below the rank of associate professor earned higher SETs [25]. From a U.S. perspective, the highest performers would include instructors, lecturers and assistant professors. Lastly, Keng’s [9] examination of SETs from a public university in Taiwan that relies upon a learning model to measure teaching effectiveness finds that instructor experience is positively related to teaching effectiveness. More specifically, the study reports that the gender bias in teaching evaluations is reduced by nearly 50% after 10 years of teaching [9].

5. Efficacy of RMP Data in Academic Research

Low response rates associated with SETs, particularly those conducted online, often raise questions about the validity of RMP data as they could suffer from non-response bias [26]. To address this issue, Layne et al. [27] examine SET data across five academic disciplines from two groups of students (n = 2453) at a large university, one of which completed paper-and-pencil SET surveys while the other utilized an electronic mode. Statistical analysis discussed in the study indicated that response rates differ by mode of administration. Even so, the actual SET ratings were not significantly influenced by the survey method, suggesting that the electronic survey mode is a viable alternative to the paper-and-pencil mode of administration [27]. A similar examination by Avery et al. [28] of SET data from a large economics-based public policy program at Cornell University revealed that although Web-based evaluation methods lead to lower response rates, these lower response rates do not appear to impact mean evaluation scores. This result suggests that SET ratings are not adversely affected by switching from paper to online evaluations [28]. Lastly, a more recent study by Nowell et al. [29] uses the Heckman [30] two-step selection correction procedure to account for potential sample selection bias in online SETs and finds no existence of such bias. Tangential research by Bleske-Rechek and Michels [31] examines data on the use of RMP from 208 students at a regional public university and finds that the characteristics of students who tend to post ratings on RMP do not differ from those who do not post on the website.
A number of academic studies compare the ratings of faculty across in-house SETs and those posted to RMP. For example, studies by Coladarci and Kornfield [32], Timmerman [33], Albrecht and Hoopes [34], Brown et al. [35] and Sonntag et al. [36] compare RMP ratings with official institution-administered SETs and conclude that ratings from the two delivery modes are highly correlated. More specifically, Coladarci and Kornfield [32] compare SET data for both an in-house instrument and RMP across 426 instructors at the University of Maine and find a correlation coefficient of 0.68. Timmerman [33] compares similar data on instructors from the University of Tennessee and the University of Colorado, Boulder, and finds correlation coefficients of 0.67 and 0.77, respectively. Albrecht and Hoopes [34] compare RMP data to official evaluation data on 243 faculty from the business schools of one large private research-oriented university and one large public teaching-oriented university in the U.S. They find correlations ranging from 0.62 to 0.81, suggesting that students could legitimately use RMP data to compare different professors within a university. A reliability analysis of RMP and SET ratings of 312 faculty at Brooklyn College by Brown et al. [35] revealed strong internal consistency between the two evaluation formats. In terms of overall quality, the two formats produced a correlation coefficient of 0.64 [35]. Sonntag et al.’s [36] comparison of RMP and in-house SET ratings of 126 faculty at Lander University produces a correlation coefficient of 0.69. This, they argue, indicates that the quality ratings from RMP provide students with information about instructors that is comparable to the information they would have if institutionally administered evaluations were made available to them [36]. Lastly, related research by Otto et al. [37] analyzes the pattern of relationships of RMP ratings for 399 randomly selected faculty. Their results indicate that the pattern of RMP ratings is consistent with the pattern expected of a valid measure of student learning [37].

6. Use of RMP Data in Academic Research

RMP provides the largest publicly available online SET data source, and it has been widely employed in the business education literature (e.g., [38][39]). Many studies discuss straightforward analyses of the determinants of RMP ratings. Constand and Pace [40], for example, compare RMP evaluation scores of finance faculty to those of other disciplines, finding that perceived course difficulty is an important determinant of students’ ratings. Boehmer and Wood [41] examine RMP data to explore the role of faculty gender and course rigor on students’ ratings of faculty. Their results indicate that students prefer male instructors and instructors who offer less rigorous courses, as these instructors receive the highest RMP scores [41]. Similarly, Constand et al. [42] report that accounting students perceive their professors to be significantly more difficult than students in non-accounting disciplines and this perception is related to lower teaching evaluations. More recently, Constand and Clarke [43] and Constand et al. [44] examined a large sample of RMP instructor ratings for undergraduate courses taught at nine Florida universities and found that instructors who teach introductory/core courses earn lower ratings than instructors who teach either advanced or very advanced/capstone courses, and that instructors who teach advanced courses earn lower ratings than those who teach very advanced/capstone courses. In other words, these studies find a cascading effect where instructor ratings increase as course level increases.
Carter [45] investigates the notion that research-active faculty offer superior instruction. Controlling for faculty gender and rank, tuition, and other student and institution effects, results suggest that faculty scholarship is positively related to RMP ratings only for male faculty who publish in elite journals [45]. A number of prior studies have used information on instructor attractiveness found in previous iterations of the RMP platform. Smith [46], for example, finds that relatively more attractive instructors, a notion captured by the proportion of an instructor’s RMP ratings that include a designation indicating that the instructor is viewed by the raters as being physically attractive, earn RMP ratings that are significantly greater than their counterparts, holding constant course rigor, institutional type and selectivity, and tuition. Mixon and Smith [47] extend this analysis by showing that relatively more attractive instructors tend to offer more rigorous courses, according to RMP data, than their counterparts. They argue that this result suggests that by trading on their attractiveness in offering more rigorous courses as opposed to using their attractiveness as a supplement to offering a less rigorous course, relatively attractive faculty are able to maintain a relatively good standing amongst their departmental or unit peers [47]. Finally, studies by Green et al. [48][49] use RMP data on instructor attractiveness to explain self-sorting in higher education. More specifically, these studies find that relative attractiveness is a significant predictor of institutional choice in higher education, whereby attractive faculty are more likely to choose employment at liberal arts institutions, where teaching is the primary responsibility, in order to capitalize on their looks [48][49].

References

  1. Ongeri, J.D. Poor student evaluation of teaching in economics: A critical survey of the literature. Australas. J. Econ. Educ. 2009, 6, 1–24.
  2. Asarta, C.; Chambers, R.; Harter, C. Teaching methods in undergraduate introductory economics courses: Results from a sixth national quinquennial survey. Am. Econ. 2021, 66, 18–28.
  3. McPherson, M.A. Determinants of how students evaluate teachers. J. Econ. Educ. 2006, 37, 3–20.
  4. McPherson, M.A.; Jewell, R.T.; Kim, M. What determines student evaluation scores? A random effects analysis of undergraduate economics classes. East. Econ. J. 2009, 35, 37–51.
  5. Liaw, S.-H.; Goh, K.-L. Evidence and control of biases in student evaluations of teaching. Int. J. Educ. Manag. 2003, 17, 37–43.
  6. Wagner, N.; Rieger, M.; Voorvelt, K. Gender, ethnicity and teaching evaluations: Evidence from mixed teaching teams. Econ. Educ. Rev. 2016, 54, 79–94.
  7. Boring, A. Gender biases in student evaluations of teaching. J. Public Econ. 2017, 145, 27–41.
  8. Mengel, F.; Sauermann, J.; Zölitz, U. Gender bias in teaching evaluations. J. Eur. Econ. Assoc. 2019, 17, 535–566.
  9. Keng, S.-H. Gender bias and statistical discrimination against female instructors in student evaluations of teaching. Labour Econ. 2020, 66, 101889.
  10. Buser, W.; Hayter, J.; Marshall, E.C. Gender bias and temporal effects in standard evaluations of teaching. AER Pap. Proc. 2019, 109, 261–265.
  11. Chisadza, C.; Nicholls, N.; Yitbarek, E. Race and gender biases in student evaluations of teaching. Econ. Lett. 2019, 179, 66–71.
  12. Fan, Y.; Shepherd, L.J.; Slavich, D.; Waters, D.; Stone, M.; Abel, R.; Johnston, E.L. Gender and cultural bias in student evaluations: Why representation matters. PLoS ONE 2019, 14, 0209749.
  13. Chávez, K.; Mitchell, K.M.W. Exploring bias in student evaluations: Gender, race, and ethnicity. Political Sci. Politics 2020, 53, 270–274.
  14. Wang, L.; Gonzalez, J.A. Racial/ethnic and national origin bias in SET. Int. J. Organ. Anal. 2020, 28, 843–855.
  15. Kreitzer, R.J.; Sweet-Cushman, J. Evaluating student evaluations of teaching: A review of measurement and equity bias in SETs and recommendations for ethical reform. J. Acad. Ethics 2022, 20, 73–84.
  16. Basow, S.; Codos, S.; Martin, J. The effects of professors’ race and gender on student evaluations and performance. Coll. Stud. J. 2013, 47, 352–363.
  17. Heffernan, T. Abusive comments in student evaluations of courses and teaching: The attacks women and marginalized academics endure. High. Educ. 2023, 85, 225–239.
  18. Jones, J.; Gaffney-Rhys, R.; Jones, E. Handle with care! An exploration of the potential risks associated with the publication and summative usage of student evaluation of teaching (SET) results. J. Furth. High. Educ. 2014, 38, 37–56.
  19. Tucker, B. Student evaluation surveys: Anonymous comments that offend or are unprofessional. High. Educ. 2014, 68, 347–358.
  20. Uttl, B.; Smibert, D. Student evaluations of teaching: Teaching quantitative courses can be hazardous to one’s career. PeerJ 2017, 5, e3299.
  21. DiPietro, M.; Faye, A. Online student-ratings-of-instruction (SRI) mechanisms for maximal feedback to instructors. In Proceedings of the 30th Annual Meetings of the Professional and Organizational Development Network, Milwaukee, WI, USA, 2005.
  22. Hamermesh, D.S.; Parker, A. Beauty in the classroom: Instructors’ pulchritude and putative pedagogical productivity. Econ. Educ. Rev. 2005, 24, 369–376.
  23. Krautmann, A.C.; Sander, W. Grades and student evaluations of teachers. Econ. Educ. Rev. 1999, 18, 59–63.
  24. Matos-Díaz, H.; Ragan, J.F., Jr. Do student evaluations of teaching depend on the distribution of expected grade? Educ. Econ. 2010, 18, 317–330.
  25. Alauddin, M.; Kifle, T. Does the student evaluation of teaching instrument really measure instructors’ teaching effectiveness? An econometric analysis of students’ perceptions in economics courses. Econ. Anal. Policy 2014, 44, 156–168.
  26. Bacon, D.R.; Johnson, C.J.; Stewart, K.A. Nonresponse bias in student evaluations of teaching. Mark. Educ. Rev. 2016, 26, 93–104.
  27. Layne, B.H.; Decristoforo, J.R.; McGinty, D. Electronic versus traditional student ratings of instruction. Res. High. Educ. 1999, 40, 221–232.
  28. Avery, R.J.; Bryant, W.K.; Mathios, A.; Kang, H.; Bell, D. Electronic course evaluations: Does an online delivery system influence student evaluations? J. Econ. Educ. 2006, 37, 21–37.
  29. Nowell, C.; Gale, L.R.; Kerkvliet, J. Non-response bias in student evaluations of teaching. Int. Rev. Econ. Educ. 2014, 17, 30–38.
  30. Heckman, J.J. Sample selection bias as a specification error. Econometrica 1979, 47, 153–161.
  31. Bleske-Rechek, A.; Michels, K. RateMyProfessors.com: Testing assumptions about student use and misuse. Pract. Assess. Res. Eval. 2010, 15, 5.
  32. Coladarci, T.; Kornfield, I. RateMyProfessors.com versus formal in-class student evaluations of teaching. Pract. Assess. Res. Eval. 2019, 12, 6.
  33. Timmerman, T. On the validity of RateMyProfessors.com. J. Educ. Bus. 2008, 84, 55–61.
  34. Albrecht, W.S.; Hoopes, J. An empirical assessment of commercial web-based professor evaluation services. J. Account. Educ. 2009, 27, 125–132.
  35. Brown, M.J.; Baillie, M.; Fraser, S. Rating Ratemyprofessors.com: A Comparison of Online and Official Student Evaluations of Teaching. Coll. Teach. 2009, 57, 89–92.
  36. Sonntag, M.E.; Bassett, J.F.; Synder, T. An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com. Assess. Eval. High. Educ. 2009, 34, 499–504.
  37. Otto, J.; Sanford, D.A., Jr.; Ross, D.N. Does ratemyprofessor.com really rate my professor? Assess. Eval. High. Educ. 2008, 33, 355–368.
  38. Hartman, K.B.; Hunt, J.B. What Ratemyprofessors.com reveals about how and why students evaluate their professors: A glimpse into the student mind-set. Mark. Educ. Rev. 2013, 23, 151–162.
  39. Chou, S.Y.; Luo, J.; Ramser, C. High-quality vs low-quality teaching: A text-mining study to understand student sentiments in public online teaching reviews. J. Int. Educ. Bus. 2021, 14, 93–108.
  40. Constand, R.L.; Pace, R.D. Student evaluations of finance faculty: Perceived difficulty means lower faculty evaluations. J. Financ. Educ. 2014, 40, 14–28, 30–44.
  41. Boehmer, D.M.; Wood, W.C. Student vs. faculty perspectives on quality instruction: Gender bias, ‘hotness,’ and ‘easiness’ in evaluating teaching. J. Educ. Bus. 2017, 92, 173–178.
  42. Constand, R.L.; Pace, R.D.; Clarke, N. Accounting faculty teaching ratings: Are they lower because accounting classes are more difficult? J. Account. Financ. 2016, 16, 70–86.
  43. Constand, R.L.; Clarke, N. How class and professor characteristics are related to finance faculty teaching ratings. J. Financ. Educ. 2017, 43, 101–119.
  44. Constand, R.L.; Clarke, N.; Morgan, M. An analysis of the relationships between management faculty teaching ratings and characteristics of the classes they teach. Int. J. Manag. Educ. 2018, 16, 166–179.
  45. Carter, R.E. Faculty scholarship has a profound positive association with student evaluations of teaching—Except when it doesn’t. J. Mark. Educ. 2016, 38, 18–36.
  46. Smith, K. Getting econometrics students to evaluate student evaluations. In Shaping the Learning Curve: Essays on Economic Education; Mixon, F.G., Jr., Ed.; iUniverse: New York, NY, USA, 2005; pp. 15–25.
  47. Mixon, F.G., Jr.; Smith, K.W. Instructor attractiveness and academic rigour: Examination of student evaluation data. Australas. J. Econ. Educ. 2013, 10, 1–13.
  48. Green, T.G.; Mixon, F.G., Jr.; Treviño, L.G. Have you seen the new econ prof? Beauty, teaching, and occupational choice in higher education. In Shaping the Learning Curve: Essays on Economic Education; Mixon, F.G., Jr., Ed.; iUniverse: New York, NY, USA, 2005; pp. 57–67.
  49. Green, T.G.; Mixon, F.G., Jr.; Treviño, L.J. Instructor attractiveness and institutional choice in economics: A decomposition approach. In New Developments in Economic Education; Mixon, F.G., Jr., Cebula, R.J., Eds.; Edward Elgar Publishing: Northampton, MA, USA, 2014; pp. 209–217.
More
Information
Subjects: Business
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register : , , ,
View Times: 62
Revisions: 2 times (View History)
Update Date: 25 Jan 2024
1000/1000
Video Production Service