Student Evaluations
The New York Times Magazine is running “The College Issue”. One of the articles is on that most worthless of documents, the student evaluation. Every semester this gnaws at me: a student can put zero effort into a class, can lose points left and right on silly stupid mistakes[1], and even not show up half the time, and then can turn around and say I’m a bad teacher for it. I can stand in front of the room, begging and pleading a stone wall for questions, subjects to review, places the students are shaky and want a little shoring up and get nothing, only to have reviews complain that I go too fast.
Overall, students don’t know what the hell they’re talking about, and to put any weight on their opinion for my future career is more of an insult than anything else.
[1] A huge portion of my college algebra classes lost points on the exam by giving a picture-perfect calculation of the slope of a line between two given points, when the problem asked for the distance. Really, what is there that I can do if the students refuse to do something as basic as read what the problem is asking for?
16 Comments »
Leave a comment
Recent Posts- Recent Comments
- Comment on Symmetric, Antisymmetric, and Hermitian Forms by John Armstrong
- Comment on Symmetric, Antisymmetric, and Hermitian Forms by Akhil Mathew
- Comment on Self-Adjoint Transformations by Symmetric, Antisymmetric, and Hermitian Forms « The Unapologetic Mathematician
- Comment on Matrices and Bilinear Forms on Inner Product Spaces in Dirac Notation by Symmetric, Antisymmetric, and Hermitian Forms « The Unapologetic Mathematician
- Comment on Matrices and Bilinear Forms on Inner Product Spaces in Dirac Notation by John Armstrong
- Comment on Matrices and Bilinear Forms on Inner Product Spaces in Dirac Notation by Adam Glesser
- Comment on Complex Inner Products by John Armstrong
- Comment on Complex Inner Products by Maya Incaand
- Comment on Dirac notation I by Matrices and Bilinear Forms on Inner Product Spaces in Dirac Notation « The Unapologetic Mathematician
- Comment on Matrices and Forms II by Matrices and Bilinear Forms on Inner Product Spaces in Dirac Notation « The Unapologetic Mathematician
As much as I look forward to teaching, I admit the future student evaluations have me more than a little scared. More so because for many of my college classes, I put little to no thought into them and hardly wrote anything, even for teachers I thought were excellent.
The use of student survey’s as an effective measurement of teaching remind me of the tradition of The Lottery.
It is a tradition in which we are engaged, but no one seems to know why we do it.
A deep statistical analysis that should be possible after decades of engaging in this practice has never been done. Why don’t professional sociologists and psychologists study this practice carefully? From the article, it seems that the value of these data is marginal.
Nevertheless the practice is expensive. At our department we hand out close to 2500 evaluations per each of fall and spring semester. Each of over 100 classes has to have its own evaluation packet prepared. The teacher sheet has to be filled in properly, the answer sheets have to be checked by hand to ensure that each student has entered the correct codes for the class. And the stack of 2500 pages is delivered to the computer service center. The Computer Center then compiles the data for our department and delivers a paper report. There are no comparisons across departments because different departments deliver different forms. The data are not stored electronically by the Computer Center because the machine scanning the data is an antique.
Written student comments are rare and often ungrammatical.
I read the summary data, I have to scan photocopies of the original forms to read the comments and also to check individual responses for prejudiced (all As or all E answers) responses. There are various data reports that the statisticians in my department do not understand.
There is anecdotal evidence that some teachers (particularly those at risk) sneak in to affect their evaluations in their favor.
In the early stages of my career, a peer once suggested that one first looks at one’s students’ evaluations and then assigns grades.
That an administrator would take the responses on this site seriously strikes me as cynicism in its most vile form. Any administrator who used those data should be required to take a course in elementary statistics and pass it with a $B$ or better!
Even those with tenure at an institution cannot get the student survey system to stop. We can’t even get our administrations to agree that the system has deeply rooted problems that need to be addressed.
Bad link. “This site” should be http://ratemyprofessors.com/
This is ridiculous. I teach at UC Irvine and I have to face student evaluations every quarter. They’re nothing to be concerned about, especially since A) the evaluating faculty are certainly aware of how some students react to a difficult class or test, and so evaluation results are aggregated and edge cases discarded, and B) most students are, actually, very honest. In your post you say “a student can put zero effort …” If the problem is truly in the singular, “a student” here and “a student” there, then your career is not being adversely affected. If your problem is more widespread than that, then perhaps you should, in fact, take a look at your evaluations seriously with the idea that your job as an instructor is to engage with students as well as convey information, and if you upped your game in the first area, you might improve in the second. If your students aren’t reading your test, and the test was truly clear, then grade appropriately and be happy, because if that’s the case then other faculty members are having the same problem.
I would say, though, that your assertions, “students don’t know what the hell they’re talking about” and “their opinion … is more of an insult than anything else”, doesn’t suggest that yours is the class every student is oh-so-excited to take.
Woof, I’d venture to say that no student is excited to take college algebra. Most in calculus classes are filling major requirements.
As for the students who are excited to take calculus, every single semester so far I have had at least one student declare a major in mathematics after taking my course. But of course that information isn’t on the evaluation forms.
I think the idea of eliminating student evaluations is rather absurd.
Let me tell you a story from when I was in college studying CS. The best class I took was on databases. On the first day of the class, the professor said that if you didn’t do very well in Data Structures and Algorithms you should drop now, because you are going to fail. He wasn’t kidding. About half the students in the class were CS grad students. It was the most hard core and most educational class I had as an undergraduate.
Sounds great, right? Well, not exactly. The class was cross-listed in Industrial Operations Engineering. Neither the Data Structures class, nor its prereq, nor its prereq were listed as prerequisites. The only prerequisite was a basic intro C++ class targeted at non-CS students.
Also, there was another professor who taught the same course on alternating semesters. His version was completely different. It matched the description in the course catalog.
So a student expecting an easy class would be completely reasonable in writing a scathing review of the professor, probably blathering about how his friend took it the previous semester and it was completely different and so much easier and it isn’t fair.
I agree about the horribleness of student evaluations as a method for central administration evaluating teaching performance, and also that the numerical rating systems used on my evaluation forms is next to worthless. However, I have occasionally found useful feedback on forms that allow students to write something. Most often they don’t, but once in a while I’ll see something that makes some sense and modify my teaching a little bit accordingly. And I think that some of my improvement as a teacher over time can be attributed to this feedback. So I don’t think that teaching evaluation forms are completely useless. However, I do think it’s important for administrations to be very careful about how they interpret form data.
I’ll also note, John, that the system they were moving towards when we were at Yale was the best one I’ve seen anywhere. Getting the students to write their evaluations on-line before getting their grades resulted in a lot more valuable information than making them sit and fill out forms in class. Students in this era are just more comfortable typing on a computer at home than filling out scan sheets in pencil in a classroom on the last day of class when they’d rather elsewhere. So, needless to say, TCU’s attempts at on-line evaluation forms bombed completely, probably mostly because they didn’t implement the Yale strategy of forcing the students to click through the evaluation option on the way to finding out their grades.
I, for one, haven’t advocated eliminating them. On the other hand,
I would like to see (a) solid evidence that there effect on teaching is worth the cost and (b) that they were administered fairly and that the DATA structures be stored and analyzed properly. Without evidence that student surveys are worthwhile, we have to be responsible and ask why do we do this.
Yes, I’m not advocating a feedback system at all (to respond to eengbrec). The problem is not the feedback, but the fact that students don’t universally know good teaching when they see it, and so using their reviews to measure teaching performance is a joke.
Yes, I expect a lot of effort from my classes. No, I don’t get very personal with my students. No, I’m not a clown at the board. But my job isn’t to be easy, or to be their friend, or to entertain them. My job is to communicate mathematics — and in this case the mathematics that they’ve already got a track record of forgetting since high school. And the goal is to have the mathematical tools at hand when they need to use them in later classes.
But as the article points out, evaluations aren’t correlated very strongly with future performance. They’re correlated with high grades (or the expectation thereof), and with good looks, and with charm.
I also agree with Greg’s point about Yale’s system, but with the catch that it generates commentary completely devoid of context. As a case in point:
Every semester I tell the students that my approach differs somewhat from the book’s approach, though they stay in rough parallel. I emphasize that this is a Good Thing because some of them will learn better from me, some better from the book, and between both sources everyone stands a better chance of learning in a way that fits them best. And yet at the end of the semester students saying that they didn’t follow my approach (n.b.: they didn’t ask questions, either in or out of class) and that they had to read the book (horrors!) to figure out how to do the problems. What part of this is on the evaluation? “Prof. Armstrong doesn’t explain well. You have to read the book to understand what’s going on.”
For an administrator to rate my teaching by culling out the handful of those out-of-context inanities each semester is a slap in the face.
“Overall, students don’t know what the hell they’re talking about,”
Friendly amendment #1: Any indivual student does not know what he or she is talking about on at least one point. Overall (i.e. in any sufficiently large set of students) there is at least one point where at least one student knows something. Lemma: see Wikipedia (overall).
“… and to put any weight on their opinion for my future career is more of an insult than anything else.”
Friendly amendment #2: The school pays us, hence they are entitled to knows what they’re getting from us. They do not seem to use an optimum data collection instrument, to be sure. But it is a basic concept of Mathematical Control Theory that if you cannot measure something, you cannot control it.
Also, I earned money as a software consultant circa 1974 in grad school by updating their Fortran IV program that took raw student evaluation data and did statistical stuff with it, and printed “3-D” graphics of results sliced and diced in different ways.
I (with approximately 3,000 publications, presentations, and broadcasts, if you count every “least publisbale unit” such as an entry in the Online Encyclopedia of Integer Sequences, or the Prime Curios web site, or MathWorld, as well as books and papers and conference presentations) had superb student evaluations from how well I taught Algebra 1 and Algebra 2 for 5 semesters at Woodbury University. Even the secretaries were impressed by how much higher my ratings were than any other Math adjunct professor.
But the Chairman (junior authore on only 1 refereed Physics publication) said out loud that I’d never taught Algebra 1 there, and the Dean (junior author on only 1 refereed publication) wanted to get back at my wife who was likewise dangerously well-rated as a Physics Professor (with over an order of magnitude more refereed publications thanChairman or Dean). Hence my contract was not renewed, and some of my algebra classes were handed over to my wife’s useless replacement lab assistant (after the good, and published, lab assistant quit because of harassment from the Dean).
“A huge portion of my college algebra classes lost points on the exam by giving a picture-perfect calculation of the slope of a line between two given points, when the problem asked for the distance. Really, what is there that I can do if the students refuse to do something as basic as read what the problem is asking for?”
I got the strong sense on reading my students’ answers in homeworks #1, and #2, of Chemistry, Biology, Anatomy & Physiology (9th, 10th, 11th grade) that they were often unclear on the differences between element, compound, and mixture. So I painstakingly defined each, carefully, with examples.
Homework #3 was 4 printed pages, asking them to read these definitions carefully and then (1) draw me a diagram or picture of a Lithium-7 atom; and (2) given a list of 18 substances, identify which were elements, which were compounds, and which were mixtures.
Many drew me a diagram of a Lithium-6 atom (having not read nor not understood the detailed descriptions of different lithium isotopes). I don’t mind that many called baking soda a mixture.
I’m annoyed that some still though that air was an element (perhaps remembering only Earth, Air, Water, and Fire that I cited on day 1) despite how many times I’d said that air was a mixture of nitrogen, oxygen, and various trace gasses, every day for a week, with different trace gasses discussed, and volunteers drawing diagrams of nitrogen and oxygen atoms on the whiteboard.
But what really creeps me out is that about 1/3 of my students wrote that beef stew and pizza were compounds. Most of them agreed that taco salad was a mixture. My wife suggests that gravy and cheese, repectively, bind beef stew and pizza together more tightly than the obviously loose taco salad. But I’ll have to spend most of Monday’s classes going over the list of 18 one at a time, after having students take turns reading out loud the definitions, a paragraph per student.
If I can’t measure what’s in their heads, I cannot control it. If the administration can’t measure what’s in their heads about ME, they cannot control that, either.
Any individual student. “Least Publishable Unit.” Hey, teach, I don’t get graded on, like, you know, spelling and grammar, right? CUlater.
I think you mean “CUl8r”
Dear sir,
an off topic question.
can you give an example of a finite lattice, where
the greatest and least element do not exist?
in other words, is every finite lattice a bounded
lattice?
This is a hard topic. In some sense the rant is well taken, for it often is true that student comments are puerile and ill-informed. But I think (based on years of experience in teaching and reading evaluations) that Woof is right: when asked to rate the overall clarity, preparedness, helpfulness and so on of a professor, most students are able to produce an answer in good faith.
John, you’re absolutely right that it’s not your job to clown around, or get personal, and so forth. [All the same, I'd be a bit surprised if you didn't let your witty self peek out now and then, or delight the class with a throwaway joke once in a while.] But my own experience in teaching college algebra or other remedial courses is that one of the teacher’s first duties is to help set the students at ease a little, in what is probably for them a slightly terrifying experience, where there is underlying fear of failure and humiliation in a subject they’ve long had trouble with. One doesn’t have to clown around, but one should be as kind as humanly possible (while still being fair and objective in grading) — there are many effective ways of conveying the desire to be a helping professional, and to show kindness and tact without being mushy about it.
I realize how preachy this must sound, but my main worry is that sentiments such as “students don’t know what the hell they’re talking about” — which are addressed to us here as if we’re sitting around in the department lounge — are somehow being conveyed in subtle but nevertheless detectable ways to the students (some of whom may be reading this blog).
To be honest, I’ve often felt the same way, but it seems to me that in such cases my own attitude — the only thing truly under my control — might itself need readjustment. My sense is that advice is the last thing you’re asking for here, but one book which seems generally helpful in addressing these general professor complaints, and what to do about them, is Steven Krantz’s How to Teach Mathematics. I also think that many of us could take a page out of Terence Tao’s “book” — his own approach in communicating to others (who are almost without exception less able than himself) seems to be deeply informed by an exquisite respect and tact, and I and everyone else would do well to try to emulate that.
Todd, you make a good point. Maybe I should be more specific: “students don’t know good teaching from bad (except egregious cases) while they’re in their freshman year of college.” They certainly do know something, and my whole enough-rope method is based on the supposition that students know a lot more than they’re often given credit for.
But I’ll even turn this around on myself: I had no idea what a good teacher was my first year in college. It was years before I really started to internalize what Jim Yorke did and what Dan Rudolph did and distinguish them from what certain other professors did. By the time I graduated I could have written a cogent evaluation of any of the professors I had that first semester, but not at the end of the term.
What a beautiful comment by Todd Trimble…..