What education reformers did with student surveys

By Felix Salmon
September 23, 2012
Amanda Ripley has a thoroughly (if inadvertently) depressing story in the new Atlantic about the rise in the way in which teachers are evaluated by means of multiple-choice tests given out to students.

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

Amanda Ripley has a thoroughly (if inadvertently) depressing story in the new Atlantic about the rise in the way in which teachers are evaluated by means of multiple-choice tests given out to students. She says the idea is “revolutionary”:

Research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most—and least—effective teachers.

This is probably true, although just how revolutionary it is remains to be seen. These tests really are a great tool for judging which teachers are the most effective and which are the least, across various axes. But from reading Ripley’s rah-rah article, it seems very much that they’re used for precisely the wrong reasons, and barely used at all for the right ones.

It’s impossible to argue with the assertion that the quality of a child’s education rises with the quality of the teacher — just as it’s impossible to argue with the assertion that kids can be very good judges of how good their teachers are. But the important thing here is how these tests are used: are they used by teachers to improve the instruction they’re giving kids, or are they used by managers to come up with yet another key performance indicator to impose on the teachers under their charge?

One way to answer that question is to look at the questions which Ripley isolates as being particularly informative.

Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:

1. Students in this class treat the teacher with respect.
2. My classmates behave the way my teacher wants them to.
3. Our class stays busy and doesn’t waste time.
4. In this class, we learn a lot almost every day.
5. In this class, we learn to correct our mistakes.

When Ferguson and Kane shared these five statements at conferences, teachers were surprised. They had typically thought it most important to care about kids, but what mattered more, according to the study, was whether teachers had control over the classroom and made it a challenging place to be.

You see what Ripley did there? Measuring how well children are being educated is an astonishingly difficult job. Increasingly, these days, and especially since No Child Left Behind, we’re using test scores as a proxy for quality of education. Everybody agrees that it’s a poor proxy, although there’s disagreement about exactly how poor.

In any case, along comes the Gates Foundation with a 36-question survey, severely chopped from a much longer one developed by Ronald Ferguson. Since there are 36 questions, the survey essentially measures teachers along 36 different axes, all of which are aligned with each other to differing degrees. In and of itself, that’s more useful than just measuring test scores, which are much less teacher-specific and which only provide one axis of educational quality.

But then what do the reformers do? They regress the survey answers against test scores, look at which survey questions align most closely with that test-score axis, and declare that those axes — the ones which test scores, by definition, are already measuring — must be the “most important”. Did you think that caring about kids was of paramount importance? Silly you! It turns out that caring about kids isn’t as correlated with test-score results as, say, whether the class learns to correct its mistakes. And therefore, we shouldn’t be worrying as much about whether teachers care about their kids; we should be worrying more about other things, instead. That’s what the test-score regressions tell us, so it must be true!

This reminds me a bit of the way in which investment banks are prone to taking tens of thousands of risk positions, reducing them all to a single value-at-risk number, and then using their VaR way too much, despite the fact that it’s of only limited utility. Except in this case it’s not even all of the survey answers which get used: most of them are simply discarded.

If Ripley and the Gates Foundation wanted to find a new and powerful and rich way of measuring how effective teachers are, they would use all the information at their disposal, and then they would underweight the answers to the questions most correlated with test scores. After all, test scores are already given far too much weight, for lack of other measures to look at. Instead, they do the exact opposite, and use the surveys to double down on the inherently flawed idea that test scores are a good proxy for educational prowess.

Now if that was all they did, it would feel a bit like a wasted opportunity. But it gets so much worse. Check out this theme running through Ripley’s piece:

Should teachers be paid, trained, or dismissed based in part on what children say about them? … This past school year, Memphis became the first school system in the country to tie survey results to teachers’ annual reviews; surveys counted for 5 percent of a teacher’s evaluation. And that proportion may go up in the future… The New Teacher Project, a national nonprofit based in Brooklyn that recruits and trains new teachers, last school year used student surveys to evaluate 460 of its 1,006 teachers… In Georgia, principals will consider student survey responses when they evaluate teachers this school year. In Chicago, starting in the fall of 2013, student survey results will count for 10 percent of a teacher’s evaluation… On average over the past decade, only a third of teachers even clicked on the link sent to their e-mail inboxes to see the results. Presumably, more would click if the results affected their pay… This school year, Washington, D.C., will make the survey available to all principals and teachers who want to use it. Chancellor Kaya Henderson says that next year, the survey may count toward teacher pay and firing decisions.

No! Stop! Do none of these people get it? What everybody wants, here, is better teachers. These surveys could be instrumental in helping to improve teaching. Teachers would be able to see where they score well and where they score badly, and ask themselves how to improve their scores in areas where they are weak. Principals could see which teachers were good on which axes, and set classes up so that students ended up with a balanced range of teachers. And generally, everybody could treat this data as an interesting and very rich way of improving educational outcomes.

Instead, reformers are rushing to use this data as a quantitative performance-review tool, something which can get you a raise or which can even get you fired. And by so doing, they’re turning it from something potentially extremely useful, into a bone of contention between teachers and managers, and a metric to be gamed and maximized. Check out Ripley’s language here:

The variation within the school was staggering—as it is in many places. In the categories of Control and Challenge—the areas that matter most to student learning—Nubia and her classmates gave different teachers wildly different reviews. For Control, which reflects how busy and well-behaved students are in a given classroom, teachers’ scores ranged from 16 to 90 percent favorable; for Challenge, the range stretched from 18 to 88 percent. Some teachers were clearly respected for their ability to explain complex material or keep students on task, while others seemed to be boring their students to death.

The first thing Ripley does here is throw out nearly all of the rich survey data, in her attempt to boil everything down to one or two simple numbers per teacher. She concentrates on things she calls Control and Challenge, and declares in an omniscient tone that these areas “matter most to student learning”. She then gives each of those metrics a neatly rankable percentage, so that any school can point easily to which teachers are the Best in Control (“clearly respected for their ability to keep students on task”), or Worst in Challenge (“boring their students to death”).

You can see how this might not go down very well with teachers, who are meant to be working as a group to broadly educate a cohort of children, but instead are being isolated and compared against each other, with potentially career-ending consequences for those who score low. The minute that the scores start being used in that way, the teachers understand what’s really going on here, and they resent it. What’s more, they do so for good reason: the more that an enormous quantity of complex data is reduced to a couple of performance-review datapoints, the less useful that data becomes.

School reformers in general, it seems to me, tend to be obsessed with the idea of Good Teachers and Bad Teachers, as though the quality of the education a kid gets in any given classroom is somehow both predictable and innate to the teacher. And yes, at the extremes, there are a few great inspirational teachers who we all remember decades later, and a few dreadful ones who had no idea what they were talking about and who had no control of their classes. But frankly, you don’t need student surveys to identify those outliers. And the fact is that schools are much more than just the sum of their parts: that’s one of the reasons that reformers love to talk about excellent principals who can turn schools around.

The trick to improving education is to make schools better, not to find ever-more-cunning ways to reward and punish teachers. Especially when there’s no evidence whatsoever that such reward-and-punishment schemes actually make those teachers better educators, rather than simply resentful. There’s a reason why certain schools develop a reputation for excellence which can last for centuries: there’s something institutional going on, a virtuous circle which lifts up everybody. Making education granular — isolating not only certain teachers but even certain aspects of how those teachers teach — is a classic example of missing the forest for the trees, for no good reason.

I don’t doubt that student surveys could, in theory, be very useful in the large task facing administrators and teachers — how to make schools better and improve the quality of the education they provide. They would show where schools were weak and where they were strong; which teachers have managed to crack certain nuts where the rest of the faculty is having difficulty; that kind of thing. In short, they could be tools for diagnosing and improving the quality of a school’s education as a whole.

But the reformers rush straight past all that, and decide that the first best use of such data is to use it in performance reviews, and use it to give raises to good teachers and pink slips to bad ones. And, of course, the minute you start doing that, it becomes impossible to use the data for anything else, since the scores then become an end in themselves, rather than a means to an end.

The toothpaste is out of the tube, now: I frankly can’t see how anybody is going to be able to use these surveys effectively at this point, now that they’re associated in teachers’ minds with performance evaluations and disciplinary procedures. This is the bit that reformers seem to have a great deal of difficulty understanding: it’s incredibly difficult to improve the quality of teachers just by promising to pay them more if certain numbers are high, or by threatening to fire them if certain numbers are low. Student surveys, as originally conceived by Ronald Ferguson, could have been a great tool for improving the quality of eduction. But at this point, I fear, it’s too late.


We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

I was an adjunct prof for awhile (too long) at a university and we were judged exclusively on our student survey results. There are downsides, in my experience. In my last class one student was gushing about another adjunct prof she had a class with. When I asked her what she liked so much she said, “She brings candy to every class!” (This was college!).

I think if you overweight what students ‘like’ you turn teaching into a popularity contest. I taught at one very expensive private university that had no tenure in my department. The impoverished adjuncts scrambled all over each other to win that popularity contest with students. By ‘winning’ they got the equivalent of a miserably paid one on one summer tutoring gig. The result was they flattered their students in a way that was far from the educational mission.

Good teaching involves exploration which can have uncomfortable elements – with material that is too challenging or difficult or controversial to go down like metamucil. It may not always ‘survey’ well.

Posted by nyet | Report as abusive

Felix – Thanks for raising red flags about the use of student surveys to evaluate teachers.
The problem with Amanda Ripley’s approach is its reliance on test data as the only useful measure of teacher performance. As she writes in her article:
“Despite all the testing in American schools, most teachers still do not teach the subjects or grade levels covered by mandatory standardized tests. So no test-score data exists upon which they can be judged. As a result, they still get evaluated by their principals, who visit their classrooms every so often and judge their work just as principals have always done—­without much accuracy, detail, or candor. …”
Perhaps it would be simpler and more effective to limit all primary and secondary class work to material covered by mandatory standardized tests. That would make testing more useful. That approach might well degrade the quality of education, but we need to keep our priorities straight: It’s better for us to know precisely how much our children have learned than to teach them more valuable lessons that tests can’t measure.

Posted by CompactCar | Report as abusive

Sigh. Haven’t any of these people ever worked in a school before? Students are not randomly assigned to classrooms (or even to schools). Using student responses/perceptions like this is just bizarre. As a male elementary teacher, I was often asked to take “troublesome” boys in my class. I also volunteered to take mainstreamed special needs kids. Don’t regret working with either, but I guarantee it didn’t help my “control” rating.

Posted by BigDonut | Report as abusive

Felix, I can’t object too strenuously to what you are saying. After all, the movement to demonize teachers was one of the reasons I left public education eight years ago. I never asked for more money — just support and respect in a difficult job.

But Ripley has it largely correct. The best teachers are those who Control (my weakness) and Challenge. “Caring” doesn’t get you very far in the classroom.

Moreover, these teacher surveys have one strength you acknowledge and then immediately dismiss. The answers you get from the students will be specific to the teacher in question. If you give a student a test, the results will be a combination of the cumulative proficiency of ALL past teachers, parents, and mentors, as well as a strong element of individual aptitude. Even if you test annually in the subject, it can be nearly impossible to tease out the impact of a specific teacher from the results. The proposed survey doesn’t have that problem, at least not nearly to the same degree.

I am still vehemently opposed to attempts to rate teachers based on student performance. Some teaching assignments are a cake walk. Others are a challenge for even the best teachers. A good department will work together, with the best teachers taking the toughest assignments (and those that best fit their skills). Pay them by student performance, and the best teachers will fight to have the easiest students to work with — and thus the biggest raises.

Want better teachers? Treat them as professionals. Good schools hold teachers accountable for their performance without resorting to cheap tricks like standardized testing.

Teach administrators how to conduct effective classroom observations, effective performance reviews. Give them the time, resources, and authority to do it properly — and hold THEM accountable for the performance of the school. It isn’t a secret (at least not within a school) who the best teachers are.

Posted by TFF | Report as abusive

1) I’m not sure if students can really tell the diff between a good and a very good teacher – based on my own experience as a student, and talking to my classmates
2) The Gates thing is an example of how much undemocratic power is wielded by the rich; here is an example from MA, abstracted from the boston globe.
There is a “grass roots” organization in MA that is pushing for student evaluations, the GRO is funded by gates, koch, walmart, etc.
They had so much money that the teachers union decided to give in quite a bit on testing.
Now to me, the point is not, is testing good or bad, but that a small number of billioniars are, essentially, dictating school policy in our state – I think, and maybe a poli sci person can correct me, that this verges on fascism
aside from whic, the idea that Bill Gates, whoose software sucks, is telling anyone how to do anything is astonishing
software sucks: in Microsoft word, you can have an automatic table of contents, based on use of paragraph syles – a very convienent feature.
When you need to update the TOC, you are supposed to press ALT-F9.
except, in many cases, pressing ALT-F9 doesn’t work: you have to do it twice.
almost the definition of a a bug.
and the thing is, it has been there for many years, and is still there in office 2007 (I’m not familiar with office 2010 and later)
and I could list many other cases where microsoft office just fails; a particularly bad one in excel 2007 concerns XY scatter charts in excel; sometimes, excel thinks that the x values in your spreadsheet are not numbers, and instead of a scatter chart, it draws a line chart (which is quite different) and excel doesn’t tell you !!
not only that, the problem is really hard to fix, and the fix is not obvious (you have to correct errors, not format the cells).

Posted by joeenuf | Report as abusive

Readers rate Amanda Ripley. She has written great stuff in her other area, disaster survival.

When she writes about education, she ranges from incredibly stupid to malevolent.

Does this rating go on her permanent record?

Posted by CarolineSF | Report as abusive

“the point is not, is testing good or bad, but that a small number of billioniars are, essentially, dictating school policy in our state – I think, and maybe a poli sci person can correct me, that this verges on fascism”

Fascism is one of those words that people throw around and don’t seem to know what it means.

One good definition I have read is that fascism is worship of the state, and the subordination of the welfare of the individual to the welfare of the state.

I’m not sure that term captures what you’re trying to convey here. I think the term you’re looking for is plutocracy (running of the state by the wealthy) … although, in this case, since these charitable efforts have no legal authority, it’s hard to see how that’s the case here.

Posted by furytrader | Report as abusive

The focus on determining which teachers are good and which teachers are bad begs one very obvious question…what do you do with this information once you have it.

The answer bandied about by politicians and anti-union acitivists is usually to fire all the bad teachers. But every time you fire a teacher, you then have to hire someone to replace them (unless you are willing to increase class sizes). And since teaching is not exactly considered to be an attractive profession these days, finding a new teacher to replace the one you just fired is often very difficult.

Posted by mfw13 | Report as abusive

@mfw13, it is worse than that…

(1) Nearly half of new teachers leave the profession within five years. On average, you need to hire two new teachers to fill one vacancy. Given the STEEP learning curve over the first few years, this means you are dealing with as many as five or six years of sub-par teaching every time you replace a veteran. (It isn’t quite as bad as this — there is much a school can do to help ease new teachers into the profession with lighter schedules and mentoring support. But you simply can’t hand a new teacher the same schedule you would ask of a veteran.)

(2) Teachers are underpaid relative to comparably skilled private-sector employment for three reasons. First, the job is attractive for those (like myself) who need “mommy hours” to care for their own family. Second, it is stable employment, relatively sheltered from the vagaries of business cycles. Third, the promise of a guaranteed pension is valuable — especially to those who would otherwise be relying on the Wall Street sharks to fund their retirement. If you start arbitrarily firing teachers, you weaken the second factor and turn the third into a net negative! A teacher who leaves the public schools here can recoup only their contributions plus 2% interest — they leave ALL investment gains behind in the pension system.

If you want to make teaching more comparable to private-sector employment, then pay six-figure salaries and get rid of the pensions. You’ll see a lot more competition for those jobs than you have today.

Posted by TFF17 | Report as abusive

First of all: students lie all the time. So surveys cannot be trusted. Second, Felix is right that managerialism does not belong in education: quantitative analysis cannot be used to make qualitative judgments. The only answer is old fashioned: hire good teachers and fund them properly.

Posted by JustJustin | Report as abusive

@JustJustin: so, what is a “good” teacher? How do you identify them ahead of time? How do you retain them, and get rid of the “bad” teachers?

Posted by SteveHamlin | Report as abusive

Well-said Felix. Here is a real live example of what you are talking about. My kids go to school in Arlington, Virginia, where, so far as I am concerned, their teachers are outstanding. A very significant percentage of teachers in Arlington started their teaching career in Washington D.C. where schools are not noted for their quality. You don’t go from being a “bad” teacher in D.C. to a “good” teacher in Arlington — you go from a district that is beset by any number of problems to one that has by and large avoided those problems, for lots of reasons, income differences being only one of them, and therefore, are not full of rabid “reformers” determined to stick it to teachers. Anyway, the goal is to make better schools, and rather than spending so much energy thinking of a zillion creative ways to identify bad teachers, they might want to find a zillion creative ways to KEEP THE GOOD ONES — the ones who get frustrated and go to Arlington or Fairfax or Loudoun or Montgomery County.

Posted by rb6 | Report as abusive

They use student surveys all the time at universities and it is an accepted and useful part of the teacher evaluation process. I don’t know why they should be verbotten in HS. Are they perfect, no. What evaluation method is?

As for your hatred of the reliance on standardized tests…well standardized tests are one of the only things we can relying on that have measurable correlations with outcomes we are shooting more. That is why they are used.

If they had some better system they would be pushing that, but they do not.

Posted by QCIC | Report as abusive

@SteveHamlin, what is a “good” production manager? How do you identify them ahead of time? How do you retain them, and get rid of the “bad” production managers?

Have you tried asking a good school district how they manage these tasks? How they find good staff? How they conduct performance evaluations? I assure you, the good school districts DO find ways of helping ineffective teachers out the door (after first doing their best to help them improve).

Teachers don’t have any more job security than anybody else in the first three years, and even after that it is possible to fire them for continuing documented performance problems. Usually doesn’t get to that point, of course.

Posted by TFF | Report as abusive

@QCIC, they **DO** have a better system, and the better districts are already using it.

(1) Evaluations must be evidence-based in best practices that have been demonstrated to lead to student learning.

(2) The evidence should be collected through a series of classroom and professional observations, both announced and unannounced. During my first three years, I was observed roughly a dozen times a year by three different administrators. A few of those were announced, but the rest were either unannounced (supervisor enters along with the students at the start of the period) or 5-minute “walk throughs”.

(3) The administrators need to be given time and resources to do the job properly. They need to be trained in both classroom observation techniques and in “best practices” to emphasize. They need to give teachers regular and detailed feedback from these observations.

Standardized testing measures the students, not the teachers. If every class entered with similar skills, then standardized testing would tell you much about teacher performance. But in the real world, we INTENTIONALLY segregate strong students from the weak students. If you believe they should be expected to hit the same endpoint by the end of the year, you are kidding yourself.

Posted by TFF | Report as abusive

TFF No one is tryign to eliminate those other methods. People understand that different classes have different student skills sets. You are arguing against a straw man.

This is why the Chicago teachers look so bad, they are arguing against positions their opponents don’t actually hold.

Posted by QCIC | Report as abusive

QCIC, I wasn’t arguing against a straw man, I was arguing against what you wrote. Or was I misinterpreting you when you said, “If they had some better system they would be pushing that, but they do not.”? It seemed to me that you were denying the existence of strong qualitative teacher evaluation systems.

I haven’t seen a detailed proposal to evaluate teachers through student test scores (I’m no longer in the public schools), but I don’t think the concern I raise is unreasonable. Nothing I have seen described in the press suggests how to resolve it. In MA we don’t test students in every subject and we don’t test them every year. How can individual teachers possibly be evaluated on that basis?

Posted by TFF17 | Report as abusive

@TFF >> “. . . strong qualitative teacher evaluation systems.

This is what we had in the military. It turns out that everyone is so far above average it isn’t even funny. Simply doesn’t work.

Posted by Curmudgeon | Report as abusive

@Curmudgeon, I’ve seen it work (and went through at least part of the training that it depends on). I’ve seen young teachers counseled out of the profession after one or two years of struggling. I’ve seen veteran teachers retire rather than be put on notice. Wish I still had the entire evaluation document, so we could compare notes, but for that school at least it worked very well.

I firmly believe that strong administration (with the authority to hire and fire) is the key to strong schools. Trying to manage a school from the outside is nearly impossible. You might have a pretty good idea whether it is successful or not, but from the outside it is very hard to know how to fix what is broken.

Posted by TFF17 | Report as abusive

@TFF, but it doesn’t as a system. It is gamed, and after a few year, everyone is above average. We used semi-secret keywords, rather than an honest evaluation. You can’t make this work in a teaching environment. Sorry.

Posted by Curmudgeon | Report as abusive

@Curmudgeon, I’ve *SEEN* it work in a teaching environment. It wasn’t new and it was already working well when I got there. It continued to work well for the five years I taught at that school. Probably still working a decade later, for all I know. (The school remains very strong.)

I do wish we could compare notes in more detail so that I might better understand why the system you reference was an abject failure while the system I experienced worked effectively?

Of course my pay never depended on these evaluations, and I wasn’t at all worried about my performance being graded “unsatisfactory”. I welcomed the classroom observations and learned from the feedback. Is that how your system worked?

Posted by TFF | Report as abusive

@TFF, and I’ve seen it fail elsewhere. I don’t think we’re going to come up with a qualitative system that works en masse. And it wasn’t my system, but that of the US military. In this case, if you received something less than a well above average, you would never make it very far. That how a qualitative system can be gamed.

Posted by Curmudgeon | Report as abusive

@Curmudgeon, I would agree. The details of an evaluation system must be tailored to the district and school, and applied with professionalism. (I wonder if the US military forgot this part?) You also need appropriate checks against gaming the system.

But I would apply exactly the same cautions to quantitative systems. A standardized test that is appropriate to judge an inner city school is the wrong target for a wealthy suburban district. A test that is challenging for College-track students is insultingly easy for those in AP-track courses. And while there are many elements of teaching that cross all subjects, I don’t see how you can possibly create truly parallel assessment standards.

School districts are hard at work gaming the mandated state testing, “teaching to the test” instead of teaching the full curriculum. (The math test, for example, ends with basic Algebra and Geometry.) One local district even eliminated History from the (middle-school?) curriculum, so they could instead spend the extra period on the subjects that are tested.

Posted by TFF | Report as abusive

Amanda Ripley may be writing about evaluating teachers, but the whole game in education is about privatization. These newer evaluations are just one aspect of how this will be carried out. Charter schools is the goal. Evaluations are nothing more than a way to start culling the current teachers. And the other side of the equation is now that we are doing things like parent-trigger laws, getting rid of collective bargaining and have been identifying teachers as “union thugs”, we can start to replace them with lower paid teachers. What was the first thing Rhee did as Superintendent of DC Schools? New teacher evaluation forms. Gates may think he’s helping students, but he’s playing right into the hands of Michele Rhee and the whole privatization movement.

Follow the money.

Posted by ComradeAnon | Report as abusive

The study mentioned only 33% of teachers clicked on the email to see the results of the test. That means 67% of teachers aren’t able to use the results of the test to improve on their teaching methods. Tying pay/evaluations to this test would presumably mean teachers would be interested to see the test results and at the margin would at least allow teachers to know potential areas of improvement.

Posted by KevyD | Report as abusive

The only way to have good teachers in a school is to hire good teachers. What is this, New speak? You can’t grade teacher’s performance because students vary so much and tests results are not a indicator of what child has learned. By these standards every teacher is a good one and just showing up with a teaching credential guarantees a job for life!

We have had the same school system for over eighty years but still have not concrete way to reliably judge performance. Many foreign schools are rated superior to ours and their student scores are higher in most or all categories. Why this matters when supposedly you can’t test student performance leaves me to believe American educators have found a feather bed and have snuggled down for a profitable rest. In spite of all this many competent teachers find their way in to classrooms and at least a few students actually learn something. With automation and massive job shortages we don’t need many educated individuals anyway.

Posted by jdmeth | Report as abusive

@jdmeth, you have a much better shot of measuring student performance than you do of assigning credit/blame for that performance to a specific teacher.

If you see public education as a “profitable feather bed”, then you’ve clearly never tried it yourself. Give it a few years and then tell me what you think. It is the toughest job I’ve ever tried.

Besides, you CAN evaluate teacher performance, quite effectively. But your best bet is to actually look at what the teacher is doing. Too many districts don’t bother to invest the effort in that endeavor.

Posted by TFF17 | Report as abusive

“There’s a reason why certain schools develop a reputation for excellence which can last for centuries: there’s something institutional going on, a virtuous circle which lifts up everybody.”

I read today a lengthy article about long-time systemic cheating in a well-known, well-respected NYC High School. That seems to contradict the basis of most of your argument.

Posted by brady41 | Report as abusive

Nice article, Felix! Thank you for covering this topic. I agree with you, for the most part. Over-reliance on metrics that are implemented in a centrally organized or generic way is not likely to improve teaching and quality of education.

Felix, you made an analogy with quantitative risk measures, and how (mis-)usage of such has over-simplified financial transactions risk. In education, it is even more egregious to rely on metrics, computer aggregated data, and a system of student critiques of teaching staff. My point is this: At least the individuals who were in charge of risk management in banking were adults. (They might not have behaved maturely, but they had the cognitive maturity to know how to). High school, or middle or elementary school students are children. As others have said, one needs to spend time as a teacher or educator or hall warden in a junior high school to realize how minors can be. They aren’t ill-willed or malicious, but they ARE still learning, and still in a nascent stage of cognitive and social development.

Let’s put that aside though. Without the passage of time, it is non-obvious whether a teacher is good or not. Very obvious measures such as teacher absentee rate exist, of course. Yet that doesn’t require sophisticated new metrics or methods to gauge accurately.

Posted by EllieK | Report as abusive