Scandalize Their Names

I met my brother the other day

And gave him my right hand

As soon as ever my back was turned

He scandalized my name.

Now, do you call that a brother? (No, no.)

You call that a brother? (No, no.)

You call that a brother? (No, no.)

Scandalize my name.

When I read the Los Angeles Times’ August 14th article, “Who’s teaching L.A.’s kids?” written by Jason Felch, Jason Song, and Doug Smith, I literally felt chills run up my spine. For those who don’t already know, the article represents the debut of the Times‘ value-added analysis of seven years’ worth of Data from over 6,000 LAUSD elementary school teachers. The paper plans to publish a database of individual teachers’ estimated effectiveness and articles on the subject over the next few months.

I’ve discussed my qualms about the current state of value-added analysis in the past, as have several professionals far more knowledgeable on the matter. Though it sounds great in theory, value-added analysis is simply too imperfect a science (at present) to be fairly used while making important decisions about public schools. The most commonly used methods of assessing a teacher’s added-value have very high error rates, and that’s in addition to the inherently problematic nature of the Data upon which those calculations are based. The writers of the Times article even note that the National Academy of Sciences has said that “the approach [is] promising but should not be used in ‘high stakes’ decisions– firing teachers, for instance– without more study.”

Felch, Song, and Smith go on to write that “no one suggests using value-added analysis as the sole measure of a teacher. Many experts recommend that it count for half or less of a teacher’s overall evaluation.” And yet, the Los Angeles Times has taken it upon itself to publish the names and statistics of thousands of Los Angeles teachers because the organization has decided that it “offers the closest thing available to an objective assessment of teachers.” Never mind that the quality of the tests may change from year to year, or that what is considered “proficient” may also change, or the lack of random assignment of students to teachers, or any of the other factors may affect the comparability of student test performance from one year to the next. And never mind the fact that it’s pretty difficult to decide how to control for factors that are known to affect student performance– socioeconomic status, language status, giftedness, etc.– let alone the ones that we can’t and don’t reliably measure (how supportive the school environment is to teachers or students, students’ resilience, emotional stability, confidence…).

Apparently, it is now OK for a newspaper to contract with whatever “expert-for-hire” they like and paint scarlet numbers on public school teachers.

There are many things about the article that I find disturbing. For instance, Felch, Song, and Smith’s language sometimes crosses the boundary between objective, accurate reporting and editorializing. For example, they state that (emphasis mine)

…year after year, one fifth-grade class learns far more than the other down the hall. The difference has almost nothing to do with the size of the class, the students, or their parents.

It’s their teachers.

Contrast that statement with the research-based finding that “more than 90% of the variation in student gain scores is due to the variation in student-level factors that are not under the control of the teacher.” (Indeed, even the report prepared for them by economist Richard Buddin acknowledges the difficulty of separating student-level factors from each other, or from teacher influence.) Also noteworthy is the way in which the writers cast the mostly Latino parents as helpless victims of bad teachers and a cowardly school district. (The “hard-working parents…trust educators to do what’s best,” while “education officials have long known of the often huge disparities among teachers…but rather than analyze and address these disparities, they have opted mostly to ignore them.”)

And let’s not even get into the politically incorrect findings regarding the relative effectiveness of teachers of different races, or the effects of certain student characteristics (giftedness, race) on calculations of teacher effectiveness. Those kinds of findings lead me to question just how “objective” or accurate these calculations are, if you can find weird effects for things like teacher race, or any effects of student characteristics on teachers’ ability to teach. (Isn’t that the point, after all? That we’re supposed to be separating out things that the teacher can’t control from the things that the teacher can control?)

But the most disturbing thing is the precedent it sets for how efforts to improve teacher and school quality may proceed in the future. For starters, adding yet another incentive– the specter of public humiliation– to teach to standardized tests can hardly be considered a good thing. If our goal in education is to produce well-rounded critical thinkers who are ready for 21st Century challenges, does it make sense to assess their progress–and teachers’ effectiveness– with 20th Century pencil-and-paper tests?

Note that when describing the two teachers who were identified as highly effective, the writers chose  to share instances where the teachers and students were engaged in some pretty old-school classroom behaviors. One of the teachers has a student calculating a math problem on the board while his peers watched from their seats. The other is described as “grilling her students on vocabulary.” (Whether the students understand the underlying logic of fractions, or are able to flexibly use new words as they express themselves while speaking or writing, is an open question.)

That’s to be expected in a system that relies upon traditional standardized tests. Traditional tests of just the basics will tend to promote traditional ways of teaching just the basics, however “animated” said teaching might be. If you’re a teacher who knows your name will be published alongside someone else’s measures of your value, you’ll probably feel a pretty strong pull to do whatever it takes to make sure that those test scores go up. Knowing you’ll be directly compared to your colleagues probably won’t foster collaboration, which is a key feature of successful schools, and it may further encourage the narrowing of the curriculum as well as the ways in which students are taught and evaluated.

And what about the emotional and professional toll of having your name listed for all to see, next to faulty numbers that attempt to communicate your worth as a practitioner? Whatever value there might eventually be in using value-added assessment to promote increased teacher effectiveness, how exactly are teachers supposed to make the best use of it when they feel stigmatized and worried about their future prospects in their profession? We’ve already seen the effects of stigma on schools, districts, and state agencies in the number of such organizations that have resorted to all sorts of shenanigans– lowering cut scores, lowering standards, reducing growth targets, cheating– in an attempt to shed their low-performing status as quickly as possible.

I can’t help but question why on Earth the Los Angeles Times would decide to do something like this. Given the shortcomings of value-added analysis, their publicly searchable database will essentially function as a blacklist, at least for those teachers who end up on the wrong side of the bell curve. How are teachers supposed to collaborate in good faith with school leaders and the broader community when facing the threat of being blacklisted? Improving educational outcomes for low-performing students is immensely difficult. How does fostering mistrust among teachers, school leaders, and community members serve that goal? (And have they even considered the logistical problems? What recourse will people have if there are mistakes in their assessment, or if it’s later discovered that there were problems with test administration or scoring? How quickly could corrections be made? How can one go back in time and restore the damage– emotional distress, reduced employment prospects– of a faulty classification? Are they prepared to handle the demands of thousands of professionals who feel they’ve been unfairly tarnished as ineffective? Who is monitoring this?)

And if the practice of publishing teachers’ names alongside faulty measures of “value” catches on, who in their right mind is going to want to become a teacher?