Skip to content

Scandalize Their Names

August 16, 2010
by

I met my brother the other day

And gave him my right hand

As soon as ever my back was turned

He scandalized my name.

Now, do you call that a brother? (No, no.)

You call that a brother? (No, no.)

You call that a brother? (No, no.)

Scandalize my name.

When I read the Los Angeles Times’ August 14th article, “Who’s teaching L.A.’s kids?” written by Jason Felch, Jason Song, and Doug Smith, I literally felt chills run up my spine. For those who don’t already know, the article represents the debut of the Timesvalue-added analysis of seven years’ worth of Data from over 6,000 LAUSD elementary school teachers. The paper plans to publish a database of individual teachers’ estimated effectiveness and articles on the subject over the next few months.

I’ve discussed my qualms about the current state of value-added analysis in the past, as have several professionals far more knowledgeable on the matter. Though it sounds great in theory, value-added analysis is simply too imperfect a science (at present) to be fairly used while making important decisions about public schools. The most commonly used methods of assessing a teacher’s added-value have very high error rates, and that’s in addition to the inherently problematic nature of the Data upon which those calculations are based. The writers of the Times article even note that the National Academy of Sciences has said that “the approach [is] promising but should not be used in ‘high stakes’ decisions– firing teachers, for instance– without more study.”

Felch, Song, and Smith go on to write that “no one suggests using value-added analysis as the sole measure of a teacher. Many experts recommend that it count for half or less of a teacher’s overall evaluation.” And yet, the Los Angeles Times has taken it upon itself to publish the names and statistics of thousands of Los Angeles teachers because the organization has decided that it “offers the closest thing available to an objective assessment of teachers.” Never mind that the quality of the tests may change from year to year, or that what is considered “proficient” may also change, or the lack of random assignment of students to teachers, or any of the other factors may affect the comparability of student test performance from one year to the next. And never mind the fact that it’s pretty difficult to decide how to control for factors that are known to affect student performance– socioeconomic status, language status, giftedness, etc.– let alone the ones that we can’t and don’t reliably measure (how supportive the school environment is to teachers or students, students’ resilience, emotional stability, confidence…).

Apparently, it is now OK for a newspaper to contract with whatever “expert-for-hire” they like and paint scarlet numbers on public school teachers.

There are many things about the article that I find disturbing. For instance, Felch, Song, and Smith’s language sometimes crosses the boundary between objective, accurate reporting and editorializing. For example, they state that (emphasis mine)

…year after year, one fifth-grade class learns far more than the other down the hall. The difference has almost nothing to do with the size of the class, the students, or their parents.

It’s their teachers.

Contrast that statement with the research-based finding that “more than 90% of the variation in student gain scores is due to the variation in student-level factors that are not under the control of the teacher.” (Indeed, even the report prepared for them by economist Richard Buddin acknowledges the difficulty of separating student-level factors from each other, or from teacher influence.) Also noteworthy is the way in which the writers cast the mostly Latino parents as helpless victims of bad teachers and a cowardly school district. (The “hard-working parents…trust educators to do what’s best,” while “education officials have long known of the often huge disparities among teachers…but rather than analyze and address these disparities, they have opted mostly to ignore them.”)

And let’s not even get into the politically incorrect findings regarding the relative effectiveness of teachers of different races, or the effects of certain student characteristics (giftedness, race) on calculations of teacher effectiveness. Those kinds of findings lead me to question just how “objective” or accurate these calculations are, if you can find weird effects for things like teacher race, or any effects of student characteristics on teachers’ ability to teach. (Isn’t that the point, after all? That we’re supposed to be separating out things that the teacher can’t control from the things that the teacher can control?)

But the most disturbing thing is the precedent it sets for how efforts to improve teacher and school quality may proceed in the future. For starters, adding yet another incentive– the specter of public humiliation– to teach to standardized tests can hardly be considered a good thing. If our goal in education is to produce well-rounded critical thinkers who are ready for 21st Century challenges, does it make sense to assess their progress–and teachers’ effectiveness– with 20th Century pencil-and-paper tests?

Note that when describing the two teachers who were identified as highly effective, the writers chose  to share instances where the teachers and students were engaged in some pretty old-school classroom behaviors. One of the teachers has a student calculating a math problem on the board while his peers watched from their seats. The other is described as “grilling her students on vocabulary.” (Whether the students understand the underlying logic of fractions, or are able to flexibly use new words as they express themselves while speaking or writing, is an open question.)

That’s to be expected in a system that relies upon traditional standardized tests. Traditional tests of just the basics will tend to promote traditional ways of teaching just the basics, however “animated” said teaching might be. If you’re a teacher who knows your name will be published alongside someone else’s measures of your value, you’ll probably feel a pretty strong pull to do whatever it takes to make sure that those test scores go up. Knowing you’ll be directly compared to your colleagues probably won’t foster collaboration, which is a key feature of successful schools, and it may further encourage the narrowing of the curriculum as well as the ways in which students are taught and evaluated.

And what about the emotional and professional toll of having your name listed for all to see, next to faulty numbers that attempt to communicate your worth as a practitioner? Whatever value there might eventually be in using value-added assessment to promote increased teacher effectiveness, how exactly are teachers supposed to make the best use of it when they feel stigmatized and worried about their future prospects in their profession? We’ve already seen the effects of stigma on schools, districts, and state agencies in the number of such organizations that have resorted to all sorts of shenanigans– lowering cut scores, lowering standards, reducing growth targets, cheating– in an attempt to shed their low-performing status as quickly as possible.

I can’t help but question why on Earth the Los Angeles Times would decide to do something like this. Given the shortcomings of value-added analysis, their publicly searchable database will essentially function as a blacklist, at least for those teachers who end up on the wrong side of the bell curve. How are teachers supposed to collaborate in good faith with school leaders and the broader community when facing the threat of being blacklisted? Improving educational outcomes for low-performing students is immensely difficult. How does fostering mistrust among teachers, school leaders, and community members serve that goal? (And have they even considered the logistical problems? What recourse will people have if there are mistakes in their assessment, or if it’s later discovered that there were problems with test administration or scoring? How quickly could corrections be made? How can one go back in time and restore the damage– emotional distress, reduced employment prospects– of a faulty classification? Are they prepared to handle the demands of thousands of professionals who feel they’ve been unfairly tarnished as ineffective? Who is monitoring this?)

And if the practice of publishing teachers’ names alongside faulty measures of “value” catches on, who in their right mind is going to want to become a teacher?

About these ads
15 Comments leave one →
  1. Jesse permalink
    August 16, 2010 8:10 pm

    Sadly, the mainstream media has decided it mission is to attack public schools and their teachers. The only way to fight this nonsense is to boycott companies/products that buy adds in the LA Times, and write to CEOs explaining why.
    Sincerely,
    Jesse

  2. August 16, 2010 8:17 pm

    Sabrina,

    I have read the Times article and your post raises many excellent points. While I agree that publishing the database will produce an incomplete and misleading picture of teacher effectiveness, I thought the Times article by itself was fair. The reporters reported the data and got the teachers’ side of the story.
    The article is generating a lot of negative reaction in the educational blogosphere; however, I think a few points are being ignored:

    1. I was surprised that the two “ineffective” teachers were unaware of their students’ test scores until informed by the Times reporters. That says something about their districts’ commitment to improvement.

    2. There seems to be continuing strong opposition among teachers to using empirical data as a tool for teacher evaluation — even if it’s only one of several tools. Teachers need to accept the fact that it’s going to happen. It already is happening in places like Washington, D.C.

    3. Life is not fair. Employees are evaluated on the basis of imperfect criteria.

    4. Public-school teachers’ salaries are paid by taxpayers. Taxpayers are entitled to more information about public employees’ performance.

    5. Our current system of teacher evaluation allows bad teachers to teach for 30-year careers.

    • August 16, 2010 8:46 pm

      Thanks for reading, John. You’re right– life isn’t fair. It’s about to get a whole lot less fair for the kids whose schools already have a hard time filling teaching positions, if this sort of practice is widely adopted.

      I agree that the public is entitled to certain information about public employees. But who benefits from publishing misleading information about them?

      Most teachers don’t object to useful accurate data being used as one of many evaluation criteria; we object to the fact that distorted data has become the primary criterion for judging schools and is becoming so for teachers. (And just because injustice is becoming more common does not mean people should accept it!)

      Finally, teachers do not have positions for life. There is a system of due process in place for removing ineffective teachers. It is in place to prevent administrators from capriciously firing teachers without just cause (as frequently happens elsewhere in the working world, and during many teachers’ probationary periods). In some places it takes a little while, but if administrators and districts are doing their jobs, it is completely possible–and not terribly difficult– to remove underperforming teachers. And of course, reckless or dangerous behavior can result in immediate termination, just like anywhere else. (There are many more instances of wrongful termination of teachers than most people realize, though; read through some of the teacher stories to learn about some of them, and keep reading the site. More are forthcoming.)

      • Espanolsrs permalink
        August 16, 2010 10:13 pm

        I can see where you are coming from, John. You have some valid and reasonable point. For me it’s so frustrating because I think that because the majority of the population went through the public school system, there is a perception out there that teaching is an easy profession, and that because they went through the schools that they know all about them and how they work/run and think that firing all the teachers would solve the problem.

        Here’s the reality: There are so many poor leaders in the school systems and that needs to be fixed first. Administrators hire new teachers, have no really good/effective mentoring program in place to help the new hires, give the new hires raving reviews because they don’t want to look incompetent for hiring bad teachers but they don’t know how to help them and so the new teachers get tenure, and then the administrator gets a job in a bigger district or a promotion and those bad teachers are now in the system. The administrator has moved on to bigger and better things. The new administrator doesn’t want to go through the paperwork to begin due process, and so the bad teachers continue to teach students who graduate and who then look back on their educational experience and remember the bad teacher. Those same students are the ones who comment on public forums, etc about how horrible and greedy teachers and the teachers’ union are. They are the folks who (and I don’t necessarily blame them) think they have all the solutions to education issues. But those same people do not realize that it is completely possible for bad teachers to be fired, as Sabrina said. But it takes a persistent leader to do it. So why don’t more administrators do it? And why isn’t more emphasis placed on mentoring new teachers right away?

        I want to finish by saying I believe strongly that there are many, many more effective teachers than ineffective ones, and that the overwhelming majority of teachers are in the profession for the right reasons!

    • August 17, 2010 2:36 am

      John – you have opened some new issues here that I’d like to respond to.

      1. I was surprised that the two “ineffective” teachers were unaware of their students’ test scores until informed by the Times reporters. That says something about their districts’ commitment to improvement.

      — Not necessarily true. I’m not saying I agree or disagree with what’s happening there, but I’m a high school English teacher (a supposedly tested subject – though the tests only touch upon a fraction of the curriculum and standards). In 11 years, I’ve never had any such data presented to me, and I’m quite confident that my district is committed to improving. It’s possible that the data haven’t been presented because the data don’t mean that much.

      2. There seems to be continuing strong opposition among teachers to using empirical data as a tool for teacher evaluation — even if it’s only one of several tools. Teachers need to accept the fact that it’s going to happen. It already is happening in places like Washington, D.C.

      — What do you mean by “empirical data”? If you mean any number of things that can be observed, measured, quantified, etc., then I agree there should be some empirical data in evaluations. If you only mean test scores, then I disagree. The scores may be empirical, and simultaneously unreliable and used for invalid purposes. Teachers do not need to accept the imposition of value-added measures as part of evaluation, though I admit the fight looks to be a long one.

      3. Life is not fair. Employees are evaluated on the basis of imperfect criteria.
      — That’s a flimsy reason to accept a deeply flawed system. And in fact, I’m not sure your assumptions about “employees” and evaluations would check out so broadly. Click over to my blog and read my series of posts about the application of business models in education. Also consider that “life is not fair” is an attitude that will do nothing to improve the system.

      4. Public-school teachers’ salaries are paid by taxpayers. Taxpayers are entitled to more information about public employees’ performance.

      — That’s just not true. Our contracts are certainly public, but personal details are not, nor should they be, except as required in legal proceedings. By the way, where do you work? If it’s a publicly traded company and I own stock in that company, I assume you’d be comfortable having me review and publish your performance review, right? Or if it’s a private company that has any government contracts, same deal. Regardless of the flaws in the review, the reviewer, or the lack of contextual information.

      5. Our current system of teacher evaluation allows bad teachers to teach for 30-year careers.
      — Only partially true, since the current evaluation system is also implemented by a grossly understaffed and undertrained administrative system.

      And lest I be pegged as a defender of the status quo, I encourage you (again) to click over to my blog and see what I’ve done to suggest changes that would actually do some good.

  3. Linda/RetiredTeacher permalink
    August 17, 2010 2:19 pm

    Thanks for this excellent description of the Los Angeles Times article. I agree completely.

    Today is the first time in my long life that I have cancelled my subscription to a newspaper. Also, when a solicitor of funds for the Democratic Party called, I told him that I no longer support President Obama and won’t contribute any more money. I am so profoundly disappointed in him. I am referring to the fact that Arne Duncan publicly supported the Times articles.

    Are these people asleep?

Trackbacks

  1. “Scandalize Their Names” | Larry Ferlazzo's Websites of the Day...
  2. Tweets that mention Scandalize Their Names « Failing Schools -- Topsy.com
  3. The Best Posts About The LA Times Article On “Valued-Added” Teacher Ratings | Larry Ferlazzo's Websites of the Day...
  4. Scandalize Their Names « Failing Schools « The Sharing Tree
  5. Don’t Pee on My Leg and Tell Me It’s Raining « The War on Mediocrity
  6. The American Version of the Cultural Revolution? « InterACT
  7. educationinformationonline.com » Blog Archive » Sabrina Stevens Shupe: How About a Measures of Effective Reporting Project?
  8. WitnessLA.com » Blog Archive » Thursday Must Reads: A Preview of New LAT Teacher Rankings…& More

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 156 other followers

%d bloggers like this: