Word Attack: “Objective”

objective (adj): not influenced by personal feelings, interpretations, or prejudice; based on facts; unbiased: an objective opinion.

Source: Random House Dictionary

As reformers of all stripes rethink teacher evaluation, and as states try to reduce budget shortfalls by cutting education spending, there have been many discussions about the best way(s) to ensure that we keep the best teachers in their classrooms, and eliminate the worst. Though there is much to be said about the various proposals people have put forth, and though I have strong opinions about most of them, there’s one oversight in the conversation that has been bothering me for months.

It appears that many people don’t quite understand what “objective” means, or are confused about when it’s appropriate to use the term.

For instance, I recently had a conversation with a political staffer, in which we debated the merits (or lack thereof) of using student test scores to evaluate teachers. While arguing in favor the idea, he defaulted to an argument I’ve heard too many times to count: “Well, don’t you agree it would be nice to have an objective measure of a teacher’s performance?”

“Perhaps,” I responded. “But standardized test scores are not objective.”

Test developers use their judgment (which will be subject to their personal, cultural, and institutional biases) to create test items they think embody the learning standards their test is supposed to measure. And aside from certain types of math items that consist solely of computation and rely on a machine to compare a student’s answer to the correct one (assuming, of course, that the company doesn’t screw up and that there are no mechanical failures), the majority of test items require someone to interpret whether, and to what extent, a student’s answers reflect an understanding of whatever the test item is supposed to measure. Students’ scores can change greatly depending on the instructions given to scorers, people’s choices about what constitutes “proficiency,” or numerous other factors that can’t be considered objective.

Even scoring machines are programmed to mimic the kinds of judgments human scorers give, by analyzing certain features of test-takers’ answers (word length, response length, etc.) that are associated with the kinds of scores generated by humans’ subjective scoring practices. (If that process still smells a bit objective to you…OK…but then we need to talk about whether it’s adequate to the task at hand! My totally biased opinion: Displays of truly intelligent performance can’t be fairly or adequately assessed by looking at how many words were put on a page, or how long those words tend to be.)

Lesson number one: Just because a process results in a number, that doesn’t mean that the process or the number are objective.

The “objective” line also comes up when comparing the current last-in-first-out (LIFO) system for layoffs to merit (another word to be explored at length!) -based layoff schemes. Though there are good arguments to be made for and against the seniority system (see this post by teacher Heather Wolpert-Gawron for an excellent exploration of the pros and cons of LIFO, and other ways to approach evaluation), it’s frustrating to see how often people pan it because they believe the alternatives are more “objective.”

For instance, New York City-based Educators 4 Excellence proposed the following system to determine which teachers would go first when layoffs were ordered. Their system would lay off teachers who:

1. Are chronically absent as defined by missing more than eight percent of a given school year, without a documented medical reason.
2. Received an unsatisfactory rating in the past year.
3. Have not been able to find a full-time position in the past six months (the Absent Teacher Reserve (ATR) pool).

Before going on to explain their rationale behind each choice, they state that “The policy team chose these three criteria because they are objective and proven measures that influence teacher and student performance.”

Now, the first criterion is objective: assuming the necessary data have been entered correctly, it should be a simple thing to figure out which teachers have been absent eight percent of the school year or more. The second two…not so much. Unsatisfactory ratings are based primarily on an administrator’s judgment, which are definitely influenced by that person’s feelings, interpretations, or prejudice. (They’re also based in part on student test scores, which cannot be considered objective– or fair, or accurate, in some cases.) And though determining whether or not a teacher is in the ATR pool should be a simple fact-finding exercise, the decision to excess a teacher is subject to the same subjective issues as the evaluation process.

(I find it funny, too, that they’re promoting their protocol over the seniority system because it’s “objective” when it isn’t…and the seniority system is. You can say whatever else you want about it, but LIFO is objective. Assuming that hire dates have been entered correctly, ranking teachers based on when they started working for the district is a process based on facts.)

Lesson number two: Just because you like a certain process, that does not make it objective. And just because you don’t like its alternative, that doesn’t mean the alternative isn’t objective.

As we continue to look for better ways to make difficult educational decisions, we should make an effort to look at the pros and cons of all proposals on the table, and discuss them honestly and openly so we can create something we can all agree is fair. Regardless of where we stand on any given issue, we should all concede that most of these dueling proposals are heavily subjective– and that’s not necessarily a bad thing. What’s really at issue here is whose judgment we decide to trust, whether their judgment is well-informed, and whether we’ve done enough to minimize the effects of a strong bias in one area or another.