this field is required

As you are likely to have heard by now, the Los Ange­les Times recently con­ducted and pub­lished a value-added analy­sis of some of the city’s ele­men­tary school teach­ers, using data that had been col­lected by the school dis­trict but never pre­vi­ously ana­lyzed in this way. There was a nice sum­mary of the value-added analy­sis and the ensu­ing con­tro­versy in the New York Times this week.

And here’s a quick but thought­ful cri­tique of that sum­mary over at the Quick and the Ed. Its author dis­pels two com­mon crit­i­cisms of the value-added analysis:

1. We shouldn’t crit­i­cize value-added analy­sis sim­ply on the basis that it shows many teach­ers’ effec­tive­ness as shift­ing sub­stan­tially from year to year. It’s pos­si­ble that teacher effec­tive­ness *does* shift from year to year, for what­ever reason.
2. Because this par­tic­u­lar method of value-added analy­sis uses indi­vid­ual stu­dents’ own pre­vi­ous scores as a base­line for mea­sur­ing progress, it does not penal­ize teach­ers for hav­ing slower stu­dents in their classes (at least, the crit­i­cism must be more sub­tle, as a com­menter sug­gests).

Still, the most inter­est­ing thing I’ve read on the sub­ject remains cog­ni­tive sci­en­tist Daniel T. Willingham’s “3 key fac­tors in teacher eval­u­a­tion (beyond the hype of value added).” Accord­ing to him, the key fac­tors are:

1. Fig­ur­ing out the goals of school­ing, and only then craft­ing assess­ments to mea­sure teach­ers’ suc­cess at attain­ing those goals. It’s back­wards to assume that what­ever we are able to test or assess must be the goal of schooling.
2. Tak­ing into account the ages of stu­dents, rec­og­niz­ing that respon­si­bil­ity for stu­dents’ learn­ing prob­a­bly falls more fully on teach­ers of early ele­men­tary school­ers than on those of high schoolers.
3. Remain­ing cog­nizant of the lim­i­ta­tions of any eval­u­a­tion sys­tem in thought­fully choos­ing cri­te­ria for fir­ing teach­ers that bal­ance the costs of keep­ing bad teach­ers with the costs of fir­ing good ones.

Each of Willingham’s three key fac­tors gets at what is essen­tially the major prob­lem with value-added analy­sis: its results, even if accu­rate, lead us away from philo­soph­i­cal ques­tions about edu­ca­tion and teach­ing, the answers to which have impor­tant prac­ti­cal and pol­icy impli­ca­tions. So I want to elab­o­rate on them, with that in mind.

The goals of school­ing are far from non­con­tro­ver­sial. Although most peo­ple would agree that pro­fi­ciency in math and read­ing are amongst its goals, there is sig­nif­i­cant dis­agree­ment as to what con­sti­tutes pro­fi­ciency and how it ranks in impor­tance as com­pared to other goals of school­ing (char­ac­ter devel­op­ment, social­iza­tion, prepa­ra­tion for the work­force — which may or may not require pro­fi­ciency in math and read­ing, etc). It’s a mis­take to estab­lish teacher eval­u­a­tion pol­icy based on value-added analy­ses with­out hav­ing clar­i­fied at least some of the goals of school­ing and their rel­a­tive impor­tances. As should be obvi­ous, this value judg­ment can’t be gen­er­ated from the value-added analy­sis itself. Rather, it will be out­come of philo­soph­i­cal dis­cus­sion regard­ing the moral value of char­ac­ter development/socialization/preparation for the work­force, the best kind of life for a human to lead, how these respon­si­bil­i­ties should be shared between schools and fam­i­lies, etc.

Willingham’s sec­ond point, about the ages of stu­dents and their respec­tive lev­els of respon­si­bil­ity for their own learn­ing, also raises moral ques­tions. The dis­course sur­round­ing value-added analy­sis has seem­ingly taken entirely for granted that teach­ers ought to be doing all that they can to raise stu­dents’ test scores, regard­less of their ages. This stands in need of defense. While we may be reluc­tant to blame 7 or 8 year old stu­dents for fail­ing to learn math and read­ing, it may be appro­pri­ate to blame 16 and 17 year olds for fail­ing to progress in those sub­jects. Some high school teach­ers may man­age to raise teenagers’ test scores sig­nif­i­cantly, and they will come out look­ing bet­ter than other teach­ers in the value-added analy­sis. But, even if rais­ing stu­dents’ test scores were of the most press­ing impor­tance in their early years, other func­tions may be more impor­tant for teach­ers to engage in at the high school level — maybe help­ing stu­dents to think about their future edu­ca­tional and career plans, and tak­ing a more lais­sez faire approach in order to begin accli­mat­ing them to the “real world.”

So this ties back into the pre­vi­ous point, about the goals of school­ing. If rais­ing math and read­ing test scores is, lit­er­ally, the one and only proper goal of school­ing, then all teach­ers should be expected to do so each year. How­ever, there may be many other goals of school­ing that are more dif­fi­cult to test. Teach­ers will need to make trade­offs in pur­su­ing these var­i­ous goals, depend­ing not only on their rel­a­tive impor­tances but based on what their stu­dents are like. Maybe, in some par­tic­u­lar class, some of the stu­dents are ok at math but have social dif­fi­cul­ties. Assum­ing that social devel­op­ment is one of the goals of school­ing, the teacher might rea­son­ably decide to devote more time to group work than to math drills. As a result, the stu­dents might progress more slowly in math than in pre­vi­ous years, while hav­ing made strides socially that do not show up on any test.

The third key fac­tor, about cri­te­ria for fir­ing teach­ers, raises even more moral ques­tions. There are costs asso­ci­ated both with keep­ing bad teach­ers and with fir­ing good teach­ers. If you keep a bad teacher, many stu­dents in his or her classes will fail to learn as much as they could have learned with a bet­ter teacher, neg­a­tively impact­ing their future edu­ca­tional out­comes and maybe even sig­nif­i­cantly harm­ing their life prospects. If you fire a good (or at least ade­quate) teacher, you unduly harm that teacher and demor­al­ize her col­leagues (and the replace­ment teacher might be an unknown quan­tity who turns out to be worse). We might priv­i­lege stu­dents’ well-being over teach­ers, erring on the side of fir­ing, or we might priv­i­lege teach­ers’ well-being in order to show respect to what many con­sider one of the most impor­tant pro­fes­sions. The meth­ods of eco­nom­ics may tell us how to set teacher fir­ing cri­te­ria so as to be finan­cially cost effec­tive, but that’s not nec­es­sar­ily the end of the story from the moral perspective.

None of this to say that value-added analy­sis is “bad,” or has no legit­i­mate pur­pose. Its results might be quite accu­rate and use­ful to some degree, as is per­haps the case in this Los Ange­les sit­u­a­tion. But we need to real­ize how it does — or doesn’t — square with our con­cep­tion of what edu­ca­tion, and teach­ing, ought to be (and, if we lack such a con­cep­tion, we need first to develop one). At the end of the day, value-added analy­sis is a descriptive/evaluative tool, and not a nor­ma­tive one.