Academics Aren’t Everything

It is repeated like a mantra in K–12 education that our focus must be on “improving student achievement.” I doubt that anyone disagrees with that as our primary goal. But I think, sometimes, that our focus on academics alone may be a mistake.

Why? Because it leads us to concentrate too much on curriculum and teaching and not enough on building better students, future workers, and human beings. We need to pay attention to the nonacademic skills and habits that are just as important to success in school, on the job, and in life.

My perspective on this is informed by nearly 20 years of work for a university and for a nonprofit organization in education outreach — that is, outside groups and individuals trying to help K–12 schools achieve their mission. Over those years, these outreach activities evolved from teacher in-service, push-in classroom activities, drop-in tutoring, and group tours and events for children to almost all one-on-one mentoring. Our focus changed as we realized what was really needed: not just changes in curriculum or teaching methods, but changed students.

Over and over, we saw students from the least supportive backgrounds succeeding through sheer determination, while others with many advantages languished. The difference in outcomes was all about personality and character — and traditional academic support does not teach those vital attributes.

University researchers are now demonstrating that “noncognitive indicators” can predict young people’s success in college and on the job. (See, for example, Michigan State University’s Group for Research and Assessment of Student Potential.) The number one predictor of success is “conscientiousness.” When a teen works hard, perseveres through difficulties, shows up regularly, and completes work as expected, he or she will almost certainly do well.

The critical step is that teens must learn to take the initiative, to realize that no one else can determine where they will go in life. Mentoring is one route to that kind of “eureka!” moment. The programs I have worked with have had as their primary goal this transformation of children into self-motivated adults.

While many teens have accumulated appalling deficits in basic skills, they need much more than subject-matter tutorials. They need help with analyzing their motivational problems, with learning to set study schedules, with devising strategies to get out of the deep holes in which they find themselves. Many have no clue, for example, how to deal with being in over their heads except by denial: ignoring homework, cutting classes, and flunking out. They need to be led through determining what can be salvaged, and negotiating with teachers and counselors over the best step to take next: intensified tutoring, makeup work, alternative assessments, changes in sections or entire classes. They need to become convinced that they are not just victims, that there are things they can do to improve their situation. Moreover, they need to learn that only they can rescue themselves. Mentors can diagnose and work on gaps in basic knowledge, can provide hand-holding and confidence-building, but the students must internalize the fact that no one can pour knowledge into them. They are the only ones who can guarantee their own learning. This realization transforms them in a way that no amount of factual knowledge ever could.

I also know, however, that high-quality, long-term, one-on-one mentoring is very difficult to arrange and to sustain. It is not, however, the only solution to the passiveness that holds so many young people back.

We are now learning ways to structure project-based learning and various kinds of team-based learning to encourage just this kind of social-emotional development. The New Tech High program, for example, which we hope to begin at our high school next fall, offers one such format for developing truly college- and job-ready students. The program site I visited last fall in Indiana offered grades for “work ethic” and “collaboration” as well as for content mastery. One could not receive an “A” without behaving in a dependable and conscientious way. Letting down the team prompted conferences that were very like performance reviews on the job, leading the student through a self-analysis of how he or she had failed and what to do differently. The student-created cultural code also discouraged absenteeism, intimidation, theft, and other bad behavior that will not be tolerated in college or work environments.

The key to the success of this program, I believe, is treating students more like adults (and teachers more like professionals, but that’s another topic). People tend to rise to meet higher expectations, after all. Children and teens can grow enormously through taking on more responsibilities. They gain poise and are empowered by success in completing real-work projects and presenting their results. Nothing builds self-esteem and confidence like accomplishing something you thought was beyond you. Challenge and collaboration, when carefully structured, can allow students to risk failure by aiming higher.

You cannot excel if you never try. Trying and succeeding whet one’s appetite for more. Enthusiastic and self-motivated students can’t be held back.

Teacher “Accountability” Systems

“Value-added” performance measures, which purport to rate teachers by whether a year of their instruction produces a year of learning gains in their students, are the new high-stakes method of deciding who goes and who stays in K–12 education. In pursuit of competitive grants through the American Recovery and Reinvestment Act of 2009 (ARRA), most states (including Michigan) adopted requirements for the use of “effectiveness data” in determining compensation levels for teachers and principals. This data is also, increasingly, being used to justify firing “ineffective” educators. I will limit myself here, for reasons of space, to discussing such performance evaluation schemes for teachers. I have a several concerns about them.

Are they accurate?

Can we all agree that the point of any teacher evaluation system should be to improve the quality of teaching? If so, then such systems must, first of all, be accurate in their analysis of the existing quality. If they do not, in fact, fairly represent the strengths and weaknesses of teacher performance, then they are worthless for the purpose of improving it.

This — the blatant inaccuracy of the evaluations — is the root reason for teacher anxiety and loathing regarding the “accountability” systems to which they are increasingly subject. While many such schemes advocate “multiple measures” of effectiveness, too many are based exclusively on students’ standardized test scores. The error rates in such data are simply unacceptable for high-stakes decisions. I imagine that teachers would not fear evaluations that mistakenly offer them extra help in improving their professional practice, but they would find it unacceptable to lose their jobs over bad data. Wouldn’t you?

Answer to the above question: Student test scores are imprecise measures — they do not fairly and accurately measure student learning outcomes. The U.S. Department of Education’s Technical Methods Report “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang, Mathematica Policy Research, July 2010, http://ies.ed.gov/ncee/pubs/20104004/] concludes that typical value-added performance evaluations of teachers are unacceptably imprecise. Specifically, “in a typical performance measurement system, 1 in 4 teachers who are truly average in performance will be erroneously identified for special treatment, and 1 in 4 teachers who differ from average performance by 3 to 4 months of student learning will be overlooked.” And that is assuming the use of three years’ worth of data; the reliability is approximately half as good for only one year of data. These error rates, they note, are greatly understated by certain assumptions, such as that students are randomly assigned to schools and to teachers. There are no controls, therefore, for differences in resources among schools or for differences among student cohorts.

The authors reference “findings from the literature and new analyses that more than 90 percent of the variation in student gain scores is due to the variation in student-level factors that are not under control of the teacher. Thus, multiple years of performance data are required to reliably detect a teacher’s true long-run performance signal from the student-level noise.”

Moreover, the paper’s authors note that error rates they analyzed are only one factor (they list six others) that must be taken into account in designing and using appropriately any value-added estimators of performance. Not attending properly to such features in design and application makes the use of such schemes for high-stakes decisions both ineffective in achieving their stated purpose and manifestly unjust.

Do they improve teaching and learning?

The inaccuracy and unfairness of judging teacher performance almost entirely by student standardized test scores is only one problem with value-added performance evaluations. If we are truly interested in improving educational outcomes — and that is the point, isn’t it? — we must also consider the collateral damage they do.

• They undermine collaboration, which effective schools must encourage. Scoring teachers in ways that make them compete against one another inhibits or even discourages the sharing of lore and techniques that can make everyone better teachers. I recall the havoc created under Jacques Nasser at Ford several years ago when engineers were force-ranked in their evaluations (my late husband was coerced into doing some of the ranking then). This zero-sum game, in which one person winning meant another losing, penalized working together and encouraged sabotage of colleagues. It has taken many years to recover from the damage done. Is this the aim of folks who want schools to be “run more like a business”?

• They distort educational decision-making. Which classes students are placed in (should they be challenged or slotted where they can safely deliver higher test scores?); which students are held back or eased out of a school (dumped by charters or shunted into “alternative” schools); what is emphasized in classes (only what is tested); what level of thinking is encouraged (rote memory versus analysis or independent thought) — practices are encouraged that explicitly undermine good education.

• They are contradictory and inconsistent. On the one hand, today’s reformer bloc assumes that anyone can teach — or can administer a school or a district, for that matter. They find solutions in recruiting and placing “outsiders” through Teach for America or via the Broad Superintendents Academy, for example, with brief training and no experience. Yet the punitive teacher evaluation systems are predicated on the assumption that firing “bad” teachers and principals will improve schooling. There is no sense that these people, who have committed time, money, and working years to their professions, can or should be helped, instead, to improve their practice.

I recognize that public education is inherently political, in that we ultimately do what the people want. That does not mean, however, that those of us on the inside cannot try to change the tenor and direction of the public conversation. I deeply believe that the punitive tone of today’s education reform talk undermines the putative aims of reform. Blaming allegedly incompetent educators for all that is wrong in our society may get the rest of us off the hook (and, I add cynically, may serve the unstated interests of certain politicians and of certain corporations that benefit enormously from public spending on testing, textbooks, software, distance learning, et cetera ad nauseum), but scapegoating will not fix those problems. Never has and never will.

What would work?

I cannot stop at stating the problem but must at least make reference to my own preferred solutions. Teacher performance schemes are not all the rage just because we, collectively, want to blame teachers for things not under their control. All of us and all of our children have experienced at least one truly terrible teacher in our lives, and all of us would like to spare others that experience. So, how should teachers be evaluated?

Classroom observation is a time-honored way of seeing how teachers actually teach, even if it is rarely done often enough or with enough specific, useful feedback to make a difference. Teacher Larry Ferlazzo writes that effective observers “know our school, our students and me — and [have] judgment and skills I … respect. I know they are genuinely concerned about my professional development. They understand that helping me improve my skills is the best thing they can do to help our students…. These purposeful visits have produced detailed and helpful feedback that has made me an even better educator.” The Mathematica report notes that “value-added measures and principals’ assessments of teachers, in combination, are more strongly predictive of subsequent teacher effectiveness than each type of measure alone” [emphasis added].

Multiple data points regarding student assessment are required for valid measurements of student learning. A single set of pre- and post-tests annually is insufficient. Teachers working together to create common curricula and assessments for the same classes can not only assure that all students get the same exposure to a subject, but also help one another jointly improve their classroom practice to achieve similarly high levels of learning. This kind of professional collaboration actually does produce the result we say we want from performance evaluation systems: teachers helping one another to consistently become better at their craft, so that students can perform better not just on tests but in life.

For the point of the evaluation is not just to find out how well or how poorly teachers are performing. It is — or at least it should be — to make them better. If we do not believe that teachers, too, can learn, then we do not believe in education at all.

Note: the statistical mathematics used in the Mathematica study cited goes beyond any “expertise” I developed in a single statistical methods course 40 years ago, so my analysis is subject to error. The conclusions that are not direct quotations may be inadvertently misrepresentative.

Publication URL corrected 8 Jan 11.

May 2011 addition: you have got to see this animation summarizing Dan Pink’s Drive work on motivation and incentives!