Measuring teachers by how much value they add to their students' tests scores is an idea that I advocated back in the last millennium, and has since become the state of the art conventional wisdom, and, now, it is being implemented into universal law in Colorado, all without anybody actually showing it does a whole lot of good. Dana Goldstein has a funny article in the American Prospect, "The Test Generation," about what that actually means in the classroom:
On exam day in Sabina Trombetta's Colorado Springs first-grade art class, the 6-year-olds were shown a slide of Picasso's "Weeping Woman," a 1937 cubist portrait of the artist's lover, Dora Maar, with tears streaming down her face. It is painted in vibrant -- almost neon -- greens, bluish purples, and yellows. ...
The test asked the first-graders to look at "Weeping Woman" and "write three colors Picasso used to show feeling or emotion." (Acceptable answers: blue, green, purple, and yellow.) Another question asked, "In each box below, draw three different shapes that Picasso used to show feeling or emotion." (Acceptable drawings: triangles, ovals, and rectangles.) A separate section of the exam asked students to write a full paragraph about a Matisse painting.
Trombetta, 38, a 10-year teaching veteran and winner of distinguished teaching awards from both her school district, Harrison District 2, and Pikes Peak County, would have rather been handing out glue sticks and finger paints. The kids would have preferred that, too. But the test wasn't really about them. It was about their teacher.
Trombetta and her students, 87 percent of whom come from poor families, are part of one of the most aggressive education-reform experiments in the country: a soon-to-be state-mandated attempt to evaluate all teachers -- even those in art, music, and physical education -- according to how much they "grow" student achievement. In order to assess Trombetta, the district will require her Chamberlin Elementary School first-graders to sit for seven pencil-and-paper tests in art this school year. To prepare them for those exams, Trombetta lectures her students on art elements such as color, line, and shape -- bullet points on Colorado's new fine-art curriculum standards.
Studies prove that Yale graduates with high-paying jobs tend to have stronger opinions on the merits of Picasso v. Matisse than do high school dropouts in prison, so, logically, that proves that we can't start lecturing children too young on Matisse and Picasso, because if they don't go to Yale, they're bound to go to jail.
Anyway, the question I have about measuring public school teachers through value-added testing is this: How many private schools do that? We have thousands of private schools in America, but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?
We have thousands of private schools in America, but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?
ReplyDeleteBecause they're probably not interested enough in the results that would be output from such testing.
Student achievement is the product, as a first approximation, of student aptitude and teacher effectiveness. My guess is that the teacher effectiveness is a very small component so long as the teacher has basic competency. A bad teacher can depress student achievement but a good teacher can't really raise it that much beyond a mere competant teacher.
IF, and that might be a big IF, this is understood by private schools and they implicitly know that their reputations are dependent on the selection effect of their students then so long as they can identify and eliminate the bad teachers, the process of measuring value added by teachers really isn't worth the effort and they wouldn't know what to do with the results if it's true that the value added by teachers is only responsible for, say, 10% of student achievement variance.
We can test the hypothesis, in broad strokes, by examining the effort and the selection process that private schools apply to both the teacher hiring process and the student selection process. If teachers really made that big of a difference then we'd expect to see a very selective hiring process. I don't believe that this is really the case with most private schools. What I see instead is far more effort expended on selecting the students.
A lot of lower end private schools seem to compete on how little they can pay teachers.
ReplyDelete"would have rather been handing out glue sticks and finger paints."
ReplyDeleteI'm shocked that helicopter parents haven't lobbied the government to have those toxins eliminated from school grounds.
That's one big downfall of value-added evaluations of teachers -- we know that they can't make a huge difference in boosting IQ. So the more that the tests become de facto IQ tests, like an art test about color and composition, the more impotent the teachers will be to add any value.
If the tests looked at more applied and trainable tasks, then the teachers would do better. Like an art test that measured how well you could make a papier mache sculpture, or use the watercolor wax resist technique in painting -- things that used to be common.
Same goes even for more abstract subjects like English. The more the test asks about how the poet uses metonymy, the more it's a de facto IQ test. But writing skills can always be improved -- why not test that instead?
Because applied arts are too real, and not intangible enough for the ed school cult that sees children as zombies who can only become fully human if parents just butt out and allow the ritual priests to breathe the soul of New Math into their virgin brains.
Private schools tend to be primarily for making connections, opening doors and advancing academic or work careers. It has little to do with any "learning" you image could be objectively measured.
ReplyDeleteStudents/parents assess private schools primarily by how successful they are in getting graduates into HYP or prestigious careers. Their teachers are assessed by students and their parents. Student evaluations are virtually tossed at public universities but critical in tenure reviews at 2nd tier small liberal arts schools.
Like Avis, second tier private schools have to try harder to please parents and students. Actual teaching effectiveness is a lesser component in the mix. More important are giving out inflated grades, being lively and engaging.
The top tier private schools have their pick of applicants. They don't have to either teach effecitvely or please their students/parents nearly as much.
"The test asked the first-graders to look at "Weeping Woman" and "write three colors Picasso used to show feeling or emotion." (Acceptable answers: blue, green, purple, and yellow.)"
ReplyDeleteOh man. I live in CO. Went to school here. I have no idea what Weeping Woman has to do with how I live as a citizen here.
The kids are going to think this is all BS, and they'll be right.
The point of an HS education, I thought, was to make you able to get a job and read and write and be a productive citizen.
This Picasso for 1st graders stuff doesn't fit into that. What's the motivation? I don't know. Probably status anxiety. Maybe the curriculum advisors were going through a group manic phase, and the kids are going to pay the price.
Obviously they aren't interested in such pedestrian things as showing kids how to function in the (ahem) globalized, post-internet world.
What do private schools offer that's worth so much, anyway?
ReplyDeleteThis question arose from recent experience.
I was looking into private schools in Lower Merion Township and thereabouts. The public high school there is great - the best in PA, I'm told. It should be, with all the money. Yet there are some private schools around, and, wow, 30k? A year? Just to send your kid up the street to a private school instead of the excellent public school?
The SAT numbers disclosed by these private schools aren't even that good - the kids are going to college, sure, but no place is boasting an average of 2100 or anything - nothing spectacular. What's the deal?
The effectiveness of public schools is a matter of bitter, heated debate. The effectiveness of private schools is judged only by the parents writing the checks. The quality of instruction might well be better in public schools; parents send their children to private schools to control their social environment, for religious or class reasons.
ReplyDeleteAre the parents actually asking for better teacher performance in a way that can be measured? If not, why should the schools bother with assessments? It will just make the good teachers more demanding.
ReplyDeleteMy experience of university teaching persuades me that the best you are ever likely to achieve is identification of the really lousy teachers. And you don't need formal measurements for that.
ReplyDelete"A lot of lower end private schools seem to compete on how little they can pay teachers."
ReplyDeleteDefinitely true where I grew up.
Because the selling point of most private schools is that they don't have NAMs and blatantly unqualified teachers. Anything beyond that is just gravy.
ReplyDeletePrivate schools can get away with lower pay because the teachers enjoy the work more. They have a less troublesome group of children to work with who are more interested in learning, on average. They have more flexibility in teaching, and in designing their curriculum. I knew a young woman with a Masters in Education who was teaching at a private middle school getting 20k/yr and she was happier there than when she was making twice that teaching in a ghetto public school.
ReplyDeleteWhy not?
ReplyDeleteprobably because private schools are free to evaluate teachers holistically at the principal's discretion.
value-added testing is mostly a way to get around union objections that principals are horribly capricious people who can't be allowed any discretion. we don't have such nonsense at private schools.
Measurement methods like value added testing are almost unknown in private schools. And given the number of variables that need modeling, private schools probably don't have enough data to do fine-grained sophisticated RAND style value added measurement. But they aren't doing the simpler things they could do either.
ReplyDeleteMost private schools do not sell themselves to parents on superior academics. They sell themselves as a place where your child will have the right peers. This is true in both parochial and secular private schools, though their meaning of "right peers" changes.
Most private schools just do what they have done before. They have no idea about modern methods of quality engineering, just as they have no idea about standards (which are like requirements in an engineering process). They don't know what "curriculum" means, they don't know what standards-based education is (where you close the loop--you have standards, define your curriculum to meet those standards, test against those standards, grade against those standards.)
What private schools have going for them is culture. If you like their culture and it matches your values, you want your child to go there.
Private schools pick teachers the way most people would think: personal impression, word of mouth, connection to the school already. Private schools often hire alums who already know the culture. Otherwise, they hire from other similar private schools. Parochial schools hire people from their parish, etc.
"Anyway, the question I have about measuring public school teachers through value-added testing is this: How many private schools do that? We have thousands of private schools in America, but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?"
ReplyDeletePrivate (non-catholic) schools consist mainly of white kids being taught by white teachers. Not only is there no achievement gap, there is very little cognitive gap either, with most kids scoring over 120 on an IQ test. So the goals of private schools are different. Liberated from the hopeless task of making NAMs sit still and do academics which are age appropriate for white kids, private schools are free to define and market their mission to parents who can afford $25-30K a year for K-12 tuition and want their kids to attend Harvard.
To guarantee the quality of their product private schools usually administer the CTP-4 exams yearly, but since students were probably admitted on the basis of performance on the WPPSI-III/ERB or SSAT, they usually perform well on the CTP-4, too, which is used by private schools more to fine tune the curriculum rather than to discover who are "at risk" students or bad teachers. Indeed,if private schools do have a mission, it's to encourage smart students to develop skills and behaviours which make them interesting to elite college admissions departments.
Change is coming - slowly but inexorably. The brain was thought by the ancients to cool the blood - sort of a radiator. Its true function has been known for some time now but only in general - interpersonal and interracial differences could not explained because they couldn't be seen.
ReplyDeleteNow they can be seen.
As some of you may know we owe a lot of CAT Scanning to the Beatles. Their recording company EMI was founded to develop brain scanning equipment. The Beatles were a sideline that they hoped might have a hit and help fund the medical research.
One of the most impactful uses of brain scanning was the resolution of the controversy raised by Steven Jay Gould in The Mismeasurement of Man. Gould claimed that the racism of early researchers biased there measurements of skull volume. He used mustard seeds. Most people now accept that it was Gould that had the bias not those he criticized.
In any case with MRIs and CAT Scans we don't have to fill skulls with seeds anymore. The machine just takes a picture of the living brain.
As it happens Black brains are about 80cc smaller in the frontal lobes than Whites. It's as simple as that.
A modern CAT Scan takes only a few minutes and unlike an MRI doesn't induce claustrophobia. You could run a class of students through in an hour. This is much faster than they could be given any group IQ test or any Picasso - Matisse test.
The low scoring public has never accepted the validity of paper and pencil tests. But we are close to having the technology to stratify students into tracks by the quality and quantity of their brains. Parents when shown the pictures of their little darling's brain will have to accept it. They can stop looking for Superman.
On that day most of our obsessive preoccupation with teacher quality differences will melt like a Popsicle in the Kalahari.
Albertosaurus
Studies prove that Yale graduates with high-paying jobs tend to have stronger opinions on the merits of Picasso v. Matisse than do high school dropouts in prison, so, logically, that proves that we can't start lecturing children too young on Matisse and Picasso, because if they don't go to Yale, they're bound to go to jail.
ReplyDeleteHilarious.
Nobody seems interested in all the tedious chemistry, optics, geometry, engineering, etc. that produce the desired effect. All the emphasis is on the effect or, if you will, the sizzle and not the steak. And to 6-year olds no less.
This same fad seems to permeate all education fields. I predict generations of future pseudo-scientists with Ph.D.'s in Global Warming.
I suppose they don't use the tests, because senior teachers are perfectly capable of identifying good and bad teachers without them. As are senior teachers in the public system. But in the public system one needs to do things impersonally.
ReplyDeleteCennbeorc
Mark the calendar. You don't often hear Steve Sailer admit a mistake!
ReplyDeletePrivate schools can keep out or kick out the thugs and low achievers, mostly students of your duskier hues, and so they don't need to go through the charade of pretending the problem with our increasingly third world population of students is bad teachers. Since you've done several entries on this in the past (such as the one on Waiting For Superman) one can only assume that you're trolling.
ReplyDeletePrivate schools are selling the same thing universities sell. An elite peer group.
ReplyDelete-Osvaldo M.
If a kid fails academically or behaviorally in a well-regarded private school, the school can dismiss him and replace him with another on the waiting list.
ReplyDeleteThey don't need to spend resources remediating...which never works well anyway.
Lots of lower-end private schools please parents by making sure there is at least a sense of order in the school. Many parents remove kids from public schools because of the chaos in their public schools and place them, if they can, in parochial schools of mediocre academic reputations (so-called Christian academies and Catholic schools).
ReplyDeleteThese schools in working class communities are indeed superior to the public schools in the area because at least the kids are required to behave. Furthermore, some kids really do begin to prosper when the atmosphere around them is orderly, wouldn't you agree?
Two new large Christian schools have been opened in my town w/in the last few years. They have a huge waiting list and many of the parents trying to get their kids in them are Catholic. The Catholic school here only goes through the 8th grade.
Our two public high schools have been inundated by illegals and by migration of blacks from the Richmond/Oakland area. Care to guess what has happened to those high schools?
"TangoMan said...
ReplyDeleteBecause they're probably not interested enough in the results that would be output from such testing."
I believe the point that Steve is hinting at is this: private schools don't care about achievement tests except for the first one they administer - whether or not they choose to accept you.
Presumably private schools would be using value added testing if it held any great value for them.
ReplyDeleteSetting aside value added testing, what do private schools actually judge their teachers on? What do parochial schools judge their teachers on? What did parents judge their children's teachers on in the 19th century, prior to unified school districts?
I'm guessing the principal or parents just makes or made a subjective judgement. And maybe that's okay. Maybe that's even the best of all possible worlds.
The value added testing fad assumes unionized government employees with contractual rights, drawing from a pool of teachers with four year degrees in teaching (essentially committed to teaching as their whole career). Granted that's the system we're stuck with, its reasonable to try to make it work.
What if, though... What if the system is irredeemably flawed at a basic structural level? What if unionization and tenure are net negatives for the teaching field? What if unified school districts are net negatives compared to decentralization and real local and parental control? What if different curriculums are better for different students? What if its better to have teachers with a degree in math or English and a few weeks of teaching instruction rather than a four year degree in teaching?
This would explain to me the success of home-schooling and the more experimental private and charter schools. The downside is it also suggests to me the way to fix public schools is a frontal assault on the foundations of the system, and I don't quite see how to win that battle.
Exposing 6 year olds to Picasso? Child abuse!
ReplyDelete--What do parochial schools judge their teachers on? ...I'm guessing the principal or parents just makes or made a subjective judgement. And maybe that's okay. Maybe that's even the best of all possible worlds.
ReplyDeleteYou are right about how they make their judgments. But it's a disaster. Now, it still might be the best of all possible worlds, but it's a dire situation.
K-8 education is the crux of the problem. 9-12 is just the end result of the sorting game that already took place. Elementary teachers are no longer capable (and may not see it as their problem) of ensuring their charges learn to read and do arithmetic. Period, end of story. K-8 is a simulacrum of education. Parents do the educating on the side, or with tutors, or informally, and those who received enough of that experience succeed in high school. If you didn't have such parents or such culture at home, you're out of luck permanently.
When I taught at DeVry in the '80s they used that method. Give the students a standardized test at the start and end of the semester. Each teacher was judged on student improvement. I don't think it made a difference in pay, but I did have a moment of glory one semester when my Basic Electronics students came out on top of the whole national system.
ReplyDeleteThe two main models in Colonial America wrt education were the private tutors for the rich gentry in Virginia, and the public schools like Harvard in New England. The latter produced just as good and learned graduates as the former, for every Jefferson a John or Quincy Adams, but with far higher output.
ReplyDeleteWhat distinguished the New England schools was that their schools reflected their communally ordered society. Discipline, diligence, and all that were not a problem because the society itself was orderly and that model spread across America represented the Pre-1965 model of teaching. In that model the only private schools were mostly Catholic or other religious. Now, the whole point of private schools is to stay away from a disordered "diverse" society.
"How many private schools do that? We have thousands of private schools in America, but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?"
ReplyDeletePrivate Schools are selective to begin with since they don't have to TAKE everyone. And since parents are willing to pay MORE, it is assumed that students are of better quality or more serious.
Also, there are two kinds of private school teachers. The ones at fancy schools make lots of money and chosen for high credentials.
Ones at non-fancy ones get paid less than public school teachers, so they are likely in it for idealism than money, which means they're prolly better teachers.
As for Picasso and Matisse(and other such stuff), they should be taught not for testing but for understanding and appreciation. And even kids for some of that stuff. I remember flipping through Picasso paintings in a book my parents--and other arts books. I didn't 'get' it but they were fun. Kids immediately 'click' with weird fun stuff. After all, DR. SEUSS and WHERE THE WILD THINGS ARE have been popular with kids.
ReplyDeleteLeave it to an American Prospect writer to not even bother getting the county right. There is no "Pikes Peak County." Colorado Springs - and the school district in question - is in El Paso County, a major military hub, and one of the most conservative counties in Colorado.
ReplyDeleteBut no matter - it's just flyover country.
"...but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?"
Civil service regs? Private schools have more leeway as to who they can hire and fire, so they probably don't need hard data comparing one teacher to another for the purposes of determining pay raises, tenure, etc. Subjective evaluations by administrators, peers, and even parents are probably more likely. It doesn't seem all that difficult to judge the general quality of a teacher by spending a few hours in their classrooms.
'A lot of lower end private schools seem to compete on how little they can pay teachers."
You would think that teachers would want more competition for their services, and more choices as to their teaching methods, not less, so they would favor vouchers that resulted in more private schools. But teachers viciously oppose voucher systems wherever they arise.
Before and after tests seem like a good idea with math and a few other objective subjects.
ReplyDeletePrivate schools don't do this because like others have said differences in teacher quality are fairly small between the very best and the 10th percentile.
Only the very worst really have an impact, which is negative.
No teacher no matter how good is going to teach 20 students math extra super well compared with an average one, but some teachers can teach 40 students math decently when others can only handle half that.
ReplyDeleteForce of personality is hard to measure objectively but a principal should be able to do it easily.
One good example is a 65 year old middle school teacher I had. She was black and had only a 2-year teacher college degree, but dressed impeccably, knew every students name immediately, enforced assigned seating and reminded the students of their grandmothers.
She informally got more students than other teachers and had no problem keeping things orderly. She should have been paid extra to keep her from retiring.
What if we could build perfect robot teachers for all schools? And if that doesn't work, maybe we could build perfect robot students for troubled schools. Robot teachers teaching robot students. Problem solved.
ReplyDeleteBut then blacks will demand that robots in the inner-cities be programmed to be funky and aggressive, and all hell will break out again.
The parochial school I attended held teachers accountable for AP test scores (which is basically the same as value added since you needed to do well in the previous year's science class to be allowed to take an AP class).
ReplyDeleteNon-AP teachers were not held accountable for how much we learned, as far as I could tell. They were judged based on lesson plans and on the vice principal sitting in and evaluating one class a year. Some of the teachers were really truly bad. (E.g. one fancied herself in the cool aunt role, and spent literally all class time talking about how someone had a fight with her mother and someone else didn't know whom to ask to the prom. One quarter she was unable to give any exam because we hadn't covered any material at all.) I don't know whether they were kept on because the administration didn't know that they weren't actually doing what their lesson plans said, or whether the administration had to bide their time until some compentent person got fed up with the public school system and was willing to take their appallingly low pay.
You can't measure teacher performance by student performance.
ReplyDeleteReason: kids have free will. They perform as they will.
Kids aren't factory products. If they were inanimate objects, rolling along an assembly line, then it would make sense to grade teachers on quality control.
Should we judge parents' performance on how well their children turn out? On an individual level, the absurdity of that is recognized. For example, we consider it simply unfortunate that Dr. x's son is a lazy drunk; and we feel sorry for Dr. x. But on a collective level, people's thinking suddenly switches. Juvenile delinquency? Call Child Services and remove the kids from the home!
You can influence a child who is, and who chooses to be, open to that influence. That ain't 100%.
The problem here is Behaviorism. If the kids don't conform to spec, there must be a flaw in the operant conditioning.... Teachers college strikes again.
So how SHOULD teacher performance be evaluated? If a teacher works the lesson plan successfully and keeps order, that's competence.
This is funny. I remember being made to look at Weeping Woman in the fourth grade, the year the painting was stolen from a gallery not far from where I went to school.
ReplyDeleteI remember calling it a piece of crap (or words to that effect) and being told that I didn't yet understand it. Uncharacteristically, I didn't answer back to the teacher but privately wondered what the hell there was to 'understand,' art being something that is either attractive or it isn't and this POS was not.
"This is funny. I remember being made to look at Weeping Woman in the fourth grade, the year the painting was stolen from a gallery not far from where I went to school.
ReplyDeleteI remember calling it a piece of crap (or words to that effect) and being told that I didn't yet understand it."
I couldn't remember which piece of Picasso's crap it was, so I googled it.
Picasso's Piece of Crap
It reminds me of the painting Bruno Anthony's crazy mother creates in Hitchcock's Strangers on a Train.
Mrs. Anthony's Piece of Crap
Seriously, I think both are much too scary for little kids.
"You can't measure teacher performance by student performance.
ReplyDeleteReason: kids have free will. They perform as they will."
This is too simple. It is true that you can lead a horse to water but you can't make him drink, but you can still do a good or bad job on the leading stage.
If a teacher doesn't present any material, you're not going to learn anything. If a teacher lets you get away with copying out of Cliff Notes, a lot of kids will take that short cut. On the other hand, if a teacher takes time to write careful individual comments on your papers, you're going to improve much more than if they just give you a red check at the top.
Now of course it's also true that if you send a message to students that they shouldn't have to work because their learning is totally the teacher's responsibility, they're probably going to slack and learn less than they could. So you do have to be very careful how you design your teacher accountability program. But it's not an incoherant concept.
"Weeping Woman with Hankerchief" is a work of genius. What it has to do with educating children is a different matter. You shouldn't teach the Heisenberg uncertainty principle before teaching addition and subtraction.
ReplyDelete"Weeping Woman with Hankerchief" is a work of genius."
ReplyDeleteCan't agree. Even after watching the whole film devoted to an appreciation of Guernica, I can enjoy Picasso's colors and shapes but not how he employs them.
I think Picasso is to painting what Dylan is to song-writing.
You want to talk a work of genius, pick a canvas--any canvas--by Rothko.
"private tutors for the rich gentry in Virginia, and the public schools like Harvard in New England"
ReplyDeleteIn England they might be called "public schools" (as opposed to "state schools"), in America we call them "private", as indeed Harvard still is today. Interestingly, the Puritans often made education mandatory for children, but that didn't mean they paid for it through public funds. Presumably people too poor to afford it weren't supposed to immigrate to their colonies in the first place.
David:
ReplyDeleteKids aren't factory products. If they were inanimate objects, rolling along an assembly line, then it would make sense to grade teachers on quality control.
Try telling that to educrats.