In the comments to my recent post on the Golden Age of Test Creation, Mitch points me to Linda Darling-Hammond, the prominent Ed Schooler from the Stanford Ed School, explaining how better tests would make American students smarter:
Whereas students in most parts of the United States are typically asked simply to recognize a single fact they have memorized from a list of answers, students in high-achieving countries are asked to apply their knowledge in the ways that writers, mathematicians, historians and scientists do.
In the United States, a typical item on the 12th grade National Assessment of Educational Progress, for example, asks students which two elements from a multiple choice list are found in the Earth's atmosphere. An item from the Victoria, Australia, high school biology test (which resembles those in Hong Kong and Singapore) describes how a particular virus works, asks students to design a drug to kill the virus and explain how the drug operates (complete with diagrams), and then to design and describe an experiment to test the drug - asking students to think and act like scientists.
This kind of testing would clearly pay for itself just from the patent rights to the anti-viral drugs designed by the high school test-takers. They must be worth billions!
Posts
45 comments:
for reasonably bright students, such tests *are* much more conducive to the development of critical thinking than typical tests... I never encountered anything like these sorts of tests till I did my phd, and preparing for them definitely developed those skills. Others in my cohort from outside the US had had similar tests throughout their school...
Slightly off topic, but of interest to readers:
http://www.nytimes.com/2010/09/07/health/views/07mind.html
In my opinion, you can run these kind of questions with just multiple choice. Compared to free-from written answers, this pushes up the objectivity of the grading (which with multiple choice is just as objective as the questions are clear), and there is not that much loss of scope for answering.
I got 800 verbal GREs and SATs. If the multiple choice questions on a *serious* multiple choice exam like the biochem GRE could be defeated, I would be able to defeat them. I did defeat large numbers of the multiple choice questions on the high-school National Latin Exam; my Latin was awful, but I could usually more-than-half trace the logic of how they were trying to trick me. Well, the biochem GRE is a different story, and it has plenty of questions where you look at some mock-up experimental data and then give a multiple choice answer about what could be inferred from those data. And no way, you could not trace out how they were trying to trick you. The same is true of multiple-choice o-chem tests I took in college, those things were seriously treacherous, full of the nastiest tricks, and if you couldn't just derive the right answer by frontal assault, you were toast.
Thus, if you have to ask how an antiviral drug should be designed, I would simply say, should it be designed like A., B., C., D., or E. And then I would ask about a second, separate trait of the supposed drug - again, should it be A., B., C., D., or E.
The chief limitation of the bochem GRE is not that it is multiple choice, but (from my perspective) that there is little payoff from doing tons of bio study on your own that is outside the normal curriculum. But there's basically not much of a reasonable way to fix this.
This NY Times article on learning mentions that testing appropriately can also be conducive to learning.
http://www.nytimes.com/2010/09/07/health/views/07mind.html?pagewanted=1&_r=3&ref=science
Better standardized tests can in fact promote more learning. But just like "buy low, sell high" the trick is in the execution.
Our lack of such tests must explain why we lag the world in drug development.
From Linda Darling-Hammond's article:
Although some states, such as high-scoring Connecticut, Maine, Vermont and Nebraska, have created assessments that resemble those in other countries, the requirements and costs of No Child have led an increasing number of states to abandon their challenging performance assessments for more simplistic machine-scored tests.
Gee, I'm racking my brain trying to figure out what these 3 states have in common. I wonder if Linda Darling-Hammond could figure it out for us...
OK but Steve, you're not understanding the nature of the War on Low Test Scores.
Like the the War on Terror and the War on Poverty, the War on Low Test Scores is not meant to be *won,* only *fought for as long as possibly, preferably forever*.
Educrats -- like Military-Industrial Complex tycoons and Poverty Pimps, don't want every student to be educated, any more than the MIC wants peace. They'd be out of a job if that happened.
So complain all you want about how simple it would be to raise test scores, or how impossible it is to raise them. It doesn't matter to the players involved. All that matters is that you write about it, and that your tax dollars fund it.
Forcing high school seniors to think creatively about complex and ill-defined empirical problems during high-stakes testing is an excellent idea. It should trim years off the time that graduate students in the physical sciences must take to earn their advanced degrees.
[I see that I now have to come up with an example to support my central thesis, or this essay will lose points. Er... designing novel antiviral drugs?]
OFF TOPIC:
The podcast episode this week (on iTunes) of NPR's This American Life, titled "Family Physics," includes a fascinating lead story that follows a suspiciously dark-looking son of an Italian-American family, who turns out to have been fathered by the black football-player classmate of his mom's, with whom she was cheating on her dad at the time. Years of denial ensued. There is a happy ending.
It seems to touch on a lot of the principal themes of the Steveosphere.
http://www.thisamericanlife.org/radio-archives/episode/214/family-physics
Strangely, "critical thinking" never seems to involve any critical analysis of Leftist beliefs, thoughts, attitudes, or assumptions. Criticize whatever you want except the prevailing Leftist orthodoxy!
Perhaps the pentagon to get some questions added, like:
Design a weapon of mass destruction, determine its effectiveness and how it would be tested.
Design a virus that will selectively attack the Chinese, ...
The uses of such critical thinking tests are endless.
for reasonably bright students, such tests *are* much more conducive to the development of critical thinking than typical tests... I never encountered anything like these sorts of tests till I did my phd, and preparing for them definitely developed those skills. Others in my cohort from outside the US had had similar tests throughout their school...
Sigh, you have missed the point. But then, if it took you until your PhD to develop critical thinking skills ...
Was it in the humanities by any chance?
Such complex tests would are open to interpretive grading (partial credit), would introduce a lot noise and be useless for the many students, especially our growing underclass.
It would be useful to the elite students who could handle it. Perhaps the best would be to keep the baseline existing standardized tests and use them as a filter to identify the smart fraction that has the interest, ability and potential to benefit from such complex tests.
Can anyone give the names of or point to samples of such tests online?
I would love to see these AP-type test administered across the entire spectrum of school-age kids in the US.
It won't ever, ever happen, because enough of the people in charge know that the results would never be acceptable.
Like you said before, Steve: two words.
Such tests only work if you have tracked your students ahead of time. If the test is designed to be taken by every single student there is simply no point and the correction overhead is ridiculous.
However, if the tests were used to help potentially determine phd track students at the high school level, then there could be some benefits.
It sounds however like people who propose such tests think somehow they will "close the gap" through obfuscation.
"Gap closers" hate multiple choice tests. Occam's razor suggests the real reason they hate multiple choice tests. It's the same reason they hate verbal analogies.
This kind of testing would clearly pay for itself just from the patent rights to the anti-viral drugs designed by the high school test-takers. They must be worth billions!
Sometimes the snark can cross a certain line and become really depressing to contemplate: These people are insane.
I think this was a story arc from Rocky and Bullwinkle, "Critical Thinking or This Is Not A Test" in which Boris and Natasha administer a test so inane that it puts innocent American schoolchildren into a state of suspended animation. Fearless Leader plans to take over the world once the children can no longer invent any weapons to oppose him. Our intrepid duo, Rocky and Bullwinkle, break the spell and foil the plot by broadcasting "If I Only Had A Brain Drain" through the schools' intercom system (invented by second-graders). At the end, Boris has to stand in the corner with a dunce cap that has a picture of a moose on it.
Or do I have that episode mixed up with Missouri Mish Mash and the race to obtain the Kerwood Derby?
Somewhere in all this we need to face the fact that Americans in general despise education, and the educated.
This quote does show what nitwits ed school people are. Obviously designing a drug is an enormous project that requires teams of people with deep knowledge of many fields. If you ask high school biology students to do this as an exercise, they will be doing it at a toy level, and their answers will of course be worthless. While there could be cases in which it would be a stimulating exercise, identifying such cases would be a judgment call requiring the understanding of an expert. The idea that such an exercise is sure to be a better use of their time than training up basic skills and knowledge is an idea that would appeal only to an ed-school moron who has no inkling of the depth and complexity of those basic skills and knowledge.
This is an odd way to narrow the racial achievement gap, unless the plan is to lower all races' average scores to single-digit territory.
Scholastic test results are like medical opinions. There is an excellent chance that a not very sophisticated medical practitioner will figure out your problem, which is probably common. Likewise, the mental abilities of most kids can be accurately measured by unimaginative fill-in-the-bubble tests concocted by not very sophisticated test makers.
Sometimes a really bright kid might be mistaken for a dunce. Sometimes throat cancer is mistaken for a mild infection. It just seems to me that only incongruous results need to be re-checked. The kid with low test scores writes beautiful poems or can fold paper into impossibly intricate shapes or can prove the Pythagorean theorem without having been previously shown how. Likewise in medicine, the patient's persistent cough doesn't go away after three weeks. In either case, further evaluation by specialists is in order.
But in the main, Forrest Gump is right: stupid is as stupid does. And stupid kids act and talk stupidly in school, on public transit, in the streets, in parks, and at the mall. Conversely, smart people spontaneously do smart and creative things that reflect their natural abilities. They also tend to engage in constructive social behaviors and converse in a sophisticated way. Repetitive testing just belabors the obvious.
Better scholastic tests won't make kids smarter any more than better medical tests will make people sicker.
This seems like a naive suggestion to me. The biggest controversy in testing as every reader of this blog must know is the racial performance gap. You should anticipate that any test format will be challenged immediately as racist as soon as it becomes obvious that blacks score lower than whites or East Asians.
The political situation in public testing is only a little more sensitive than that in normal college classroom testing. There is the need to gauge the knowledge of the students of course but just as important is the need to defend the teacher against charges of unfairness. Maybe it shouldn't be that way but it is. It most certainly is.
That Australian high school and the Chinese high schools can get away with testing that involves considerable subjective ex post facto answer judgement because their classrooms have few minorities and the students are not accustomed to challenging the authority of the teacher. That's not the way it is in these United States.
After years as a programmer and manager I taught Microsoft classes at a California College about a decade ago. I had taught many of these classes commercially. The commercial classes were about $1,000 for three seven hour days and had no tests. The idea of the classes was to prepare the students to pass the machine administered multiple choice test that were required for a Microsoft certification. So while there were no tests given in the commercial classroom the students were very much focused on testing.
For example, I taught SQL Server database design as a semester long
class. The same material that was taught in 21 hours commercially was spread over 54 hours in the college format. I had to devise my own tests.
Obviously I modeled my tests on the official Microsoft tests for which this class was preparatory. But less obviously I prepared my own defense. I anticipated that the students would complain to the administration as they did for all the other computer science teachers.
The senior instructor there spent much of her time in hearings defending herself against student complaints of unfairness. This usually meant they hadn't gotten an A. Her response had been to dumb down the tests. She gave all her tests as "take home exams" and she gave no grade lower than a B.
She had never had a student in any class she had taught pass a single Microsoft exam. My approach was different. I typically got rid of over a third of my students. Some I flunked but most just dropped out before their records would be effected. None of them had had a hope of making it in the marketplace - they were just too dumb. My service was to demonstrate this to them. BTW many if not most of my students passed the relevant Microsoft tests. They lost their fear of those tests. The word got out that if you could pass my tests, the Microsoft tests were no threat.
So how was I able to give appropriately tough tests in an environment where the students routinely got the teachers in trouble? I gave only multiple choice tests with unambiguous correct answers clearly written in the text. I taught a lot of material that wasn't in the book (I had a lot of hours to fill) but I never tested on it. I didn't want to get in any arguments involving class notes or student memories.
Before the final exam or midterm I wrote the grade distribution on the blackboard. Then I gave them the test. Afterwards we graded the tests publicly. I read the question and gave the page number in the text where the correct answer appeared. I wrote each student's score on the board with a disguised name. I then went to the board and fitted the obtained scores into the apriori grade distribution.
I rarely got any complaints. If I did get someone who wanted an A. I would ask them to choose which student who they would lower to a B so as to make room for them among the As. That shut them up.
All these criticisms of object tests ignore their defensive utility.
Albertosaurus
>I never encountered anything like these sorts of tests till I did my phd<
There was a reason for this.
Well in Russia which had up to 90s fairly good scientific education there was no tests. Exams consisted of limited number of problems (3-10 or so for physics and math) .
Those problems were relatively complex and required basic analysis skills as well as all basic skills which are typically tested in american tests (the solution for the problem was multistep and they were designed in such way to require use various basic skills )
So I think that system has merit. But NAM would be too dumb for this
People to whom the basics of a subject have become second nature understandably find "just the basics" boring and imagine that little kids and teenyboppers have the same intellectual context and interests that they do. It just ain't so. (The one-in-a-million kid who is intellectually ten or fifteen years ahead of the average kid seeks out and finds stimulating material on his own - ideally with the help of adults, but not crucially - provided, however, that he is not impeded from pursuing his interests. But this kind of kid is, again, a rare exception.)
The funniest example of this phenomenon I know concerns the famed orchestra conductor Leonard Bernstein - a very experienced and advanced musician, to say the least. He was giving a master class to budding conductors, young students, one of whom stopped waving baton in the middle of a Brahms symphony movement in order to ask an elementary question about conducting technique. "What do I do here?" asked this confused beginner. Instead of engaging with this student and giving a specific, helpful answer, Lennie roared: "Here you take us to heaven!"
This is the kind of "beneficial" education Ph.ds and their cohorts daydream about using to transform American society and maybe even the world. A lot of hot air about "multivalent options" and "interpretative thinking." Everyone has to have a hobby, I suppose, but can't they pick one less harmful?
our current education regime is designed
to enable the full potential of all peoples!!!
not just the select few
the problem is that one can't use science or statsitcs to idedntify these people..
yes one can!!!
but.... we believe before we give up on these kids that we are absolutely certain that they do not have the potential.
First thing we do is dynamite all the Ed Schools. Then we have a big communal piss on the grave of whoever dreamed up "education degrees." Educators are cargo cultists who think they can wish outcomes into existence.
Anyhoo, if the tests are actually devoted to applying the scientific method and performing logical or mathematical analysis, that's great but I predict the achievement gap widens significantly. Rote mathematics, grammar and logic are how students acquire the skills for critical thinking. If we throw everybody into the deep end, who do they think will end up swimming?
Even worse, I bet there's this big push for oral exams where a panel of suitably diverse edumacators judges students on the recitation of politically correct tropes in response to whatever chimera is fashionable at the moment: climate change, HIV/AIDS, old growth forests, etc.
Then we can unleash all these Critically Thinking young geniuses on antibiotic-resistant superbugs, rising ambient levels of mercury, and cancer metastasis. Yep.
Correct me if I'm wrong, but won't NAMs do worse on such tests?
Multiple choice questions, typically pencil-in bubbles for answers, are popular because they can be marked by scanning machines. Cheap, in other words.
Syriana got an 86% rating from the Rotten Tomatoes Top Critics. That's a boffo rating. The ordinary critics still gave it 72% - also a very positive score typically achieved only by big box office hits.
Meanwhile over two and a half million NetFlix viewers gave it only 3.2 stars - a rather poor rating. I generally skip movies that only get 3.2 stars.
So you get your choice - the opinion of the roughly 35 movie watchers who are the elite media critics or the millions who aren't.
Albertosaurus
Do they really want Americans to be smarter? They just want them to have better test results and corporate productivity - without the flip side of dropping out of society and becoming beardy bookstore owners and street poets.
The problem is that our system is geared toward the bottom half of students. No one will tolerate failure so the bar is lowered. If a teacher fails his students, then his job is at risk. It is that simple. It will not change until the system collapses under an untenable cost structure.
I know many will disagree with this position, but the reason I prefer multiple choice test is because liberal-test-graders cannot unfairly penalize conservative (especially white) students, and inflate the grades of preferred minorities.
If you allow essay-style questions, lefty teachers can grade tests with extreme bias.
These are liberals we are dealing with here, and if you blink they will cheat you.
OT
The REAL stuff white people like:
http://blog.okcupid.com/index.php/the-real-stuff-white-people-like/
Reminds me of authentic assessment. If nothing else, aligning the curriculum to these tests could make school less boring.
Before the final exam or midterm I wrote the grade distribution on the blackboard.
So what did the curve look like [mathematically speaking]?
"Strangely, 'critical thinking' never seems to involve any critical analysis of Leftist beliefs, thoughts, attitudes, or assumptions."
In my experience, most people who trumpet "critical thinking" are anything but critical thinkers.
But that's what's so insidious about Leftism. Not only does it make people believe silly things, it convinces people that they are highly enlightened and rational for believing such silliness.
Most people are not creative. And even creative people aren't capable of much until they have learned a great deal. These "educators" are insane if they think that high-school pupils will be able to answer questions that would stump a PhD candidate in virology.
I know many will disagree with this position, but the reason I prefer multiple choice test is because liberal-test-graders cannot unfairly penalize conservative (especially white) students, and inflate the grades of preferred minorities. If you allow essay-style questions, lefty teachers can grade tests with extreme bias. These are liberals we are dealing with here, and if you blink they will cheat you.
If we had comment moderation with this software package, then this would be a +∞.
This doesn't seem inherently silly to me. You're not looking for a useful answer, you're looking for the ability to apply what the kids have learned and new information in an appropriate way.
Test scores cause political/social problems in the US because their results don't conform to political necessity. Scores differ across races in ways that cause all kinds of offense and upset. When reported for schools or teachers, they undermine the position of well-protected interests. They expose the nature of affirmative action programs that were intended to remain obscured in careful language.
This is a property of data about the world: it doesn't know or care about political parties or coalitions of keeping the powerful happy or what would be convenient for anyone. That's one reason the most devoted partisans are often proud of their ignorance. You can't serve two masters, right? Either you can be devoted to saying truth or devoted to saying whatever will best serve your party or society or church or race or whatever.
One of the worst things about our society is that so many people at the top of government and media and academia and think tanks, have gotten an kept their jobs largely by bwing willing to stick to the party line in preference to the truth as they best understand it. That's going to f-ck us over in the end.
Anonymous said..."These are liberals we are dealing with here, and if you blink they will cheat you."
Yes, and if you don't blink, they will try to blindfold you.
I would not be surprised if within my lifetime, they progress (pun intended) to putting your eyes out.
I guess all this testing is supposed to be a squiffy metric of kids future value in the economy, but I've never seen any breakdown of GDP by race, yet. We all know about the academic achievement gap, but what's the economic productivity gap? It would seem this is not the same thing as median salary, since a lot of blacks are well paid government employees and affirmative action contractors. And how accurately could such a breakdown of GDP by race be done?
For example, how do you judge the value of Hispanic labor in the construction industry, where they are prevalent? Any duffer who walks the aisles of Home Depot realizes how dirt simple building construction and repair is with modern materials, methods, and machines. Basically, if you have an opposable thumb (and can pass a sobriety test) you can do any job a Hispanic can do.
These are liberals we are dealing with here, and if you blink they will cheat you.
They will allow black and Jewish student to molest 12-year-old white gentile girls.
They will intervene to stop fights the moment a white gentile boy starts to defend himself, and not a moment before.
They will tell you Abraham Lincoln (white gentile) had nothing to do with the 13th Amendment, or that the 13th Amendment didn't free the slaves in the border states.
They will tell you America put the Shah of Iran on the throne.
The will tell you lynching only affected blacks.
The will tell you the most common type of firearm-related death is firearm accidents.
They will tell you dogs' mouths are cleaner than humans' mouths.
Etc.
Signed,
A Different Anonymous
But what's 6 x 7?
You can't memorize it, or even the procedure for calculating it, because MEMORIZATION IS EEEEEEEEVIL.
Thanks, retards. Thanks for misunderstanding education. Thanks for BEING THE REASON education in this country sucks.
Post a Comment