October 30, 2013

David Coleman is redesigning the SAT. Does he know what he's doing?

In 1994, Richard Herrnstein and Charles Murray pointed out that the SAT was such an effective college admission test that it was having profound effects on society as a whole (with Herrnstein more positive than Murray about the net impact).

Ever since, the College Board (the sponsor of the SAT -- the Educational Testing Service writes it up for the CB) has been trying to do something about that.

For example, one problem with the SAT back in 1994 was that it was too easy to get an 800 on the Math part of the test. When I was at Rice U., it was common to hear the better STEM majors say things like, "Well, sure I got an 800 on the Math SAT, but I'm not a real 800 like Joe or Bill are." In contrast, back then the SAT verbal test (on which only double digits scored 800 annually) was, as Henry Harpending has pointed out, perhaps the best high end IQ test in the world.

So, in 1995, the College Board changed the scoring. But not to make the Math scoring more like the Verbal, but vice-versa.

In the early 21st Century, the chancellor of the College Board's biggest customer, the University of California, demanded changes in the SAT. So the analogies questions were thrown out because they were like, well, you know, they were kind of like ... well, I don't know what they were like, but they were annoying. And a Writing section was added, featuring a large essay which has proven over the last decade to be a probing test of how legible and swift is one's penmanship (or pencilmanship, in this case), although perhaps not much else.

The old head of the College Board, former West Virginia governor Gaston Caperton, resigned last year around the time of his third divorce to pursue his fortune on Wall Street. The new head is David Coleman, who is also the author of the Common Core standards for K-12. He soon announced he was going to revamp the SAT by 2015.

In other words, the educational establishment has decided to bet the country, at least the K-16 part, on one guy.

Coleman sounds like not the worst choice for The Guy. Ideologically, I see a lot of similarity between Coleman and E.D. Hirsch, of the Core Knowledge Foundation.

Hirsch was a top academic literary critic who got involved in trying to help local schoolchildren learn. He decided that the reason average children's reading comprehension was so bad was because they didn't know anything -- they were constantly stumbling upon the text's references to events or concepts that they didn't know anything about, and were thus losing the thread of what the author was saying.

So, Hirsch argued that schools should teach less poetry and fiction and instead use more informative nonfiction texts that would help students build up their core of common knowledge. Hirsch published long lists of the facts that he felt came up the most often as the assumed background knowledge of the reader in the better sort of writing.

This is not a bad idea, although it's not as much of a panacea as Hirsch thought it was.

Politically, Hirsch came out of the liberal Jewish mainstream of, say, 1960. But by the late 1980s poor Hirsch was denounced as a flaming rightwing advocate of Dead White European Males. (Here's a 1990 Christopher Hitchens article on Hirsch.)

Perhaps because of all the vitriol spewed at Hirsch's Core Knowledge campaign, there have been many denials that Coleman's Common Core has the slightest to do with Hirsch. But, my impression is that they are pretty similar: coming out of the right of center side of the liberal Jewish mainstream.

Coleman's philosophy basically appears to be the same as every other educational reformer: Be Like Me. Coleman is a bright, cultured former McKinsey consultant, and thus his Common Core write-up would be a good outline for the home schooling of David Coleman Jr.

Unfortunately, I haven't seen much evidence that he knows much at all about, say, psychometrics. 

Details were not terribly forthcoming about his New SAT, but Inside Higher Ed has now reported:
Hints on the New SAT 
September 20, 2013

Scott Jaschik 
TORONTO -- David Coleman, president of the College Board, was fairly general here Thursday in describing the changes coming to the SAT. His theme was that the new SAT would be more closely tied to high school and college curriculums and less coachable than is the current version. 

Aren't those tradeoffs?

I've long argued that the rise in test prepping should lead to more weight in college admissions being given to Advanced Placement test because if you are going to test prep for years, you might as well learn something, like chemistry or French literature. So, maybe this is brilliant ju-jitsu on Coleman's part by applying my AP idea to the SAT.

Or maybe he just doesn't get it.

The point of having standardized college admission tests is to fill in the obvious shortcoming of just using high school GPA. Making the SAT more like high school seems pointless. If we are all supposed to be into "critical thinking," maybe we should make high school more like the old SAT?

Coleman presumably thinks his Common Core is going to revolutionize American K-12 school by making it much more rigorous, so, assuming that will happen real soon now, well, then of course we should change the SAT in 2015 to be more like his Common Core.

More likely, the Common Core will just be another K-12 fad, like the Small Learning Communities boondoggle that Bill Gates wasted $2 b-b-b-billion on in the 2000s, but Coleman will manage to do some genuine damage to the SAT.
But in his remarks at the annual meeting of the National Association for College Admission Counseling, he seemed most comfortable in predicting big changes for the essay portion of the test. He admitted that there are serious flaws in the test now (namely that test-takers are not rewarded for being factual and that in fact a common strategy for students to do well is to ignore the facts). Further, he floated a specific idea for changing the essay. (He was not specific in describing most other changes.) 
Coleman asked attendees to imagine replacing the current essay (in which students respond to a prompt) with a writing assignment in which students analyze information. "What if you were analyzing a source" in a short essay "and using evidence from that source?" he asked. Might such an essay prompt "celebrate analytic writing?"

On the AP history tests, they have data-based essay questions where they give you eight or ten brief documents to use in answering a question. Extending that to the SAT isn't the worst idea.

Of course, that's pretty much how the Fire Department of New York hiring test worked: they gave you a few paragraphs of fire fighting instructional material -- e.g., what kind of chain saw blades to use on steel doors and which kind on wooden doors so you wouldn't accidentally rip your face off when the chainsaw bucked -- then asked questions based on the presented material. Test-takers could do well either through good reading comprehension while taking the test or by learning the firefighting material beforehand. As a potential fire victim, that seems pretty okay to me, but the whole test was junked by a federal judge in 2009 because blacks and Hispanics didn't score as well on it.

But, AP essay questions are about the AP subjects such as European History, while the SAT is supposed to be a more general test. One of its functions is to find the "diamonds in the rough" who haven't necessarily had every educational advantage.

The essential problem with essay questions are small sample sizes, on multiple dimensions, which leads to scores being less reliable. In other words, the endlessly denounced multiple choice question has some advantages, such as that you can ask a lot of them, thus giving a reasonable sample size.
He said that he was bothered by the status quo, where the essay "does not grade you on the correctness of what you write." And he described feeling somewhat ashamed when he spoke to a friend who taught SAT test-prep in Hong Kong when she told him how she helped students who asked about how to come up with examples to back up their points in their essays on the SAT. "You make them up," she said. 
In the decade since the College Board announced and then started giving the essay portion of the SAT, reaction has been decidedly mixed. College Board officials predicted that the addition of the writing test would send a message to high schools to take writing seriously. 
But almost from the start, many writing experts questioned the kind of writing the College Board was promoting, saying that it emphasized using a few impressive words and paying little attention to facts or logic. A writing professor at the Massachusetts Institute of Technology famously coached students on how to write laughably untrue statements in SAT essays that would receive good grades from the College Board. Many colleges have pointedly not used the writing scores of the SAT. 
Until Coleman became president of the College Board last year, however, officials there have defended the essay.
Coleman didn't focus on the issue in his prepared remarks, but did so when urged by Jon Reider of San Francisco University High School to "get rid of the essay." 
Reider said that the essay test has forced counselors like him to tell students to do things that are educationally unsound -- just so they can get a decent score on the essay. 
He said that when smart students who are good writers talk to him about disappointing scores on the SAT writing test, he tells them that they are doing poorly because "you are a good English student and you've been taught to stop and think before you write, but that's not an asset on this test."

Sheer amount of verbiage emitted into the test booklet correlates with essay scores.
... It's also wrong, Reider said, for the College Board to promote writing without any relationship to facts. "I challenge anyone in the room: Have you ever sat down for professional purposes to write about a question you have never seen before, in which the accuracy of what you write is totally and utterly meaningless?" 
Coleman's response to Reider was that "you've got a real point. You really do." 
But Coleman cautioned that eliminating the essay, instead of fixing it, might send the wrong "message."

Fixing essay grading sounds extremely expensive. Are they really going to pay graders to stop grading and go look things up on Wikipedia to see if the student knows what he's talking about?
He did not dispute any of Reider's criticism, but said that "there may be alternatives beyond dropping it." 
Coleman first announced plans to redesign the SAT in February. 
Most of his comments here were consistent with the goals he announced previously for the shift -- that he wants to more closely align the SAT with the high school curriculum, for example.

The SAT college admission test was long dominant over its Iowa competitor, the ACT, but recently the ACT has surpassed the SAT in sheer numbers. The ACT positions itself (I don't know how accurately) as the meat and potatoes test that's an achievement test of what you learned in high school, not some fancy-pantsy "aptitude" test.
On Thursday, he took swipes at how the SAT has evolved, noting that the test has created a huge test-prep industry and that students aren't necessarily being encouraged to focus on important learning. 
He noted that when someone says a person has just used "an SAT word," the idea isn't that the person has shown eloquence or clarity but that "they have used a word they would never use again." 
Coleman also talked about how a better SAT shouldn't be something one has to learn at a test-prep service. The revised SAT "shouldn't be a sudden departure" from what students learn in class, he said. A student's plan to take the SAT "should inspire excellence in the classroom." 
One questioner (and a number of people in comments after the presentation) suggested that Coleman was describing a test more like the ACT, which has long claimed that it is more curricularly based than the SAT. And the ACT has gained substantially against the SAT in recent years in market share. Amid all these discussions, one questioner here, warning the audience to prepare to be shocked, said that he liked the current SAT, and feared the new SAT might be something like New Coke.

Ouch.
Coleman responded that "I'm not interested in a Coke/Pepsi kind of debate here."

You may not find the New Coke fiasco of 1985 interesting, Mr. Coleman, but fiascos are interested in finding you.
And he asserted that what he was talking about was a new kind of test, one that would promote educational values. 

This could be ugly.

Presumably, there are some good psychometricians in the backrooms at ETS whose jobs are mostly to keep the politicans running the College Board from messing up too badly.
Another question -- from someone who used to work in admissions at an elite university -- highlighted how challenging that may be. The questioner said that his instructions at the university -- straight from the president's office -- were to increase average SAT scores and to increase minority enrollments. He said that he found it impossible to do both. 

27 comments:

  1. Dumbing down the SAT (again, more) will reduce disparate impact (good for race-and-sex-obsessed college admins) and increase the influence of "connections" (good for corrupt college admins). What's not to like?

    Despite the last round of reforms in Richard Atkinson's time, the SAT has continued to embarrass college admins by revealing their biases against smarter candidates. They hate that and want Coleman to fix the problem by making the SAT less probative.

    ReplyDelete
  2. I wrote about the changing nature of the SAT; can't remember if you mentioned it: College Admissions, Race, and Unintended Consequences--is the huge improvement in Asian performance simply due to increased Chinese, Korean, and Indian immigration or has the test gotten easier, are they cheating in larger numbers, or some combination? I spent the month of October trying to figure out how to write about it, and my ass is still kicked.

    Also, a primer on the difference between the ACT and SAT.

    The essay is actually a leveler. It's very hard to get a 6, and it's fairly easy for a poor writer to get a 3. It's very satisfying to coach weak writers in it; you can actually see them improve. Coaching strong writers is depressing.

    My big beef with the AP is the restricted range of scores and its scoring priorities, invented when everyone was whining about girls doing poorly--god, isn't that a while ago? A kid can turn in a blank test and get a 1. Perfect scores are given to kids who fail the multiple choice but write two good essays.

    I'm starting to become incredibly distrustful of Asian test scores--not all of them, but enough. Too much reading this month, coupled with a white girl who told me she got a 790 on the Math 2c when she went to an Asian academy (not mine) and paid $400 for a series of programs. She did no math, just plugged in the numbers. She didn't feel very good about it, but on her own, she only got a 690 (which is a great score), and every Asian kid in her school had an 800. I used to think that was just talk. I don't any more.

    Anyway. Coleman's an idiot. A smart idiot, but an idiot. It's all a waste. But Atkinson hurt the SAT more with the last series of changes. The ACT is still a good test, mostly because Asians don't spend much time gaming it. I think

    ReplyDelete
  3. "Dumbing down the SAT (again, more) will reduce disparate impact"

    Nope. Percentile rankings will remain basically unchanged, save for some bunching/tying at the top.

    ReplyDelete
  4. I wish SAT exams would be consisted of 50% technical question and 50% about life in general because that is the reality of life after college. You can't just live on graphic charts and numbers, one way or another a problem will occur in our life that wasn't asked on your SAT.

    ReplyDelete
  5. Actually, I'm for the approach Coleman appears to be using. Even went on record saying so. http://www.aei.org/article/society-and-culture/abolish-the-sat/. The data finally prevailed and forced me to accept that a lot of my thinking about the differential effects of the SAT and the achievement tests doesn't pan out in practice (i.e., the SAT doesn't identify diamonds in the rough any better than the achievement tests. Charles M.

    ReplyDelete
  6. I'm inclined to think Coleman knows what he's doing. Which is to say, he seems to agree with me. http://american.com/archive/2007/july-august-magazine-contents/abolish-the-sat

    ReplyDelete
  7. "He admitted that there are serious flaws in the test now (namely that test-takers are not rewarded for being factual and that in fact a common strategy for students to do well is to ignore the facts)."

    The SAT, as now constituted, sounds like an excellent filter for admission into our nation's yammering elites.

    ReplyDelete
  8. Just re-center the re-centered scoring. Why go to all this trouble devising a new test.
    What we really need is something equivalent to a GED for college. Just a simple test in 127 languages, lets say just three questions, your name, date of birth, social security number. Upon passing the college GED you are then eligible for a student loan, lets say in the amount of 1 million dollars, which, of course, will immediately be forgiven.

    ReplyDelete
  9. Well, if we must have Jews redesigning our educational system, we can at least give thanks that they are from the right of center side of the liberal Jewish mainstream!

    ReplyDelete
  10. @ Education Realist:

    I read your primer on SAT/ACT comparison; you seem to favor the SAT overall (my impression, but I haven’t read through your blog extensively, although enjoy thoroughly). I would think that, in a fast-paced information economy, we would want kids who can read, comprehend, analyze and decide within time constraints, as we do in the real world. This would lean one toward favoring the ACT over the SAT. Full disclosure: I took the ACT in 1982 and scored a 30 overall (I was born/grew up in the Midwest, though I’m an East Coaster now). Perhaps we should develop a high-school method to track which kids will enjoy more success cognitively taking the ACT (time-constrained, less abstract reasoning) or the SAT (more abstract reasoning, less time constraints)? Both modes are necessary for future American thinkers and leaders. They’re complimentary ways of thinking about/solving problems, not necessarily mutually exclusive.

    ReplyDelete
  11. For example, one problem with the SAT back in 1994 was that it was too easy to get an 800 on the Math part of the test. When I was at Rice U., it was common to hear the better STEM majors say things like, "Well, sure I got an 800 on the Math SAT, but I'm not a real 800 like Joe or Bill are." In contrast, back then the SAT verbal test (on which only double digits scored 800 annually) was, as Henry Harpending has pointed out, perhaps the best high end IQ test in the world.

    It would also appear to me that it was easier to score higher in the Math SAT than in the Verbal SAT.

    I didn't do an extensive survey of SAT scores in my high school class of ~160 [well before 1994], but I had definitely heard of more high SAT Math scores than SAT Verbal scores. Off the top of my head I can think of eight members in my class who scored above 750 on the SAT Math section. I do not recall hearing of so many high scorers on the SAT Verbal.

    From The Journal of Blacks in Higher Education.: we find that in the entire country 244 blacks scored 750 or above on the math SAT and 363 black students scored 750 or above on the verbal portion of the test.

    Which means that all the blacks in the country produced only 30 times the number of +750 SAT Math scores compared to a small high school. I do not know why that is so, but it is not good news.

    I am very skeptical of the efficacy of standardized testing for writing ability.

    ReplyDelete
  12. Henry Canaday10/31/13, 6:30 AM

    Have no fear. Help is on the way:

    "I Got Schooled: The Unlikely Story of How a Moonlighting Movie Maker Learned the Five Keys to Closing America’s Education Gap"

    By M. Night Shyamalan

    M. Night will be at the Local Lefty Bookstore in November. I will report.



    ReplyDelete
  13. The SAT and the ACT are both highly g loaded, despite efforts to make them less so. Since they are both effectively IQ tests, scores correlate with IQ and with each other. Once you convert ACT scores to their SAT equivalent (tables exist for this), a student taking the ACT is about as likely to get the same score on the SAT as he would be if he took the SAT two different times. Despite small differences at the margins which are of concern to some who are especially interested in such things, they are basically the same thing to the average person, just like Coke and Pepsi. (Even New Coke did well in objective blind tastings - the whole fiasco was a marketing fiasco, it had nothing to do with the merits of the product itself).

    The main impression that I have is that ACT questions are more straightforward while a lot of SAT questions involve trying to trick you into choosing the wrong answer. But the end result is basically the same - smart people do well, less smart people do less well, the same as on virtually any paper and pencil test man can device.

    Coleman may be a liberal Jew but he is not a leftist idiot and he knows (or will soon find out) that it is more or less impossible to do anything to get URMs to score better on the test without degrading it into something laughable. So he himself is not making noises in that direction at all, as much as the college admissions officers would love for him to do so to make their job easier. Instead they, with the help of the courts, will have to find other ways to thwart the will of their state's voters and Equal Protection clause of the Constitution.

    Of course the goal of making the test less "coachable" is also impossible. You can change what needs to be coached, but coaching of highly motivated test takers is going to help no matter what. I think people underestimate the willingness of Asians to work hard at perfecting their skills. I watch NHK (the Japanese version of Voice of America - it's better than the mindless drivel on most stations) and they just showed that in Japanese high schools there are calculator clubs (all the members are female) where they practice totaling long columns of numbers. They spend hundreds of hours practicing and the result is that they are lightning fast. It is simply inconceivable that American teens would do this but in Japan it is commonplace.


    As Steve says, might as well make them coach something useful (but any move to increase the amount of factual knowledge needed to take the test is only going to move those URM numbers in the wrong direction).

    ReplyDelete
  14. "So, in 1995, the College Board changed the scoring. But not to make the Math scoring more like the Verbal, but vice-versa."

    About a 1000 and change used to get 800 in maths, now about 12000 do. The verbal scores however declined in the 60s(about 60 points) and without demographical changes(white students scored same in maths in 1991 as they did in 1941 but far lower on verbal) and never raised up again to those high, but SAT got around to make up the deficiency by recentering. The blame goes to dumbing down of textbooks, i.e. it's not SAT that uses words that won't be used again, but that words that should be in common use aren't being used today.

    http://www.huffingtonpost.com/e-d-hirsch-jr/computer-analysis-overtur_b_4166092.html

    ReplyDelete
  15. Bunching at the top is fine with college admins.

    ReplyDelete
  16. a white girl who told me she got a 790 on the Math 2c when she went to an Asian academy (not mine) and paid $400 for a series of programs. She did no math, just plugged in the numbers.

    I don't see what's wrong with that. She didn't cheat. She obviously was doing "math" whether she knew it or not - she learned a technique or algorithm for producing the correct answers consistently. If that's not math, then what is? People do this all the time. When we do multiplication or long division, we don't give deep thought every time into the fundamental mathematics involved - we just run thru the rote steps and tables we have memorized for producing the right answer quickly. If we get the right answer every time, we are doing "math" whether we know it or not. The Asian academies do the same thing with more types of problems and its exactly the right thing to do if you are trying to get the right answer every time on a time constrained test. We do exactly the same thing when we teach little kids the multiplication table but at some point our teens become rebellious and would rather watch TV than put the time and hard work into memorization. That's our problem, not the Asians. Deep thinking about the philosophy of math is wonderful, but having the skills to produce the right answer quickly every time have even more real world value.



    ReplyDelete
  17. I've gone over to the dark side and just want everything to burn. So I want the SAT to be dumbded down as much as possible so as to destroy American universities.

    ReplyDelete
  18. a white girl who told me she got a 790 on the Math 2c when she went to an Asian academy (not mine) and paid $400 for a series of programs. She did no math, just plugged in the numbers. She didn't feel very good about it, but on her own, she only got a 690 (which is a great score), and every Asian kid in her school had an 800.

    You need clarify. Programs? How did the Asian academy facilitate any cheating?

    ReplyDelete
  19. Another question -- from someone who used to work in admissions at an elite university -- highlighted how challenging that may be. The questioner said that his instructions at the university -- straight from the president's office -- were to increase average SAT scores and to increase minority enrollments. He said that he found it impossible to do both.

    This is a tiny version of the future. Don't social fail from internal contradictions? This is one for the blank slaters.

    ReplyDelete
  20. "So the analogies questions were thrown because they were like, well, you know, they were kind of like ... well, I don't know what they were like, but they were annoying."

    Nice one! :-)

    ReplyDelete
  21. A modest question.

    Is there any connection between test prep and the Flynn Effect?

    No one mentions seems to care anymore about individual tests like the Stanford-Binet. I seem to be among the last generation of people who actually had my IQ tested by an individual. But even then group tests were beginning to dominate.

    I took a lot of general ability tests in high school but I don't remember their names or my scores. I must have taken the SAT as a HS senior, but I don't remember what I got.

    I do remember my GRE scores but that was in the sixties - before test prep was much of a topic. If today's college juniors were to take the same GRE that I took many of them (most?) would have also have taken a test prep course. So should I adjust my GREs up a couple points?

    Am I really smarter than I think I am? (Hardly seems possible).

    When I was twenty five those tests told me I was in the ninety ninth percentile. But there were a lot fewer people then - billions fewer. And many of those recently added have been those on the left hand side of the curve. It's great. I'm doing nothing and getting smarter compared to the rest of the population.

    That's a benefit of illegal immigration and the third world population explosion that you never mention.

    Albertosaurus


    Albertosaurus

    ReplyDelete
  22. "… his instructions at the university … were to increase average SAT scores and to increase minority enrollments. He said that he found it impossible to do both":pah!

    All he had to do was select his minority carefully.

    ReplyDelete
  23. What a pleasant surprise to see Charles Murray commenting here. I think I'll tuck my shirt in!

    ReplyDelete
  24. That's interesting about presenting untrue info on the SAT essays to get good scores. I teach a lot of test prep, mainly LSAT but I used to teach a lot of GRE. I independently came to the same conclusion and started teaching my GRE students to lie rampantly when writing their essays. My spiel went something like this: "The GRE is not grading for prose style, nor are they grading for factual accuracy. So, if not form or matter, what are they grading? Structure. They want to see that you can clearly present an opinion and support it with reasons buttressed with examples. It doesn't matter if the examples aren't accurate. Of course, in a real essay, you would cite sources but you would have lots of time and access to them. Here there is no time but we want to to do the same thing. The scores are highly correlated to length. And almost nothing else so write, write, write. The best way to pad out your essay is with detailed examples illustrating your position."
    "I suggest you pick a statement/politician, musician/artist and a scientist/inventor. If you are ambitious, you can also choose a philosopher. Don't choose overdone people like Gandhi, Hitler or MLK. Go for people that educated people *should* know about but don't really. My go-tos are Churchill, Mozart, Edison and Kierkegaard. Go to Wikipedia and memorize some dates and bio stuff. Odds are you can use one or more of them in your essay. And if the details are a bit off, who cares. They grade your essays in 2 minutes, and aren't experts or historians anyway and they aren't grading for factual accuracy.
    So, you have your people then just insert them into your paragraphs as needed. Don't just say 'Bach, for example, died in obscurity.' rather, say "J.S. Bach, both in the winter of 1687, although a brilliant and hard-working organist for royalty and composer of thousands cantatas, fugues and other works, faded into obscurity, and eventually an impoverished death."
    "Keep your thesis simple and clear (not long). Make your reasoning transparent (not long), use "structure" words like "nevertheless", "although" "Secondly" etc. Use examples to lengthen your essay. "
    So, I don't encourage them to like, but rather to not be too concerned with details. In normal writing, if you don't remember a date, you get vague (last half of 19th century) or look up the exact date. Here, you don't have the latter option and the former option doesn't allow for length. So, make it up.

    ReplyDelete
  25. "you seem to favor the SAT overall"

    No, not at all. I prefer the ACT.

    I actually read Charles Murray's article when it came out and disagreed with it for a couple reasons.

    First, the Subject tests are far more difficult than the SAT. The research involved the two tests that were considerably easier (Math 1c and Writing). Those were the two that had predictive value.The third test, the harder one, had less predictive value.

    The two easier tests have been eliminated or just aren't used anymore. The Math 1C is gone--most schools won't accept it. The English test is now the Writing section of the SAT--exactly the same test, just shorter with a different type of writing prompt and a little more time.

    So the Subject tests are incredibly hard. Moreover, they are only taken by the top half of the ability curve, unlike the SAT. We don't really know how the tests will scale to the lower half. But they all require specific knowledge.

    Doing well on the subject tests is a good indicator of ability. But doing poorly on them doesn't mean you aren't college ready. It's quite possible for a student to have 500+ sections on the SAT, which is more than adequate for college as defined today, but get low scores on all the Subject tests (lower than 450, possibly lower than 400), simply because he or she doesn't have the domain specific knowledge.

    This creates a huge problem for colleges that use the SATs as placement tests. They'd have to start giving placement tests themselves, at significant added expense and then *they'd* be the ones getting the complaints about racist tests.

    Ain't going to happen and honestly, until we start limiting college to the top 30%, we shouldn't let it happen. The SAT is an egalitarian test. It's not really needed to identify diamonds in the rough, but it separates the functional from the illiterate.

    I think it should be harder, and I have no problem with domain specific tests as well. But we need a general knowledge testlike the SAT or ACT.

    ReplyDelete
  26. When one is being analytical, one is thinking about some natural process or the nature of some entity. An analytical thinking process requires the use of some established facts.

    When thinking about something, is it better to start from a blank page of no knowledge, or is it better to use the prior thinking of the greatest minds that humanity has ever produced? When thinking about the movement of objects in space, isn’t it better to start with Newton?

    The starting point of analytical thinking must begin with the standard knowledge of the subject at hand.

    Knowing the standard is what should be tested. All knowledge comes down to “words.” Knowing the meaning of words best predicts the intellectual capacity to get things done.

    Well constructed multiple choice questions about the meaning of individual words, is the best way to measure the capacity of an individual person to actually get things done in a culture.

    Multiple choice questions are often answered correctly, not by not actually knowing the answer, but by a process of eliminating the other choices. This intellectual capability shows useful practical analytical thinking, that uses the best knowledge available to a culture and a capacity to think.

    ReplyDelete
  27. Uh, that dateline is Toronto. Not Toronto, Ohio, either. Why is the head of the SAT there at all? Are Canadians now using these tests, too?

    In 1984 I took a summer language course and one classmate was a Canadian academic. He was completely baffled by the US obsession with standardized testing. I asked him how his country's universities selected their students. He said, "Your marks!" That speaks to the homogeneity of Canada 30 years ago. Treating all US schools equally wasn't feasible 60 years ago, let alone 30.

    By the way, this fellow had a bizarre pronunciation of the vowel in the word "marks". I've never heard it anywhere but in people from the Ottawa Valley, people from the Massachusetts Berkshires, and people imitating pirates. Anyone familiar with it?

    ReplyDelete

Comments are moderated, at whim.