Yesterday, David Coleman of the College Board announced a new SAT that will go into use in early 2016. Some of the ideas sound pretty good, some not so hot, but my big question is: Is Coleman just making this up as he goes along? Or have they actually tested these planned changes and found they work as hoped?
The usual practice with the SAT is to slip experimental questions into real SATs. The unvalidated new questions don't count for the students' scores, but ETS checks to make sure they aren't worse than the old questions. I don't see much evidence that this has been done with Coleman's reforms yet.
Ready Aim Fire or Fire Ready Aim?
The usual practice with the SAT is to slip experimental questions into real SATs. The unvalidated new questions don't count for the students' scores, but ETS checks to make sure they aren't worse than the old questions. I don't see much evidence that this has been done with Coleman's reforms yet.
Ready Aim Fire or Fire Ready Aim?
From the College Board website:
Research Base
A fuller review of the evidentiary backbone for the redesigned SAT will be available on April 16, 2014. ...
Many institutions have devoted considerable resources to developing the skills of source analysis and evidence use in their students.
Please refer to research by these leading universities:
But these links don't go to psychometric studies, just to pages of general advice for undergraduates on how to write college papers.
Test Design
Work has begun to build strong evidence for validity by testing item types, exam questions, essay prompts, and test forms over time. The College Board will review student performance metrics to ensure that exam questions and test forms are measuring the knowledge and skills they are intended to measure.
Pilot Predictive Validity Study
We will launch a pilot study of the predictive validity of the SAT with the partnership of colleges and universities.
"We will launch" -- So they haven't actually launched even a "pilot predictive validity study" even though they've announced what they're going to do.
This study will allow us to gather early evidence of the validity of the SAT for predicting college performance. In particular, we will examine the relationship between high school grade point average (HSGPA) and SAT, and the incremental prediction of the SAT over HSGPA in predicting college English and math grades as well as first-year GPA. As sample sizes permit, we will also examine differences in predictive validity by student subgroups.
Disparate impact much?
National Predictive Validity Study
After the introduction of the redesigned SAT, we will begin an ongoing process of documenting the relationship of performance on the assessment to outcomes of interest. Beginning with the first cohort of students to have primarily taken the redesigned SAT (the entering college class of fall 2017), we will launch a longitudinal national SAT Validity Study in partnership with colleges and universities to examine the relationship between SAT scores and college outcomes such as GPA, course grades, persistence, and completion. We will conduct extensive validity analyses by subgroup. The timeframe for this work will be:
Before summer 2018: Institutions sign up to participate.
Fall 2018: Data file is received by the College Board, including completed data-sharing agreements.
2019: Validity study is complete and distributed.
Those interested in participating should contact nsatvastudy@collegeboard.org.
Concordance Tables
Concordance tables will compare old and new exam scores to enable admission offices to have longitudinal consistency in their behavioral models and to evaluate applicants that have taken the different exams.
So, the College Board won't be able to tell you whether a 500 on the new SAT is better or worse than a 500 on the old SAT until after hundreds of thousands of kids take the new one for real in April 2016.
Delivery Schedule for Concordance Table Concordance Available Redesigned SAT to current SAT May 2016 Derived Concordance:
Redesigned SAT to ACTMay 2016
Is Coleman just making this up as he goes along and is hoping the psychometricians can eventually come up with data to support his intuitions? My guess is that Coleman's intuitions are less stupid than those of most figures in the education reform biz, but still ...
This might be me being paranoid, but I think all this is duck speak for coming up with a less "disparate impactey" test.
ReplyDeleteWhen putting the changes to the SAT in perspective over a long chronological time frame, it is just another indicator of the decline and dissolution of WASP history, accomplishment, and civil management. Just, as the other institutions the WASPs built, have been denigrated, the SAT will too.
ReplyDeleteAs a Pole I sometimes wonder, if only we as a nation we had 20% of the old WASP societal framework and mentality along with our fervent nationalism, we could be a top 10 world economy and power.
This fiasco has the potential to be more disastrous even than turning over the Vietnam War planning to little Bobbie McNamara and his team of best-and-brightest handsome young geniuses.
ReplyDelete[I'd also mention the lunacy of turning over our Middle Eastern policy to the "Neocons", but as we now know, they had a parallel (and completely segregated) objective in mind, so the analogy isn't quite as good there.]
Steve, you know exactly what this is about---closing The Gap. Countenance has it exactly right--overemphasizing the essay portion, de-emphasizing the math/science/vocab/reading comp? C'mon, this about whites and Asians persistently scoring too high and NAM's just as persistently scoring too low.
ReplyDeleteIn the announcement yesterday, the spokesman didn't reference The Gap by name, but he threw negative oblique references out there such as "cramming for the tests", "unequal access to preparatory classes" "designing tests for access as well as excellence" and other stuff. So what if Jamaal and Jose can't multiply their way past a 300 Math score--just write me up 3-5 paragraphs about the racism you've overcome and you're in!
Every toxic teaching theory of the past 40 years has had as its source a bunch of Frankfurt School hippies/commies who conducted coups at every ED department at every major university in the country. It's been on autopilot since then.
okay
ReplyDeleteThe 1600 SAT was and is the peak. It's all downhill from there in a effort to close "the gap". The verbal half was never stacked against black people. Proof of this was Vietnamese refugees and other poor Asians arriving here and doing great on the verbal. Where English was the second language they learned. Contrasted with American blacks having English for their first language (unless it was an Ebonics variation)
ReplyDelete"So, the College Board won't be able to tell you whether a 500 on the new SAT is better or worse than a 500 on the old SAT until after hundreds of thousands of kids take the new one for real in April 2016."
ReplyDeleteIt kind of sounds like, "We have to pass the bill to find out what is in it." -- It's all just a little ass backwards (pardon my French!) and not very transparent. These things seem to be requiring a lot of trust from the American public.
Interesting theory I read on AR tonight:
ReplyDeleteThe fact that the man spearheading the redesign of the SAT and the man heavily involved in the development of Commune Core is the same man is not a coincidence. He's redoing the SAT around Commune Core, in order to "prove" the "success" of Commune Core and lock it in to the education system.
Jon Duce
ReplyDeleteLike I said in Sailer's other thread about the SAT redesign, if these people were only worried about formulating a standardized test that is impervious to cramming and prepping, then just give a straight up IQ test. But that will never happen because of that which is too taboo to discuss.
If the SAT becomes a less valuable predictor, then what will take its place? Conjectural answer: pull and privilege among elites. Why elites? Because the proles are no longer needed, since Sputnik was a long time ago. Why pull and privilege? Because what test is a good replacement for the SAT? Probably there isn't one that's as widely accepted. Bottom line: making the SAT more "just" could result in increased privilege for the regatta set, paradoxically enough.
ReplyDeleteThe SAT is moving further and further away from an aptitude (IQ) test, as was the case with the last update as well, toward a mere achievement test, achievement measured by how well students demonstrate having inculcated the tenets of Common Core.
ReplyDeleteOh, my goodness. My oldest child is a high-school freshman right now. I really regret that she may suffer from Coleman's arrogance and incompetence, and too her siblings following soon after. Phooey.
ReplyDeletePeople assume that the SAT is being redesigned to reduce disparate impact, but we all know that the SAT never stands in the way of AA, so it's irrelevant. The vast majority of colleges in the US are not selective and don't care about SATs. It looks like this redesign was targeted to benefit Asians and other science oriented types.
ReplyDeletecoleman doesn't want the new questions validated against the old ones. it would weaken the rationale behind the redesign if scores on new and old questions were directly compared and found to strongly correlate. the blow wouldn't be decisive, but it would be inauspicious if the same kids who can't define "depreciatory" can't figure out which facts support a conclusion.
ReplyDeleteSince this seems to be the currently applicable edumakshun thread at iSteve, here's an ugly new piece on the UNC-CH AFAM/NCAA revenue sports debacle:
ReplyDeleteUNC Fake Classes Scandal: Key Administrator Agrees to Tell All
The reader commentary following the article is particularly devastating.
Really devastating.