tag:blogger.com,1999:blog-9430835.post3791495441701091899..comments2024-03-28T16:22:14.888-07:00Comments on Steve Sailer: iSteve: More from Audacious Epigone on how well schools are doingUnknownnoreply@blogger.comBlogger29125tag:blogger.com,1999:blog-9430835.post-22531527103169383412007-11-22T15:28:00.000-08:002007-11-22T15:28:00.000-08:00Justin's perspicacity is pretty evident in this gr...Justin's perspicacity is pretty evident in <A HREF="http://services.alphaworks.ibm.com/manyeyes/view/S5kPCKsOtha6WCVfDbANK2-" REL="nofollow">this graphic</A> (brown indicates greater improvement, blue indicates relative deterioration) of the states by relative improvement (in math and reading) from the 2003 class of fourth graders to 2007 eighth graders, for whites only. Excepting California, the West does better than the South and the Northeast, and the differences are stark.Audacious Epigonehttps://www.blogger.com/profile/07495507254628580077noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-43406335537744104512007-11-22T04:37:00.000-08:002007-11-22T04:37:00.000-08:00Steve--Every state is in the grip of NCLB and must...Steve--<BR/><BR/>Every state is in the grip of NCLB and must provide a state test that measures Adequate Yearly Progress (AYP). These state tests are based on a state's curriculum--and each state curriculum is different.<BR/><BR/>And I mean widely different from state to state.<BR/><BR/>Variance could be from the paroxysms of curriculum upheaval that have been going on since the onset of NCLB legislation--some states have a much better state curriculum (scope & sequence, etc.) than others.SavRedhttps://www.blogger.com/profile/17437258864899902195noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-44192521977391943312007-11-21T13:58:00.000-08:002007-11-21T13:58:00.000-08:00I had another thought today about the cause of the...I had another thought today about the cause of these divergent results on standardized tests:<BR/><BR/>In some states, teachers may take the NAEP test seriously; in those states, teachers would earnestly urge students to take their time and try to do well.<BR/><BR/>In other states (such as those with powerful teachers unions), the teachers would not care. They would say something like, "George Bush is making you take this test, so you have to do it. But it doesn't apply to your grade, so don't worry about it. If we finish early, we'll watch a video."<BR/><BR/>I call this the Blowoff Theorem, where there are some states where teachers basically blow off the NAEP.<BR/><BR/>To test this theorem, research would be needed as to which states apply any sort of reward or punishment based on NAEP scores.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-71533280723263951002007-11-21T11:38:00.000-08:002007-11-21T11:38:00.000-08:00Beowulf's idea about sleep deprivation is the kind...Beowulf's idea about sleep deprivation is the kind of thing that could be analyzed in a multiple regression study. If states with 8:30 starting times do better than states with 7:30 starting times, well, then we've learned something actionable. (although that would probably show up most in the 12th grade NAEP scores).Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-66352665233230005992007-11-21T11:11:00.000-08:002007-11-21T11:11:00.000-08:001. It could be as simple as states having varying ...1. It could be as simple as states having varying tolerance (if not encouragement) of cheating. <BR/><BR/>2. Or perhaps states have different policies on social promotion-- I don't know at what level of school administration such policy is set but perhap a struggling 7th grader, in say, Kentucky is more likely to to be held back to repeat a grade while across the river in West Virginia the same student would be socially promoted to 8th grade. Tthe KY 7th grader (to use my hypo, who knows what each state's policy is) will have mastered 7th grade curriculum and will be a year older when he takes the 8th grade assessment and the WV 8th grader will bomb.<BR/><BR/>3. Perhaps there are different levels of chronic sleep deprivation. A school that starts an hour later means students get an hour more sleep. Chronic sleep deprivation affects cognitive performance in children and adults (in basic training, army recruits given more lights out time do better on both physical and academic tests). <BR/><BR/>4. Related to that, is there a difference in the percentage of students on ADHD drugs? Leaving aside whether ADHD is over or under-diagnosed, stimulants have been used to treat sleep deprivation since World War II.<BR/><BR/>5. Does family stability vary? Even if the family itself is stable, there are many transient students. If a student goes to 3 schools in three years (or worse, 3 schools in one year), the lack of a national curriculum means each school teaches required subjects in a different order. He might read Red Badge of Courage three times and never learn about photosynthesis.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-46226317615483029442007-11-21T10:47:00.000-08:002007-11-21T10:47:00.000-08:00I've thought about this some more and I think this...I've thought about this some more and I think this data is useless for figuring out which state has the best school system.<BR/><BR/>If you could give every kid a genetic-sequencing-based IQ test, and see to the extent the kids performed above or below their genetic potential, then you'd know.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-65836573105206575572007-11-21T08:48:00.000-08:002007-11-21T08:48:00.000-08:00There exists standards for educational testing. T...There exists standards for educational testing. They require the test publisher to describe the instrument and how the results may be interpreted. ETS, which represents the NAEP's publisher, recommended that NAEP score comparsons of 2003-4th to 2007-8th be avoided. <BR/><BR/>The standards also permit uses of the NAEP data other than those recommended by the test publisher. However, the validity of other uses of of NAEP data must be demonstated. I would like to see the validity evidence that "Audacious Epigone" has put together to justify this particular use of NAEP data. I doubt that it exists (but I may be wrong).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-76697001224514857712007-11-21T04:06:00.000-08:002007-11-21T04:06:00.000-08:00Probably a better solution would be to pay to have...<I>Probably a better solution would be to pay to have the 6209 children of the NLSY women given the 4th grade NAEP tests as they reach the proper age, and then given the 8th grade NAEP test. We could then do a multiple regression analysis based on information about the schools they've attended and about their own early childhood IQs and their mother's performance on the AFQT!</I><BR/>Charles Murray ought to write his next book on exactly that. Without NAEP data for these kids, I'm not sure how much getting at their IQs and breaking it down by state would help, beyond making for a better variable to correlate against improvement than IQ estimates based on other NAEP data.<BR/><BR/>If data by state is available, it should definitely be looked at. Setting school performance aside for a second, the state IQ estimates using NAEP data could easily be replaced with this data, which would, if representative, probably be more accurate (although sample size would be a problem in states with fewer than ~5 million people). <BR/><BR/>It seems so obvious I wonder why it hasn't been done.Audacious Epigonehttps://www.blogger.com/profile/07495507254628580077noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-22359694666619760622007-11-20T20:26:00.000-08:002007-11-20T20:26:00.000-08:00Having lived in Connecticut until a decade ago I'm...Having lived in Connecticut until a decade ago I'm rather surprised by the state's last-place finish. All I can think of is that Connecticut has many parochial and private schools, and they may skim off some of the better students. The main weakness with this theory is that children generally enter these schools in first or ninth grades, and therefore the "skimming off" factor wouldn't really account for the fourth to eighth grade drop.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-90472069034921904912007-11-20T20:15:00.000-08:002007-11-20T20:15:00.000-08:00By the way, here's my VDARE.com article with links...By the way, here's my VDARE.com article with links to Charles Murray's paper on the IQs of the 6209 children of NLSY moms.Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-82493233187764061842007-11-20T20:13:00.000-08:002007-11-20T20:13:00.000-08:00I wonder if it is or will become possible to link ...I wonder if it is or will become possible to link the National Longitudinal Study of Youth database to the NAEP? That could be hugely useful because we have IQ scores for 6209 children born to roughly the same number of women who took the AFQT IQ test back in 1980.<BR/><BR/>In 1979, the Bureau of Labor Statistics established a nationally representative sample of about 13,000 young people born from 1957 to 1964. In 1980, the military paid to have the entire sample take its enlistment IQ test, the Armed Forces Qualification Test. In 1990, the NLSY methodically checked up on how they were doing in life. The military provided the data to Charles Murray and Richard J. Herrnstein and it wound up as the centerpiece in the 1994 bestseller The Bell Curve.<BR/><BR/>The NLSY is still going on. It has now even measured the IQs of 6209 children of women in the original panel—2557 of whom were born to black female panelists.<BR/><BR/>If they keep it up, they'll probably get up to, what, maybe 8000 children and they must have the data somewhere broken down by states.<BR/><BR/>That's probably a big enough sample size to look at California, Texas, Florida, and New York and maybe at outliers like Connecticut and Massachusets.<BR/><BR/>Probably a better solution would be to pay to have the 6209 children of the NLSY women given the 4th grade NAEP tests as they reach the proper age, and then given the 8th grade NAEP test. We could then do a multiple regression analysis based on information about the schools they've attended and about their own early childhood IQs and their mother's performance on the AFQT!Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-73918262826047303902007-11-20T19:44:00.000-08:002007-11-20T19:44:00.000-08:00Ian,Is there a standardized national sample of par...Ian,<BR/><BR/>Is there a standardized national sample of parental satisfaction? I'm just getting 'Parental satisfaction in Dallas district schools' in searches.<BR/><BR/>Justin,<BR/><BR/>Interesting observation. A graphical representation would be nice. It would be a nice augmentation I will draw up.<BR/><BR/>Φ,<BR/><BR/>Yes, having actual IQ averages would be enormously beneficial for a countless number of sociological work (in the broadest sense of the phrase).<BR/><BR/>Agnostic,<BR/><BR/>The best stat to use? Population density would only be a rough proxy, as some states have little urbanization but are still not that rural, like Nebraska, where there are lots of little towns all over the place.<BR/><BR/>Anon,<BR/><BR/>Of course the average scores are estimates, using representative samples of a state's population. What's your point? Exit polls rely on estimates. So does marketing research. And on and on.<BR/><BR/>I <A HREF="http://anepigone.blogspot.com/2007/11/state-rankings-by-naep-improvement-from.html" REL="nofollow">looked at fourth and eighth grades over a period that spanned eight years</A>. The gaps correlated quite strongly for that entire time frame. Secondly, I converted score variances into standard deviations.Audacious Epigonehttps://www.blogger.com/profile/07495507254628580077noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-26167275790132951732007-11-20T18:49:00.000-08:002007-11-20T18:49:00.000-08:00"ETS does the analysis and reporting for NAEP. ETS..."ETS does the analysis and reporting for NAEP. ETS recommends avoiding 2003-4th to 2007-8th comparisons. The NAEP scale doesn't support such comparions."<BR/><BR/>Of course ETS is a private company. If they irritate too many customers their employees and managers might actually have to go find real jobs. Their customers are actually schools and universities, not the students who pay for the testing - if the schools start not accepting SATs, the NAEP, and such, then ETS would be in trouble. So ETS panders to the educrats, in this case by avoiding, even misleading other users of the data, the obvious use of their data for inter-state and inter-school comparisons.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-86834799142321801642007-11-20T14:44:00.000-08:002007-11-20T14:44:00.000-08:00Steve -- Homeschooling? One thing jumps out is tha...Steve -- Homeschooling? One thing jumps out is that due to large distances, a lot of kids in the Mountain West are homeschooled or distance learning or both.<BR/><BR/>What is the degree of homeschooling in say, VT vs. CT?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-80495229211675229262007-11-20T14:05:00.000-08:002007-11-20T14:05:00.000-08:00Steve,Congratulations on publishing AE's table. Ex...Steve,<BR/><BR/>Congratulations on publishing AE's table. <BR/><BR/>Explaining the differences among states is now the task at hand.<BR/><BR/>I think readers are off the track if they look for differences in the white student populations. I believe the table actually shows what it hoped to show: the relative effectiveness of each state's corps of teachers. This means (per Occam) that we should look for the explanation in differences among teachers and their teaching practices.<BR/><BR/>What can explain the differences, then, in the effectiveness of teachers? One determinant could be the toxic effect of union power. Under a certain kind of powerful-union regime, teachers no longer have any incentive to care about the quality of work they do. These union-protected teachers can't be fired or disciplined.<BR/><BR/>To examine this relationship, one must look at indicators of teacher union power. One indicator is average teacher pay, and a table of that can be found at this URL:<BR/><BR/>http://www.aft.org/salary/2005<BR/>/download/AFT2005SalarySurvey.pdf<BR/><BR/>Go to page 32 in the Adobe reader.<BR/><BR/>Connecticut is known for having the highest-paid teachers. Michigan is known for having a powerful teachers union. Adjusting for relative income levels and cost of living, Michigan's teachers' pay is higher than Connecticut's. To me it is no coincidence that these states are at the bottom of the list.<BR/><BR/>In my guess, however, a better correlation will be found if you look at relative numbers of teachers fired. I believe that states where very few teachers are ever fired will rank near the bottom of your chart. <BR/><BR/>That data on firings, which I don't know where to find, would be an indicator, in my mind, of a "toxic" union presence that would undermine teacher performance.<BR/><BR/>Keep up the good work.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-23063677302286739702007-11-20T13:28:00.000-08:002007-11-20T13:28:00.000-08:00Regression to the mean (as noted above) should be...Regression to the mean (as noted above) should be checked. Assuming a certain noise component exists in any state aggregate (even at the state-wide level if the exam is given on the same day, there can be bad weather, bad hair days etc.). Those bad-hair-day states would likely show the greatest improvement four years later. Regress the reported four-year gain against the actual level of the fourth grade results and see if there is a negative relationship. <BR/><BR/>Another tactic to proxy a noise effect is to see if the fourth-grade result is much different from previous fourth-grade results for that particular state. If say Conn. has a higher than average fourth grade result compared to earlier years, it is possibly reflecting some one-time or extraneous event. The eight-grade result would not be affected by the same event and the gap between the four years would be unusually low.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-18559270831622509232007-11-20T12:11:00.000-08:002007-11-20T12:11:00.000-08:00Obviously, there is a problem with noise in the da...Obviously, there is a problem with noise in the data, which means that when looking at absolute results, you only be sure at the 95% confidence level that, say, Massachusetts ranks from first to sixth in the 4th grade reading test. For more typical states, the 95% confidence level is pretty broad for any single administration of a single subject. <BR/><BR/>One easy way to deal with this problem is to look at more data points: 4th grade, 8th grade, 12th grade, reading, math, science, history, 2007, 2003, 1999, etc. They're multiplicative, so you can quickly come with a lot of data points (e.g., 36 in this example).<BR/><BR/>If you lump a bunch of these data points together, however, you can conclude that, yeah, you can be quite confident that Massachusetts students score higher than West Virginia students overall. I've looked at many of these data points casually, and Massachusetts _always_ scores higher than West Virginia.<BR/><BR/>Okay, but that's for absolute data, which is driven at least as much by the innate quality of the students (Massachusetts's many colleges have been importing smart people into the state for generations, while West Virginia's lack of upscale jobs has been exporting smart people) as by whatever the schools are doing.<BR/><BR/>For relative rankings, noise is likely to become a much bigger problem. So, here are some strategies to judge how much effect noise has: <BR/><BR/>First, do the analysis for other NAEP subject tests than reading and math, such as history and science.<BR/><BR/>Second, do the reading and math analysis for, say, 1993 vs. 2003. <BR/><BR/>Do you see consistent patterns or is it all just random?Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-12146198509193657532007-11-20T11:36:00.000-08:002007-11-20T11:36:00.000-08:00You ask, "Is this the best ranking yet available o...You ask, "Is this the best ranking yet available of how the states differ in how good a job their public schools are doing?" NO!!!!!<BR/><BR/>Using NAEP to rank order states is NOT a valid use of NAEP scores. See the reason/ proof at http://pareonline.net/getvn.asp?v=10&n=9<BR/><BR/>ETS does the analysis and reporting for NAEP. ETS recommends avoiding 2003-4th to 2007-8th comparisons. The NAEP scale doesn't support such comparions.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-45552853306002514772007-11-20T11:32:00.000-08:002007-11-20T11:32:00.000-08:00Right, if we had IQ scores at age 7 or whenever, w...Right, if we had IQ scores at age 7 or whenever, we could further adjust for innate differences among white students. Still, take a look at four East Coast affluent, liberal states that all likely have fairly smart white kids. In 4th to 8th grade growth, we see:<BR/><BR/>Maryland - Excellent<BR/>Massachusetts - Good<BR/>New Jersey - Average<BR/>Connecticut - Awful<BR/><BR/>That's either very indicative of something or other apart from the innate quality of the kids or very random.Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-21656854666064538542007-11-20T11:21:00.000-08:002007-11-20T11:21:00.000-08:00Steve: A solid effort given what we have, and yet...Steve: A solid effort given what we have, and yet: there are all sorts of opportunities for within-race aptitude variation to bias the results. It would not surprise me that, for instance, West Virginia's whites are not as bright as those in Maryland. And I bet there is plenty of aptitude variation within a state as well, between whites in, say, Baltimore and those in Columbia.<BR/><BR/>Comparing progress between 4th graders and 8th graders corrects for this, but do we not need to assume that the effect of a lower IQ on educational performance isn't cummulative? What if "slow" learners fall <I>even further</I> behind quick ones between 4th and 8th grade?<BR/><BR/>What we really need is to compare a school's average IQ with their average z-score on the NAEP. These ratios could then be compared between individual schools and districts, where is suspect most of education policy is determined.<BR/><BR/>I assume, however, that we don't have IQ data for a sufficient number of students, IQ testing not being very widespread. I suppose we could use parental wealth/income as a (very) rough proxy, but this introduces regional biases that are not easily corrected.Burkehttps://www.blogger.com/profile/14252946969701576139noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-29767909000136912922007-11-20T11:01:00.000-08:002007-11-20T11:01:00.000-08:00It's possible the the low scoring states are artif...It's possible the the low scoring states are artificially boosting their 4th grade scores by "teaching to the test."<BR/><BR/>The 8th grade test is harder to coach for because it tests a wider spectrum of cognitive ability.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-28462527804806749352007-11-20T10:28:00.000-08:002007-11-20T10:28:00.000-08:00The most startling thing I noticed was how well th...The most startling thing I noticed was how well the West does compared to the East. Aside from the stupid liberal states of California and Hawaii, only one Western state was on the minus side, and that just barely (WY at -0.07). <BR/><BR/><BR/>Have you looked at the data relative to school choice options, such as prevalence of charter schools? As I understand it, Western states lead the nation in school choice options. <BR/><BR/>As you noticed, those Western states tend to have less demographic diversity, i.e., they are more white. Maybe this is just another data point of proof that whites do better in environments away from minorities?Justinhttps://www.blogger.com/profile/01023125641719686613noreply@blogger.comtag:blogger.com,1999:blog-9430835.post-12700144782680347752007-11-20T09:04:00.000-08:002007-11-20T09:04:00.000-08:00This is a little off topic but it occured to me th...This is a little off topic but it occured to me that there might not be any room for improvement at some of the top performing groups from 4th to 8th grade. Wasn't that one of the theories presented here, that interventions are more likely to produce significant improvements in people from lower in the IQ spectrum than in those with higher IQs? I'd take this to mean you'd have to have absolute and relative test scores in order to make a fair analysis.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-48795451503652482732007-11-20T08:23:00.000-08:002007-11-20T08:23:00.000-08:00I would look at a scatter plot (with one axis four...I would look at a scatter plot (with one axis fourth grade performance and the other eighth grade performance). I am not convinced this is a good way of evaluating schools. Some regression to the mean could be going on, states that perform well with K-3 education are likely at a disadvantage. I believe school starting ages are not uniform (some preschool, some no K) which which might affect the fourth grade scores more than the eighth grade scores.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9430835.post-53586164993240254012007-11-20T08:20:00.000-08:002007-11-20T08:20:00.000-08:00Some of the states cheat and lie re all their edu-...Some of the states cheat and lie re all their edu-stats.Anonymousnoreply@blogger.com