March 21, 2009

Calling all datanauts

The federal New Immigrant Survey, which surveyed 9,400 new legal immigrants in 2003 and intends to track them for decades, offers a fascinating source of data, much of it free to the public. The databases can be downloaded formatted for users of each of the three high-end statistical packages: SAS, SPSS, and Stata. So, if you are a data analysis heavyweight, here's a relatively unplumbed but important data source.

Jason Richwine calculated the Wechsler Digit Span IQ subtest scores for different immigrant groups from it, but nobody has yet published the IQ scores from the four Woodcock-Johnson III IQ subtests given to 3 to 12 year-old children of immigrants with legal permanent residency in the NIS.

You'd have to be careful in evaluating the scores, since on most of the subtests, between 25-50% of the kids got scores of zero right out of up to 76 points. Presumably, they were utterly defeated by language problems due to the Woodcock-Johnson IQ test not having been translated into their exotic languages. The zeroes should be left out. (In contrast, the very simple Wechsler Digit Span subtest was translated into a huge number of languages. One immigrant's child took it in Amharic.)

However, 704 of the Woodcock-Johnson tests were given in Spanish, so those scores are probably more or less fair. And they're the most important ones for understanding the impact of massive Hispanic immigration on America's future.

And you'd also need to work on how to convert Woodcock-Johnson scores to IQ scores.

It would be an ambitious project, but a significant one.

My published articles are archived at -- Steve Sailer


Anonymous said...

This kind of thing is a real public service. Actually doing and publishing the study you suggest would be a bigger one.

testing99 said...

If you don't have SPSS, SAS, Stata, or so on, don't despair.

The R Project has you covered.

Make sure you get "R Commander" which works well on Linux, Windows, and Mac OS X, as it provides a friendly front end to R's rather complex command line.

There are a LOT of example graphs and the code to generate them there as well, so you can visualize the data more completely.

Yes, R can also read in from MySQL, so if you are a decent PERL or other coder, you can take the downloadable data and dump it into a MySQL Database, and hook R into it to extract subsets, etc.

That is the great thing about Open Source, for a little trouble learning the software, you have the same power as high-end but costly statistical programs.

Technology is increasingly a commodity. Often, effectively free.

Linus said...

so if you are a decent PERL or other coder....

Ah, that brings back memories ... of the days before PHP and Python, when non-dinosaurs actually wrote code in that unwieldy beast called Perl.

That is the great thing about Open Source, for a little trouble learning the software, you have the same power as high-end but costly statistical programs.

And the great thing about The Pirate Bay is that, for a little trouble searching for SPSS, SAS or Stata there, and a little uTorrent client, you can get the real thing. For free.

And that real thing will install without hassles, and will be both usable and pretty, as opposed to the open-source standard of user-unfriendliness (being built by geeks, for geeks) and butt-ugliness (being built by people with absolutely no aesthetic sense whatsoever).

XKCD had a great comic a while back about installing Linux, which just about sums up the effects of open-source on anyone who has better things to do with his life: Cautionary.

Your conscience may feel better about taking the free, open-source product, rather than stealing the commercial one, byte-by-byte. But since you're contributing no money for it either way, it has exactly the same negative effects on the industry and the people working in it. That is, either way, no one's getting paid for their work, so no real jobs are being created or supported.

Not to mention that no small impetus of the open-source movement is the desire of adolescent geeks to "stick it to The Man," for having the temerity to charge "too much" for his product--thereby rather brilliantly undercutting their own employment opportunities and wages. Consider Richard Stallman, on Why Software Should Be Free:

"Fifteen years ago, there were articles on 'computer addiction': users were 'onlining' and had hundred-dollar-a-week habits. It was generally understood that people frequently loved programming enough to break up their marriages. Today, it is generally understood that no one would program except for a high rate of pay. People have forgotten what they knew fifteen years ago."

"When it is true at a given time that most people will work in a certain field only for high pay, it need not remain true. The dynamic of change can run in reverse, if society provides an impetus. If we take away the possibility of great wealth, then after a while, when the people have readjusted their attitudes, they will once again be eager to work in the field for the joy of accomplishment."

"The question, 'How can we pay programmers?' becomes an easier question when we realize that it's not a matter of paying them a fortune. A mere living is easier to raise."

Betcha he voted for Obama, eh?

Supporting open-source is sort of like sending money to Africa to save starving children ... so that they can wantonly reproduce another generation of even-more starving children. It may make you feel good and "moral," but it doesn't actually accomplish anything.

(Remember '99 posting some time ago about Microsoft's supposed hiring philosophy, where he got nearly everything demonstrably wrong [and some of it was outright just made-up]? Do you see how his posting above, on this thread, is just a continuation of the same line of thinking? Not that the pointer to open-source, the R Project, and a [separate, of course!] front end for that isn't helpful; it is ... especially if your goal is to break up your marriage, and sleep with your computer instead.)

Anonymous said...


Off topic, but if you haven't seen the proposal to solve our housing crisis by selling the houses off to immigrants, I think it's worth your time to take a look at it.

Planetary Archon Mouse

Anonymous said...

I hope they make illegal immigrants legal asap. We need someone to fill up all these empty houses sitting on the market.

SPLC Fan said...

You're actively encouraging people to search for hate facts? Shame on you!

Anonymous said...

I live in Salt Lake City and have noticed an increasing number of immigrants from all over the world

My understanding is that the Church of Later Day Saints converts folks in different countries and then brings some of them back here to Salt Lake City

Does anyone know whether the church wants these folks to live in Utah forever or to go back to their home nations eventually?

Also - does the church encourage these immigrants to marry within their own racial group or to marry church members of other racial groups?

Finally, certain religions are known for puahing the highest IQ young men to marry the highest IQ young women and other religions generally don't do this

which category is the church in ?

Richard Hoste said...

"Does anyone know whether the church wants these folks to live in Utah forever or to go back to their home nations eventually?"

When have third world immigrants voluntarily gone back to their own country? And with our birthright citizenship, forget about it.

Does anybody have links to good reading material on the LDS and race? I know they were late to the whole civil rights thing, are they completely "caught up" when it comes to race issues?

Ronduck said...

Linus, despite all of your contortions to justify theft, that is what you are doing.

Ronduck said...

Testing99 said...

Technology is increasingly a commodity. Often, effectively free.

Older tech is increasingly free. Newer tech is still expensive.