As I've mentioned, one of the rules of polite journalism in discussing testing firefighters is to assume that paper and pencil tests must be irrelevant to the obviously moronic job of spraying water on burning buildings. Never refer to the voluminous data assembled over the decades by the Pentagon on the relationship between performance on paper and pencil tests and performance on similarly physical jobs.
When researching my 2004 article on John F. Kerry's and George W. Bush's IQ scores judging from their performance on the Officer's Qualification Tests they took in the later 1960s (Bush 120-125, Kerry 115-120, which turned out to fit with their GPAs at Yale), I read a lot of studies from the 1960s by the military's psychometricians documenting the predictive validity of these exams. I then tried to track down the authors to help me understand Kerry's and Bush's scores.
I spent two hours on the phone with a very helpful gentleman, now a college professor of statistics, who had retired after many years as the head psychometrician for one of the major branches of the Armed Services.
Among much else that was interesting, he mentioned that in 1990 he had provided to Charles Murray the U.S. military's scores from the renorming of its AFQT enlistment test. In 1980, the Pentagon had paid the Department of Labor to give the AFQT to all 12,000+ young people in its National Longitudinal Study of Youth database. The middle section of The Bell Curve is devoted to tracking how these ex-youths, now 25 to 33 in 1990, were doing in life in relation to their IQ scores a decade before.
My source had nothing but praise for The Bell Curve.
The psychometric expert said something that seemed puzzling to me. He said that the General Factor of intelligence completely dominated job performance as a pilot to such an extent that it really wasn't worthwhile to give multiple intelligences tests of specific piloting skills, such as the one George W. Bush took in 1968 to measure his 3-d visualization skills.
For example, a question might ask:
Bush only scored, I believe, at the 25th percentile on this test, but I don't think this kind of thing came up much in the Oval Office.
My source said that he recommended getting rid of flying-specific tests for admission to pilot-training, but the brass wouldn't go along with it because they insisted their had to be pilot-specific skills separate from the g Factor.
Listening to him, I certainly agreed with the brass. After all, I have a decent IQ, but I'd make a terrible pilot during the brief interval before I became a smoking crater due to making some stupid mistake.
And, this is not something I only recently realized. I can vaguely recall being 16 and looking at the catalog from the Air Force Academy and deciding that, based on my experience driving a car, riding a bike, playing sports, and generally bumbling about in the physical world, that I wasn't cut out to pilot Air Force jets.
I've wondered about this expert's finding over the years, and I think I've finally started to figure it out: People with high IQs who would be bad pilots generally figure out for themselves that they would be bad pilots; so, they never take the tests to be pilots. Thus, the high correlation between the g Factor and pilot performance: high IQ individuals are already selected for having pilot-specific skills.
Similarly, high IQ guys who would make lousy firemen already know it, so they don't take the firemen's test much.
Thus, a hiring test like the New York ones ruled too discriminating by Judge Garaufis tend to work well. They are combination aptitude and achievement tests with all the questions solely about firefighting, but all the information needed to answer the questions given on the test. Still, under pressure, it's not too easy to decipher passages about technical details of chainsaw maintenance.
Thus, to score perfectly on these kind of tests, it's helpful to be both reasonably bright and to have studied firefighting guidebooks. High IQ guys who wouldn't make good firemen tend to figure out while they're studying that this isn't the career for them and thus don't take the tests. So, these kind of aptitude/achievement tests work quite well.
When researching my 2004 article on John F. Kerry's and George W. Bush's IQ scores judging from their performance on the Officer's Qualification Tests they took in the later 1960s (Bush 120-125, Kerry 115-120, which turned out to fit with their GPAs at Yale), I read a lot of studies from the 1960s by the military's psychometricians documenting the predictive validity of these exams. I then tried to track down the authors to help me understand Kerry's and Bush's scores.
I spent two hours on the phone with a very helpful gentleman, now a college professor of statistics, who had retired after many years as the head psychometrician for one of the major branches of the Armed Services.
Among much else that was interesting, he mentioned that in 1990 he had provided to Charles Murray the U.S. military's scores from the renorming of its AFQT enlistment test. In 1980, the Pentagon had paid the Department of Labor to give the AFQT to all 12,000+ young people in its National Longitudinal Study of Youth database. The middle section of The Bell Curve is devoted to tracking how these ex-youths, now 25 to 33 in 1990, were doing in life in relation to their IQ scores a decade before.
My source had nothing but praise for The Bell Curve.
The psychometric expert said something that seemed puzzling to me. He said that the General Factor of intelligence completely dominated job performance as a pilot to such an extent that it really wasn't worthwhile to give multiple intelligences tests of specific piloting skills, such as the one George W. Bush took in 1968 to measure his 3-d visualization skills.
For example, a question might ask:
Which picture represents how the horizon would look straight-ahead out the cockpit window when you are in the midst of turning from flying north to flying east while banking 60 degrees?
A. _
B. /
C. \
D. |
Bush only scored, I believe, at the 25th percentile on this test, but I don't think this kind of thing came up much in the Oval Office.
My source said that he recommended getting rid of flying-specific tests for admission to pilot-training, but the brass wouldn't go along with it because they insisted their had to be pilot-specific skills separate from the g Factor.
Listening to him, I certainly agreed with the brass. After all, I have a decent IQ, but I'd make a terrible pilot during the brief interval before I became a smoking crater due to making some stupid mistake.
And, this is not something I only recently realized. I can vaguely recall being 16 and looking at the catalog from the Air Force Academy and deciding that, based on my experience driving a car, riding a bike, playing sports, and generally bumbling about in the physical world, that I wasn't cut out to pilot Air Force jets.
I've wondered about this expert's finding over the years, and I think I've finally started to figure it out: People with high IQs who would be bad pilots generally figure out for themselves that they would be bad pilots; so, they never take the tests to be pilots. Thus, the high correlation between the g Factor and pilot performance: high IQ individuals are already selected for having pilot-specific skills.
Similarly, high IQ guys who would make lousy firemen already know it, so they don't take the firemen's test much.
Thus, a hiring test like the New York ones ruled too discriminating by Judge Garaufis tend to work well. They are combination aptitude and achievement tests with all the questions solely about firefighting, but all the information needed to answer the questions given on the test. Still, under pressure, it's not too easy to decipher passages about technical details of chainsaw maintenance.
Thus, to score perfectly on these kind of tests, it's helpful to be both reasonably bright and to have studied firefighting guidebooks. High IQ guys who wouldn't make good firemen tend to figure out while they're studying that this isn't the career for them and thus don't take the tests. So, these kind of aptitude/achievement tests work quite well.
My published articles are archived at iSteve.com -- Steve Sailer
"And never refer to the voluminous data assembled over the decades by the Pentagon on the relationship between performance on paper and pencil tests and performance on similar physical jobs."
ReplyDeleteYet journalists will be happy to tell you how intelligent you have to be to perform well at basketball and football.
europeasant
ReplyDeleteI scored 125 on the Army test in 1974, when I was 24 years old. I qualified for all army jobs. On the day of inductment I was offered either med tech or wheel and track mechanic job. I chose the mechanic job thinking it would lead to civilian employment after my service contract. The average score, I found out later on, for Mechanics was around 90. The behavior and general talk of the personnel around me bore out that score. I have no doubt that the army tests give a realistic assessment of an individual's capabilities.
On another note, it's time for you idiots to shut your mouths! Hawaii finally released the missing Birth Certificate today.
ReplyDeleteI've seen this assertion before, that a test measuring general intelligence is a better predictor of job performance than one tailored to a given occupation (and thereby resistant to disparate-impact claims), and that tailored exams have predictive validity primarily to the extent they measure g.
ReplyDeleteIt seems to me that self-imagined skeptics of employment tests, such as that in the Ricci case, stumble onto this reality when denouncing paper and pencil tests as mere exercises in "memorization" or "test-taking skills"--these criticisms more aptly apply to the sort of "job specific" tests they, and disparate impact theory, demand.
Think of that unintential Onion copy you pointed out in one of the articles denouncing Ricci, a complaint to the effect that the test required reading and thinking. Disparate impact theory has been grinding away at occupational testing as if in a deliberate effort to render all such tests basic exercises in rote memorization. Of course, this hasn't worked, as the more capable and dedicated will still outperform on any meaningful test, and the standards must be rendered farcical to achieve "balance."
The Ricci test, with the answers contained within readings on the same page, seemed a reasonable enough measurement of what might be the most important skill for such as firemen, the ability to understand and follow directions--that is, procedure, which I think you've pointed out before is how we maintain relatively superior fire and emergency services (for how much longer, we shall see).
So we can presume following Ricci it's on to the next phase, an attack on testing itself in favor of board selection processes (helped along by credulous media reports of how much more effective are these politically stacked, personally subjective means).
From this link, kindly provided by the La Carrefour de la Sagesse (with emphasis added):
ReplyDelete"Perhaps the strongest indicator that SLDR is large enough to matter is the well documented decrease in test to test correlation as a function of IQ group.
Detterman (1991) wrote: 'Low IQ subjects showed much higher correlations than high IQ subjects. Intercorrelations of IQ subtests, correlations of cognitive ability measures with each other, and correlations of IQ with measures of cognitive abilities all displayed the same effect. For both the WAIS-R and WISC-R, average subtest correlations were highest in the low ability group. Correlations declined systematically with increasing IQ. In both studies, correlations were found to be two times higher in low IQ groups than in high IQ groups. Measures from the basic tasks correlated more highly in the low IQ group than in the high IQ group.'
[...]
Below the right tail, the external validity of IQ tests is almost entirely due to its g loading. Jensen has repeatedly pointed out that if the g loading is factored out at the group factor level, the external validity of all of the residuals combined is nil. But at the upper end of the spectrum, he commented (Jensen 2000): 'In groups of people with high levels of g, relatively more of the variance in test scores lies in the lower-order group factors and in test specificity. Higher g persons tend to invest their g in a greater variety of intellectual activities and interests than lower g persons; that is, cognitive abilities are more differentiated at higher levels of ability. Analogously, wealthy people spend their money on a greater variety of things than do poor people.'
This observation is easily seen in daily experience and confirms the significance of SLDR at the individual level, not only as it applies to g loading, but also as it applies to human behavior.
Jensen was arguing that a battery of the most heavily g loaded subtests would be optimum for measuring intelligence, but pointed out that such tests would essentially hide non-g abilities that might be important at high levels."
So, Steve, it appears that your self-selection argument does have some facially supporting evidence.
By the way, anyone interested should look at the chart linked! It's informative.
SLDR stands for Spearman's Law of Diminishing Returns, which means that Howard Gardnerish multiple intelligences become more important the higher the IQ. Charles Spearman discovered the g Factor in 1904. Billare's link provides much more info on SLDR
ReplyDeleteThe answer is B. Not really fair though, since my IQ is around 163.
ReplyDeleteSteve sailer says:
ReplyDeletewhich picture represents how the horizon would look out the cockpit window when you are banking 60 degrees to the right:
a) _
b) /
c) \
d) I
It's got to be "c)", right?
Remove g-force and you're weightless as well as clueless...
ReplyDelete"One of the rules of polite journalism in discussing testing firefighters is to assume that paper and pencil tests must be irrelevant to the moronic job of spraying water on a fire."
ReplyDeleteNot when they want to justify lowering physical standards to get more women firefighters.
"which means that Howard Gardnerish multiple intelligences become more important the higher the IQ..."
ReplyDeleteI imagine if the liberal creationists ever get a hold of this, they'll ignore the bolded part above.
Answer to sample question: It's either B or C. It depends on the definition of "banked to the right". Is that counterclockwise or clockwise?
ReplyDeleteNice going Mr 163. Don't go into geometry.
ReplyDeleteOh, hell, Mr 163's right. What a waste of a snarky remark.
ReplyDelete"Yet journalists will be happy to tell you how intelligent you have to be to perform well at basketball and football."
ReplyDeleteI think Steve has proven that most journalists are not too bright.
Yeah, Mr 163's is right.
ReplyDeleteCourse 163 is right. Say the "pipe" below is you, and the straight line is the horizon.
ReplyDelete_|_
You turn right (eastward), you and the horizon become (relative to each other)
_/_
Turn the slash up to straighten yourself up, and you get the horizon relative to you:
|/
/|
So, it's /.
JD
Here's how I did it:
ReplyDelete"To the right" means clockwise.
Draw a straight line. Since only the relative angle between horizontal and pilot's view matters, turn the paper 60 degrees COUNTERCLOCKWISE. The pilot rotating in one direction with the horizon unmoving is equivalent to the horizon rotating in the opposite direction and the pilot unmoving.
You get B.
In regard to Chinese basketball players, the same is true of American tennis players.
ReplyDeletehttp://www.atpworldtour.com/Rankings/MatchFacts.aspx
3 of the top 6 servers, as measured by aces and first serve points won, are Americans (Roddick, Fish, and Querrey) but only 4 of the top 50 ranked players are American (the three big servers plus James Blake). So if you ask why the US producers so many big servers, it's because if you're an American and haven't been hitting tennis balls since you were four, the only chance of making it on the pro tour is if you can hit a 135 mph serve. Same goes for Chinese basketball players. The alternative theory is that the US plays on hard courts or Americans serve big because they play baseball and football, but if that were the case, you'd see more short Americans on tour with big serves, which is not the case . . .
Interesting theory, Steve. about how the High IQ, Low pilot skills candidates weed themselves out. I guess its the flip side of the way incompetent people overestimate their abilities.
ReplyDeleteParticipants scoring in the bottom quartile grossly overestimated their test performance and ability, and analysis confirmed that this miscalibration was due to deficits in metacognitive skill (the capacity to distinguish accuracy from error).
http://www.damninteresting.com/unskilled-and-unaware-of-it
The test question illustrates the point I made on this thread.
ReplyDeleteFlying requires superb hand-eye coordination, but since it's travel in three planes instead of two, it also requires a degree of g intelligence to grasp the logic of many counter-intuitive procedures. I, with my 45-year old mind clinging desperately to 120 IQ, initially said C. But I'm trainable in the logic: turning east at 60 degrees will cant the horizon up and away relative to you, not down and toward. You're canted down, so the horizon's canted up. Dum de dum.
I've never been a first responder but I can guess the same sort of thing with entry level firefighting: wanting to leap up and wave your arms around to try and make you're way through a smoke filled room instead of getting on the floor and crawling along a wall.
Incidentally, I've watched some (and I stress, some) firemen in my hometown in action. Let's just say that 'Gangs of New York' is not a terribly far-fetched scenario.
Here's how I did it:
ReplyDeleteAnd here's a loll for showing your work. Honestly, no one here's ever played Flight Simulator or even "shoot the aliens"?
Pens and paper will be really useful at 10'000 feet! I groked the right answer in a fraction of a second.
ReplyDeleteI then confirmed my answer by dimly imagining I was in a cockpit going North, banking 60 degrees to the right and observing what was happening to the horizon.
I would feel very uncomfortable if I had to rely on pen and paper explanation. The thought of working it out that way is to me much harder and more tedious.
Though we are both smart, I'd definitely prefer a pilot to have more of my gift than yours, if they were mutually exclusive. The more a pilot has abilities hard-wired the more spare processing capacity he has for quick-decision making.
Just imagine you're looking at the horizon...now bank right...see it? B.
ReplyDeleteYou can see it if you look at the top of your monitor, then put your hands out like you're flying a fighter jet, then turn your head to the right. Making "vroooom!" and "whoosh!" noises is optional.
..."how the horizon would look straight-ahead..."
ReplyDeleteThe obvious answer is B...if you keep your head fixed WRT the plane. But clearly the answer is A if you tilt your head to keep the horizon horizontal, as do motorcycle riders (both views are "straight ahead").
Whether or not pilots do this, I don't know, but it sure helps minimize orientation problems when cornering on a bike because the horizon doesn't appear to rock back and forth.
http://www.robhoyles.com/images/gallery/large/Road_bike_pics/BMW%201200GT%20Cornering.JPG
http://www.lazymotorbike.eu/tips/distance/cornering.jpg
This post hits home because of this line:
ReplyDeletePeople with high IQs who would be bad pilots generally figure out for themselves that they would be bad pilots; so, they never take the tests to be pilots. Thus, the high correlation between the g Factor and pilot performance: high IQ individuals are already selected for having pilot-specific skills.
My late father was a pilot. He had an FAA issued Private Pilot license and owned his own airplane. I grew up around airplanes and flying. I flew with my father and his friends, went to airshows and airplane museums with him, built model airplanes (still do), read books about airplanes (still do), and every classmate that knew me absolutely knew I was going to be a pilot when I grew up.
Then in 8th grade we had Career Day in our school and I signed up for the USAF and USN recruiters. I found out both services required pilots have 20-20 uncorrected vision. I have worn glasses all my life, and this was back in the early 1980's, well before Lasik was an established procedure. Needless to say I was crushed. I gave up joining the military to learn to fly and went to college to major in Computer Science.
Once I started working in IT in the early 1990's, I had enough money to start taking flying lessons and get my Private Pilot license. At the time, to get a Private Pilot license you needed to
1) Pass an FAA created 50 question written exam
2) Take at least 40 hours of flying instruction, including both dual instruction and solo flying.
3) After completing 1) and 2), pass another oral exam and a ride-along checkride with an FAA examiner.
I aced the written and passed my checkride and oral exam, but it took many more than 40 hours of instruction to get from 1) to 3). At one point it sadly dawned on me that even if I had perfect 20-20 vision and made it into pilot training in the US military, I would have been washed out based on my flying abilities. If I couldn't master a Cessna 172 in 40 hours as a civilian, I probably wouldn't have mastered a military trainer fast enough for the USAF or USN to keep me around. I probably would have ended up a navigator or flight engineer. Either that or I would have been in the same boat as Steve:
Listening to him, I certainly agreed with the brass. After all, I have a decent IQ, but I'd make a terrible pilot during the brief interval before I became a smoking crater due to making some stupid mistake.
BTW, The answer (from experience doing it) is B. Here is a sailplane making a 40 degree bank right turn.
The fundamental law of the HBD blogs: whensoever a g-loaded question is posted, a race to answer it shall ensue in the comments.
ReplyDeleteI got 'B' too! :) But as far as pilots go,Steve never mnentioned the one thing that stopped ME from ever trying to pursue pilot training:lousy eyesight. This was before lasik--which I wouldnt do anyway. (have some guy stick a laser in my eye?No thanks. )Then again...Anyway,I wonder about the diferences between guys w/perfect eyesight(you dirtbags!!) and guys like me??
ReplyDeleteFrom Slate, Why dumb recruits cost the military big time:
ReplyDelete> "...military analyst Jennifer Kavanagh* reviewed a spate of recent statistical studies on the various factors that determine military performance—experience, training, aptitude, and so forth—and concluded that aptitude is key. A force "made up of personnel with high AFQT [armed forces aptitude test] scores," Kavanagh writes, "contributes to a more effective and accurate team performance." "
> "The evidence is overwhelming. Take tank gunners. You wouldn't think intelligence would have much effect on the ability to shoot straight, but apparently it does. Replacing a gunner who'd scored Category IV on the aptitude test (ranking in the 10-30 percentile) with one who'd scored Category IIIA (50-64 percentile) improved the chances of hitting targets by 34 percent. "
(boldings mine).
g has a physiological basis, notably better reaction time. So, it need not be surprising that it helps even in physical activities. Plus, quickly adapting to little patterns that come up (even subconsciously) must be helpful in any activity.
Having grown up on airports with two pilot parents and soloing at 16 years, 1 day..I got so bored with airplanes and flying I never even bothered getting the PPL (which is nothing more than an advanced student license: you have to have the commercial license with instrument rating before you're even considered really a pilot). We had a lot of blue collar guys flying in those days because of the cheap gas and rental rates and the GI Bill.
ReplyDeleteMost people take 60 hours to get the private license because most flight instruction is so bad. CFI is the _entry level_ flying job in a general aviation career. In my day you went from CFI to CFII to maybe flying the occasional charter to a job flying checks at night. After earning the ATP rating you might be lucky enough to-if you were willing to put a second mortgage on the house to get the type rating-get a right seat Learjet job. Occasionally, very rarely, it was discussed in hushed tones someone had hit the Superfecta of aviation, an airline job.
Most pilots I knew were of average overall intelligence. The smartest were the doctors and lawyers, but their safety record was at the bottom.
We had a standard question. "What's the last thing that goes through the mind of an OB/GYN with a Bonanza?"
Answer: the accessory section of a TIO-550 Continental......
It does get me thinking. Since low IQ high self-esteem folks are often foolishly trying to be a pilot, fireman or soldier, is there a (no doubt smaller) cohort of high IQ low self-esteem people who think about trying their hand at something challenging, but don't want to start something they may fail at or end up hating.
ReplyDeleteIn terms of the military, that's one defect of an all-volunteer army, the service no longer gets (criminals "recruited" by their judge excepted) the soldier who thinks he's going to hate it but turns out to be a good fit for the soldier life.
The Army's ROTC program has the right idea. It has a month long summer camp for rising juniors with no military experience thinking about joining ROTC. The Basic Camp (or "Leader's Training Course" as they call it these days) is softer and shorter than basic training but its a better approximation of the service than sitting in the basement playing America's Army.
The important element is, whether a cadet loves it or hates it, there's no enlistment contract promising Uncle Sam years of his life. If the cadet gets through OK (and the instructors are evaluating each cadet along the way), he can choose to sign an ROTC contract or just say thanks but no thanks.
If young men were obligated to attend a basic camp like this after high school, the services could recruit from a very deep pool of prequalified candidates who already have, as the Krauts say, "fingertip knowledge" of whether the military life suits them.
http://www.tamukrotc.com/leaders_training_course.aspx
Steve - I was thinking of joining the Navy as an officer a few years ago (early 2000's) straight out of civilian life with a college degree(but didn't). The test they gave me had a regular standardized test component as well as a surprisingly number of tests designed to measure aptitude for flight, even though I had no interest in becoming a Navy Pilot. So if you want some data where people not interested in flight took flight-specific tests, check with the Navy instead of the Air Force.
ReplyDeleteBen H.
I agree that self-selection can be a large factor in who takes a test, but that effect will diminish once knowledge of that fact becomes general.
ReplyDeleteSome of these comments remind me of the old question, "what is the quickest way flying from (City A) to (City B), and the answer is a great circle, not the straight line that most people think of.
ReplyDeleteProblem is, this only works on mercator projections. On a polar projections, great circle routes are straight lines. (And even on a Mercator, due North and South great circles or East and West along the Equator are straight lines).
When I look at a horizontal line and tilt me head, it doesn't look as if the line moves at all. It just feels as if I tilted my head, and the line kinda stays as the reference. When I visualize the process, I 'see' the line move. I wonder if that's part of vertigo.
ReplyDelete