February 21, 2013

NFL adds personality test to complement IQ test

From the NYT
N.F.L. Tries New Method for Testing Mental Agility 
By JUDY BATTISTA 
INDIANAPOLIS — For decades, hundreds of college players have gathered each year at the N.F.L.’s scouting combine, where their strength is tested, their speed is timed and, in a test to measure their intelligence, they are asked questions like “When a rope is selling 20 cents per 2 feet, how many feet can you buy for 30 dollars?” 
That query is part of the Wonderlic Personnel Test, a 12-minute, 50-item quiz that has been used by N.F.L. teams since the 1970s. It is, however, infamously unreliable in predicting football success — forgettable players have scored high, stars low —

Alternatively, you could say that it's amazing that the Wonderlic glass is a little bit full at all (when it suggests that, say, Tom Brady (124 IQ) might be smart enough to make himself useful).
and there have been quiet concerns that its reliance on knowledge taught in school might result in a racial bias.

Whereas tests that look like they come from Mars, such as the Raven's Progressive Matrices, don't have that problem!
So the players at this week’s combine are facing a new segment in their extended job interviews: an hourlong psychological assessment designed to determine and quantify the nebulous qualities that coaches have long believed make the most successful players — motivation, competitiveness, passion and mental toughness — and to divine how each player learns best.

Generally, you'd want to give the test to successful NFL players and washouts and see if it adds predictive power in differentiating them. There's no mention of whether that has been done.
The new test, like the Wonderlic, is mandatory for the more than 300 players who attend, and it will be given for the first time Friday. 
While many coaches and general managers consider the Wonderlic particularly useful in evaluating quarterbacks and offensive linemen, positions that are believed to demand the greatest intellect because of the need to decipher complex defenses, the hope is that the new test, called the Player Assessment Tool, will give teams clearer insight into a broader range of players. 
“I knew players who didn’t score well on the Wonderlic but had great instincts,” said Ernie Accorsi, a former Giants general manager, who was consulted during the creation of the new test. “I had a player once, this guy played in a good league in college, but the psychological testing indicated he didn’t handle pressure well. You know what? He didn’t, as it turned out. The Wonderlic can’t tell you that.” 
The new test was devised by Harold Goldstein, a professor of industrial and organizational psychology at Baruch College in New York. He worked with Cyrus Mehri, a lawyer in Washington who leads the Fritz Pollard Alliance, which monitors the N.F.L.’s minority hiring practices. 

Cyrus Mehri and the late Johnnie Cochran
Presumably, the NFL is interested in things such as the likelihood of going to prison for shooting yourself in a nightclub or of killing a teammate in a drunk driving crash. But all that has to be whitewashed through the proper channels; although in this case, Cyrus Mehri isn't black, but that hasn't stopped him from doing well off the diversity biz.
Personality tests have been a staple in other industries, and some N.F.L. teams have used them during their scouting efforts, which often take months. 
But last fall Goldstein and Mehri began the process of producing the first such test for the entire league. They asked a group of general managers what qualities they wanted in a player. They came up with 16 aspects thought to be predictors of N.F.L. success, including learning agility and conscientiousness. 
The test closely resembles those given to firefighters, Mehri said, because they, like football players, must be able to quickly assess a situation and decide how to proceed under stress. 
The goal was to eliminate the impact of prior knowledge — subjects taught in school, like math, in which racial and socioeconomic factors may have an influence. 
To determine their personalities, the test will ask players a series of questions about their preferences and behavior. To evaluate their cognitive abilities, it might tell them to look at four diagrams and figure out how they relate. Then, to measure how quickly they can adjust their thinking, the items they are comparing might change, forcing the players to determine their relationships anew. 
To see how they learn best, the test will present questions in verbal and graphic form. Players will have an hour to take the exam on a computer. 
... The league did not allow players and agents to see the test in advance, angering some agents. The N.F.L.’s goal was to minimize the kind of preparation that players do for the Wonderlic: reviewing past exams in an attempt to boost their scores. 
“This is the Super Bowl of their college career, the culmination of everything they have worked for,” the agent David Canter said. “You don’t want them to be prepared for it?” 
Damien Woody, a former offensive lineman for the New England Patriots, the Detroit Lions and the Jets, said he did not prepare for the Wonderlic, though he was determined to do well. He said others essentially shrugged it off, wondering what it had to do with football. 
At least in the first year of the new test, Woody said, the element of surprise could be a factor. 
“It might give you a sneak peek,” Woody said. “This will be the year it’s most beneficial because after this year I’m sure guys will try to train for it. This year, you’re going to get guys at their most vulnerable position.” 

Well said. The advantage of IQ tests like the Wonderlic is that the people who outsmart them tend to be pretty smart. Personality tests ... eh?

43 comments:

Anonymous said...

every so often there's an article telling us the Wonderlic failed to predict LaDainian Tomlinson's aptitude for finding and running through open spaces so maybe the NFL should stop using it. that's kind of interesting, but when will an intrepid journalist investigate the Wonderlic's ability to predict off-field shenanigans like by looking at, say, the percentage of total salary eventually paid in child support?

Steve, inquiring minds are looking in your direction.

Alice said...

Maybe Roid Rage is hard to predict when they have all steroids on their system, so they need a personality test to find the most likely to blow.

Or maybe they are just pro actively avoiding a disparate impact suit.

Assistant Village Idiot said...

"If your injury will take three months to heal but there are only two months left until the playoffs, how much do you need to increase your regular intake of PED's?"

Anonymous said...

"The goal was to eliminate the impact of prior knowledge — subjects taught in school, like math, in which racial and socioeconomic factors may have an influence."

You know, by NFL rule, each and every player at the combine has been in college for at least the last three years.

wwwww said...

fire fighter tests have not held up in nyc. valid for nfl though

Silver said...

I've got a feeling personality is going to matter as much to NFL performance as personality does when it comes to sexual attraction.

Anonymous said...

How does the NFL get away with the Wonderlic if it's understood by everyone to be an IQ test? I thought IQ tests by employers were prohibited?

Anonymous said...

How does the NFL get away with the Wonderlic if it's understood by everyone to be an IQ test? I thought IQ tests by employers were prohibited?

C'mon.

The new test was devised by Harold Goldstein, a professor of industrial and organizational psychology at Baruch College in New York. He worked with Cyrus Mehri, a lawyer in Washington who leads the Fritz Pollard Alliance, which monitors the N.F.L.’s minority hiring practices.

bjdubbs said...

I took a personality test for a temp job that included questions like "On a scale of one to ten, lying is always wrong" and "On a scale of 1 to 10, have you ever contemplated shoplifting." I picked 7 or 8 because I figured the liars would pick 10.

Anonymous said...

The NFL should give the Wonderlic and the personality test to players both at the beginning and the end of their careers. The results might be sobering.

Anonymous said...

Battista may not be the most quotable or side-splittingly hilarious sportswriter working, but just imagine how much worse this story could have been if bylined to one of the bros at Extra Mustard

class of 2003 said...

Fritz Pollard Sr. must be the last person graduated by Brown University with saleable work skills...

Anonymous said...

Do any other employers besides the NFL use the Wonderlic to screen job candidates?

Anonymous said...

Here's a page that lists some businesses and organizations that use the Wonderlic to screen job applicants:

http://www.wonderlic.com/resources/client-stories

The Clayton County Fire Dept. uses it and a representative from the dept. said that "I rely on the Wonderlic Classic Cognitive Ability Test because it gives me the best measurement of what we’re looking for in a trainee.”

http://www.wonderlic.com/client-stories/clayton-county-fire-department

I thought fire departments couldn't use IQ tests anymore after that thing in Connecticut?

ben tillman said...

Fritz Pollard Sr. must be the last person graduated by Brown University with saleable work skills...

Really? Class of 1919?

Chicago said...

If they want a psychological test that'll gauge mental toughness and determination then forget about the pen and paper stuff for which people can be coached. Make them demonstrate it. One possible test: kill and eat an alligator using only a bowie knife. That should weed out the unmotivated quickly. Other tests can be added, such as ordeal by fire, water,etc. Namby-pamby types need not apply.

Thursday said...

I wonder if you couldn't accurately test conscientiousness through making people do the most tedious stuff imaginable for a few hours and see who still gets the job done.

DYork said...

Is Cyrus Mehri a "Persian Jew"?

Shouldn't a black lawyer be doing his job and receiving the benefits?

Black Sea said...

When Bill Parcells was asked about the personality of Lawrence Taylor, specifically, and that Super-Bowl-winning team, more generally, he did point out that (approximate quote) "Well adjusted people don't play professional football."

DCThrowback said...

Couple of Wonderlic examples: I once took one for a small insurance company that was looking for salespeople. After the test, the office manager asked why I was here? are you sure you want do this? - he told me to go on a sales call - which forcefully showed me that I didn't have the stomach or conscious for suckering old folks into buying more supplemental insurance.

JP Losman, drafted in the first round by the Bills in '04, scored a less than impressive 17 his 1st time taking the Wonderlic. He re-took it a month or so later and scored a 31. The Bills GM at the time said the second score showed who Losman was, justifying the selection.

The GM was fired in 2008 and Losman finished his NFL career with a 10-23 overall record. The things was, it was noted that Losman had all the physical skills...but...was a bit of a rockhead. Huh.

FWG said...

I wonder what those Clayton County Fire Department scores are in 2013.

Pat McInally, the only known NFL player to score a 50, said he scored a 49 when he took it again in 2007, after a career in which he purportedly sustained six concussions.

http://en.wikipedia.org/wiki/Pat_McInally

Anonymous said...

In the link you provide, Cyrus Mehri states:

Any way you look at it, the fact that a top civil-rights attorney has commissioned a survey of diversity in ad agencies does not bode well for the agencies whose ranks are still overwhelmingly white.

The preliminary results of this study are showing shortfalls that are rare to have in this magnitude in this modern day and indicate purposeful discrimination.


He'll certainly soon investigate this firm with its overwhelmingly (...) attorneys.

Anonymous said...

http://on.cnn.com/YpyTtL

The skunk method

Anonymous said...

It may add a little something, but conscientiousness only correlates 0.25 to work performance and other traits much less than that. Meanwhile, the old-fashioned IQ correlates 0.6.

@Thursday,

The problem is that they may be highly motivated and act out of character. People with ADHD vary a lot in IQ for this reason.

The best way to sort out people likely to crash and burn is probably to look at their life history. Impulsiveness tends to get you in trouble from an early age.

Anonymous said...

http://blogs.the-american-interest.com/wrm/2013/02/22/californias-hollow-comeback/

‎"California has the nation’s highest poverty rate despite being the wealthiest state."

Rich get richer , poor get poorer.

Anonymous said...

http://www.youtube.com/watch?v=uv4yz1v8lIA&list=UU4EY_qnSeAP1xGsh61eOoJA&index=8

Rise of the mulattos

not-so-black yammering about black

Anonymous said...

One would think that years of playing football would be predictive of the personality types that do well at football.

Frank Layden said...

Karl Malone was not drafted until the second half of the first round of the 1985 NBA draft because he did so badly on the personality test, but he turned out to be the greatest power forward in the history of the league.

Anonymous said...

http://www.thedailybeast.com/articles/2013/02/21/america-s-new-mandarins.html

The people who pass these sorts of admissions tests are very clever. But they're also, as time goes on, increasingly narrow. The way to pass a series of highly competitive exams is to focus every fiber of your being on learning what the authorities want, and giving it to them. To the extent that the "Tiger Mom" phenomenon is actually real, it's arguably the cultural legacy of the Mandarin system.

Anonymous said...

"Do any other employers besides the NFL use the Wonderlic to screen job candidates?"

Fortune 500 companies use it routinely to screen candidates for executive positions. Also, some executive talent placement agencies require you to take it; they use high scores to promote their clientele.

Anonymous said...

The San Diego Chargers contracted with a company that my brother works for to do personality testing on Michael Vick and others before the 01 draft. The Chargers found the results very helpful and passed on using the 1 pick on Vick, instead trading it to Atlanta. There were contract number issues that factored in, but my bros company's results were important. Then SD drafted Tomlinson with the 5th pick and nabbed Drew Brees first in the 2nd round. They, the Chargers, wished they had done the same with Ryan Leaf in '98.

Anonymous said...

"While many coaches and general managers consider the Wonderlic particularly useful in evaluating quarterbacks and offensive linemen"

Funny... positions that tend to be white.

FWG said...

I'd imagine Karl Malone was one of the more conservative black NBA players.

Anonymous said...

Just ever-so-slightly off-topic, but there's a new brother/brother combo to add to the iSteve Pantheon of Fame [following in the footsteps of Dick & Tom Van Arsdale, Eli & Peyton Manning, Jim & John Harbaugh, etc etc etc]:

Tyler & Cody Zeller, now both 1st Team Academic All-Americans in NCAA basketball!!!

And their third brother, Luke, has appeared in 16 games so far this season for the Phoenix Suns.

Anonymous said...

The new test was devised by Harold Goldstein, a professor of industrial and organizational psychology at Baruch College in New York. He worked with Cyrus Mehri, a lawyer in Washington who leads the Fritz Pollard Alliance, which monitors the N.F.L.’s minority hiring practices.

Somebody mis-spelled MacGoldstein and MacBaruch and MacMehri and MacPollard.

Anonymous said...

Fritz Pollard Sr. must be the last person graduated by Brown University with saleable work skills...

Really? Class of 1919?


Tukey graduated in 1936, and he discovered [or at least re-discovered] the Fast Fourier Transform [among many other things].

BTW, he was home-schooled by his mother, Adah Tasker Tukey, who was valedictorian at Bates College in 1898 [his father, Ralph H. Tukey, was salutatorian that year].

For all intents and purposes, she quit her career in order to concentrate on tutoring her son.

Anonymous said...

Rich get richer , poor get poorer.

Illiterate illegals are imported by the rich in the interests of cheap lawn care.

Anonymous said...

Re: Karl Malone


He is an interesting case. He truly did seem to reform his life and adapt perfectly to Salt Lake City. Say what you want about gays is SLC you can't go around knocking up 13 year olds for very long there.

Anonymous said...

Does the U.S military record the average IQ of special forces personnel?

Anonymous said...

Does the U.S military record the average IQ of special forces personnel?

They'd certainly keep track of that. Because of their high cost of training SF are intensely studied.

Anonymous said...

Maybe off-topic, but that picture of Johnie Cochrane made me think back to when that verdict was read about O.J. Simpson.

Now Cochrane is dead and Simpson is in prison.

Truth said...

"Maybe off-topic, but that picture of Johnie Cochrane made me think back to when that verdict was read about O.J. Simpson."

Not off topic, off century.

Anonymous said...

NFL IQ test: Spell "duh"