July 9, 2011

The Moneyballization of college admissions

Steve Hsu points to an article from Chronicle of Higher Education on how college admissions departments are, like baseball teams in the 1990s and basketball teams in the 2000s, increasingly hiring quants to statistically manage the admissions process. 
Those Tweedy Old Admissions Deans? They're All Business Now 
A profession that once relied on anecdotes and descriptive data now runs on complex statistical analyses and market research. Knowing how to decipher enrollment outcomes is a given; knowing how to forecast the future is a must. Which students are most likely to apply, submit deposits, and matriculate? At what cost to the college? How likely will they be to graduate? Such questions echo in the modern enrollment office, which is often supported by one or more institutional researchers, as well as consulting firms that sell recruitment strategies in various flavors. 
Search the job listings for top-level admissions and enrollment openings, and you will find that many colleges seek a "data-driven" leader, someone who will develop "data-informed" strategies. This past winter, for instance, Pomona College, in California, began a national search to replace Bruce J. Poch, who had stepped down after 23 years as vice president and dean of admissions. Among the qualifications listed in the job advertisement: "an ability to analyze and use data to guide decision-making and measure results." 
David W. Oxtoby, Pomona's president, led the college's search committee. The modern admissions dean, he says, must have a "technical, quantitative facility," the ability to delve into the relationship between a student's SAT score and her subsequent performance in college, or why some kinds of students are more likely to enroll than others. Moreover, Pomona had decided to merge its admissions and financial-aid offices (a change many colleges have made already). So the new dean would need to speak the language of costs.

Poch was the one admissions professional I've met who really impressed me. The lower levels of the business are full of nice young ladies (a large fraction middle class black) and nice young gay guys who couldn't get better jobs, but Poch definitely seemed like The Brains Behind the Operation. 

And clearly there are some brains somewhere, because American prestige education has been one of the big success stories of the last generation, at least at maintaining and enhancing its prestige. Granted, this is awfully easy to do in a business driven by brand names where the older the brand name the better, but only a handful of colleges in recent decades have screwed up so badly that they've fallen notably in prestige.

Digression Alert: I also once spoke on the phone to Poch's boss, David Oxtoby, now the president of Pomona college. Around 1990 in Chicago, I lived in a six-flat that had "The Oxtoby, 1923" carved over the front door. One day I started wondering who or what an Oxtoby was. So, I looked in the Chicago phone book (remember phone books?) and there was only one Oxtoby in Chicago: David Oxtoby, Hyde Park. So, I called the number and a man with superb diction, like a college president's speaking voice, answered. I asked him if his family had built The Oxtoby. He said no, but he kindly gave me a full, scholarly etymology of the name Oxtoby (it means oxen-go-by, kind of like "Oxford" is where the oxen forded the river). 

I never asked Mr. Oxtoby what he did, but I wasn't surprised to read in the newspaper years later that this Hyde Park resident with distinguished diction had been appointed president of a famous liberal arts college.

Being a college president is a tough job. Basically, you need to be Angelo Mozilo on the inside, a pushy Alec Baldwin in Glengarry Glen Ross-style salesman always always asking for a donation. On the outside, though, you need to come across like Mr. Chips, an unworldly scholar who makes guys who've made a pile of money in real estate or metal bending (and their wives) feel classy by associating with you and giving you money. Elegant diction helps.
Pomona interviewed a dozen candidates before hiring Seth Allen, dean of admission and financial aid at Grinnell College, in Iowa. Soon to occupy one of the premier jobs in admissions, Mr. Allen, 43, represents the next generation of enrollment chiefs. They've ascended during an era of high competition, learning how to market their colleges and massage the metrics that define success in admissions. 
Although idealism may inform their work, they are clear-eyed realists. They are not introverts, for they must collaborate constantly with faculty members and other campus offices. They are diplomats who must manage competing desires: those of administrators who want to enroll more first-generation and low-income applicants, professors who want more students with high SAT scores, trustees who want to lower the tuition-discount rate. "Twenty years ago," Mr. Allen says, "there were not as many wants." 
Drawn to statistics at an early age, Mr. Allen earned a bachelor's degree in economics at the Johns Hopkins University in 1990. He first worked as an admissions counselor for his alma mater, a cutting-edge laboratory in the then-burgeoning science of enrollment management. Mr. Allen learned how predictive modeling could project net tuition revenue, how many biology majors would enroll, and a hundred other outcomes. 
Later Mr. Allen became the university's director of enrollment planning, research, and technology, a title that captured the profession's increasing sophistication. While most colleges were still operating in a pen-and-paper world, he helped create the university's first online admissions application. 
In the 1990s, selective institutions intensified their recruitment of prospective students, and Dickinson College was no exception. As dean of admissions at the Pennsylvania college, Mr. Allen oversaw a surge in applications that enabled it to become increasingly selective. Average SAT scores rose, as did enrollments of minority students.

Dickinson did two smart things. One, it made submitting SAT/ACT scores optional but reported to USN&WR the scores not of all freshmen but just those who felt like submitting their scores. So, it could let in more rich dumb kids and the like without taking a hit in the USNW&R rankings. Also, Dickinson appears to have a policy that any time a reporter, especially one from the New York Times, needs a quote about college admissions, a high-ranking Dickinson official will supply one at any hour of the day or night.

What's not mentioned in this rather starry-eyed article is that holy grail of admission moneyballers: alumni donations. Which type of applicant is most likely to donate the most to his alma mater? As far as I can tell, the big givers tend to be whites and males and legacies and jocks -- i.e., basically, the same kind of people they were most likely to admit back in the Bad Old Days. Ironies abound.

Moneyball turned out not to be a very big deal in baseball. A few underdogs did a little better for awhile, but ultimately the Yankees wound up back on top. I doubt if better statistical analyses of college applicants will change much in the visible world. When all the dust settles, Harvard will still be on top. But it would be very interesting to read those confidential analyses.

Have any college admissions moneyball stats ever been released to the public?

I like baseball statistics a lot, but the amount of public brainpower devoted to analyzing baseball statistics in contrast to real world statistics is striking. Brad Pitt is starring in Moneyball this year, but I've never even seen an economic professor's paper on the topic of what kind of college applicants give the best financial return to the college. These analyses exist, and I bet ambitious parents in Seoul are reading some of them translated into Korean right now, but Americans don't seem very interested in the topic.

33 comments:

Nanonymous said...

All this is part of the subversion of the idea of college as a societal good. Today colleges are corporations run like corporations for the benefit of investors and top management (professors).

Al said...

"Today colleges are corporations run like corporations for the benefit of investors"

Colleges don't have investors (except for bogus for profits like U Phoenix). They run some programs in a revenue seeking fashion, but on the whole they aim to maximize prestige for their core undergraduate and phD programs.

Anonymous said...

Oxtoby was a chem professor, later dean of physical sciences at UofC. He should do his own number crunching.

Nanonymous said...

Colleges don't have investors

Rrright. Ever heard of donors? Steve has written here about $5M Harvard rule (or smth along those lines - search for it). Various "technology transfer" arrangements with industry? There are a lot of people with deep investment in the success of this or that university. Formally speaking, the main investor is general public, a.k.a. taxpayers. Because every research university today is fed by federal money (that applies equally to private and public universities). But public interests in operation of universities are subverted in the same way as they are in governments.

but on the whole they aim to maximize prestige for their core undergraduate and phD programs.

That's the official line that you swallowed along with hook and sinker. The truth is that the maximization of the prestige (note: NOT the actual quality; prestige only) is one of them main ways of maximizing what really matters - management's profits. University are run by professors for professors and the results are not pretty (to give only one example).

Diminishing Power of Professors said...

University are run by professors for professors

No. Like other non-profits, the management of universities and college has become more specialized and focused on the business side. This often puts them at odds with the education and research side of higher education pushed by professors.

The wages of college and university presidents have raced ahead of that of typical professors and much more so than the growing cadre of TAs, RAs and non-tenured visiting/temp teaching positions. Same as has happened in most industries in general.

I also suspect that the top admins running universities and colleges are more outside guns hired for the business experiences/interests than home-grown and/or academic risen through the ranks via scholarship. Some are even hiring complete non-academics in some of these admin positions that were traditionally held by academics only.

Anonymous said...

Digression Alert: I also once spoke on the phone to Poch's boss, David Oxtoby, now the president of Pomona college. Around 1990 in Chicago, I lived in a six-flat that had "The Oxtoby, 1923" carved over the front door.

Back in 1999 I used David Oxtoby's chemistry textbook for freshman chem. I think it's a pretty commonly used book. Strange how I immediately recognized the name, even after all these years.

stari_momak said...

Academia is extremely strange. Most actual academics do administrative chores (e.g. chairing the department) out of a sense of noblesse oblige. Anything 'above' that is strictly for failed academics. Prestige is measured in articles published and their impact (i.e. citations).

So we end up with some strange situations. For example, if I were a prospective PhD student in the social sciences, I'd much rather be accepted to Michigan than Harvard. If I really was an 'area studies' geek I'd take Indiana over Harvard any day. There are a myriad examples.

As for undergrad -- the smartest kids in my high school class -- admittedly decades ago, went to UC Santa Cruz or USC. Go figure.

Anonymous said...

Moneyball worked well for the Boston Red Sox, but eventually everyone was doing it so the advantage cancelled out. So it shifts to the next field or sport (soccer) where it can give early adopters an advantage.

Anonymous said...

Ever hear of the Tampa Bay Rays? They've gone from cellar dwellers to contending in the AL East by implementing Moneyball strategies. Are teams still able to achieve arbitrage gains in the market for high on-base percentage hitters? Of course not--that secret's out. But it doesn't mean that quantitatively focused teams aren't looking for (and finding) the next Moneyball-type insights. Most recently, teams have been focusing on defense--see, e.g., Billy Beane's recent A's lineups.

Nanonymous said...

@Diminishing Power of Professors said: You are right. All these things are happening. Which doesn't really contradict what I was claiming (corporatization of the business side of higher education). All it means is that corporations are restructuring so that universities are run by some professors for some professors.

Anonymous said...

@stari_momak: Your picture of academia is decades out of date. You'd be a fool to attend grad school at Michigan or Indiana over Harvard these days.

Reg Cæsar said...

Perhaps Oxtoby is an exception, but -by almost always refers to a settlement in the Danelaw, being the suffix used to this day in Scandinavia, and means "town".

Cottle's surname dictionary doesn't list it-- he has an Oxby, however-- nor does the AA's road atlas, which shows every flyspeck in Great Britain, including New York, Lancashire.

Incidentally, Simon Kuper recently did a big piece for the Weekend FT about how Moneyball is revolutionizing English soccer. Beane himself is involved, taking calls at odd hours of the morning.

Nate said...

Just curious, which colleges would people say have fallen in prestige, due to poor leadership, over the last few decades? Brown is an obvious one. I suppose you could say CCNY, due more to a change in admissions policies and city demographics. Not sure who else would be on this list.

Jack Aubrey said...

Half-assed moneyball strategies have existed for some time. About a decade ago Gordon Gee, then the chancellor of Vanderbilt University, proposed discriminating in favor of Jews because they, among other things, were supposedly more likely to become bigger donors. Gee's efforts increased the Jewish share of Vandy's student body from 3% to 15%.

Anonymous said...

Universities are run by their boards of Trustees the same way that corporations are run by their Boards of Directors.

The trustees are essentially focused on growing the prestige of the University.

The trustees work hard to grow the endowment not simply so that the school can have more money but so that the school can spend on things that add to its prestige.

What is interesting is that the "white survivalist" crowd often says that they, the white survivalists, are unhappy that so many of the white students at the ivies are jewish. At the same time the white survivalist crowd says that they are unhappy that so many of the students at the ivies are asian.

Statistics show that, apples to apples, jewish students go on to donate dramatically more than white christian students.

And asian students go on to donate dramatically less than white christian students

So it is logical for the trustees to listen to the "white survivalists" and accept fewer asians, but it is logical for the trustees to IGNORE the white survivalists and accept plenty of jews

Anonymous said...

Here's the thing: whites think of academics as a priest caste who have taken a vow of near poverty in order to serve as purveyors of higher truth.

Rich profs and Uni's rollin' in dough profane the temple of knowledge.

Anonymous said...

Here's the thing: whites think of academics as a priest caste who have taken a vow of near poverty in order to serve as purveyors of higher truth.

Rich profs and Uni's rollin' in dough profane the temple of knowledge.

Anonymous said...

"... but I've never even seen an economic professor's paper on the topic of what kind of college applicants give the best financial return to the college. These analyses exist, and I bet ambitious parents in Seoul are reading some of them translated into Korean right now, but Americans don't seem very interested in the topic."

That's because whites not only aren't interested in making themselves over to appear to be something they're not, but because whites mistrust ambition as a motive in just about any pursuit. The underlying cultural belief is that If you have to strive to fit in at a university or in a socioeconomic group, it's a sure sign you don't belong there.

Anonymous said...

The University of Chicago is an example of a school that has declined in prestige and relative endowment size due to being overly rigorous and high minded.

Anonymous said...

Hey, I realize you write these blog posts for free, but just because you've "never even seen an economic professor's paper on the topic" doesn't mean there haven't been dozens released in the last few years. Start with Richard Vedder.

As to whether a concrete ROI attains to a prestigious degree, the answer is yes, if you're black or Latino. For the rest it's conspicuous consumption and haute money laundering of dubious benefit. Also your view of a modern college president vs. admissions dean is laughably out of date. Trustees and faculty "government" are more powerful than in Charles Eliot's day (and they had power back then). Presidents are figurehead fundraisers with people skills and some fig-leaf doctorate. Mozilo doesn't come to mind. Expect them at 70% female by the end of the decade.

Admissions execs, while certainly banking less than the football coach, are subjects of intense scrutiny to a degree unimaginable in 1950 Palo Alto or New Haven. The ones from the current breed burn out faster than the pre-moneyball types (Fred Hargadon would be an example of the latter). It's a long string of hassles with mediocre remuneration, and a lot less fun than being the athletics director.

Steve Sailer said...

"Hey, I realize you write these blog posts for free, but just because you've "never even seen an economic professor's paper on the topic" doesn't mean there haven't been dozens released in the last few years. Start with Richard Vedder."

I'm familiar with Vedder's work on whether college is worth it to students and society as a whole. What I'm interested in here is what kind of applicants are most worth admitting by universities -- that's the kind of the behind-the-scenes stuff that this article refers to, but I've seldom seen published.

map said...

"Oxtoby was a chem professor, later dean of physical sciences at UofC. He should do his own number crunching."

I believe that is Eugene Oxtoby, not David Oxtoby. EO taught chemistry and they used his textbook.

Reg Cæsar said...

Forgive my heat-stroke cerebroflatulence above-- New York is in Lincolnshire, just up the road from much larger Boston.

David said...

>The underlying cultural belief [of whites] is that If you have to strive to fit in at a university or in a socioeconomic group, it's a sure sign you don't belong there.<

Don't know if that really is the underlying belief, but it sounds like a wise one. The world is surfeited with stressed-out lying phonies who muscled their way up to their level of incompetence.

Diminishing Power of Professors said...

The University of Chicago is an example of a school that has declined in prestige and relative endowment size due to being overly rigorous and high minded.

The University of Washington, St. Louis is a counter example to UofChi.

The deep pockets of UWash enabled them to sweep up many very talented and motivated Ivy rejects who lacked a hook (legacy, minority, athlete, woman in STEM, etc). UWash rose in the ranks accordingly.

Stanford's focus on STEM talent enabled them to also buck historical trends and rise up the ranks over the past several generations. A necessary historical requirement was the rise of tech and the finance behind it since the 1960s.

The problem with UofChi is that they focused primarily on uber-nerds in the liberal arts whose most likely prestigious career outcome is becoming a famous professor in the liberal arts.

Yale has fallen behind Harvard and allowed Princeton to challenge it by focusing on spawn of the elite and others who are the most liberal. Princeton, meanwhile, focused on admitting families (students by proxy) striving for real world power.

There is a purported trend among elite Liberal Arts Colleges to sweep up powerful Jewish families (students by proxy) who are losing Ivy slots to Asian kids raised by Dragon Moms.

Diminishing Power of Professors said...

To understand how moneyball is played by colleges, you have to understand what the the metrics of a successful college are as measured by it's alumni:

1) Endowment/Fundraising
2) Power/Influence Realworld
3) Power/Influence Cultural
4) Ranking(s)

Note that education/learning are not on the list. That is an increasingly marginalized outcome to the process as measured by lowered core requirements, reading/essay demands and grade inflation.

Ranks are last because they are often subjective, purposedly scrambled for variety by the likes of US N&W Reports and ultimately are limited by the endowment. Institutions like Stanford, Duke and WashU show how newfound wealth or an early focus on cultivating wealthy families is ultimately reflected in higher rankings.

NYU's rise is because they also have increased cultural and realworld influence due to their great location in NYC and AVIS-like #2 we try harder vis-a-vis Columbia.

On the other hand, institutions that have willfully failed to focus on the fundamentals of College moneyball have suffered by comparision. Yale, Columbia and UChi.

Yales endowment has fallen further behind Harvard while Princeton and Stanford catch up. Where Columbia used to be clearly the next university after HYP - it is often nosed out by the likes of Stanford, Duke, Brown, CalTech, etc depending on a student's interest.

Diminishing Power of Professors said...

The most interesting cases are the relatively weak academic colleges that are nonetheless highly ranked due to endowment or power in the real world. For example, compare the current US N&W Report rank (USN#) with an Australian ranking of High Impact Universities (AHIU#) (likely similar to other non-US rankings like those in Asia or purely US academic rankings like Gorman's Report):

College/U USN# / AHIU#
========================
Dartmouth 9/106
Notre Dame 19/133
Georgetown 21/160

There is some explaination in that smaller and less research-focused colleges are penalized, but these are 3 have the largest gaps by far of the top 20 USN which can easily be explained by the 4 moneyball criteria listed:

Dartmouth - powerful and wealth alumni network

Notre Dame - less powerful but rabidly loyal alumni network

Georgetown - deep network into the DC governement network

Conversely, the biggest gaps between the highest ranking AHIU# and lowest ranking USN# are tend to be public universities:

UC Berkeley 5/22
UC Los Angeles 4/25
UC Michigan 6/29
UC San Diego 10/35
U of Minnesota 12/64
U of Wisconsin 18/45
Penn State 20/47

Arbitrage opportunity: short the typical graduate from overranked private institutions and long the honors graduate from underranked public ones.

not a hacker said...

Stari - sorry but I didn't follow. Could you explain the connection between your 1st and 2d paragraphs?

Hacienda said...

"That's because whites not only aren't interested in making themselves over to appear to be something they're not, but because whites mistrust ambition as a motive in just about any pursuit. The underlying cultural belief is that If you have to strive to fit in at a university or in a socioeconomic group, it's a sure sign you don't belong there."


Actually, I'd say this is truer of Korean culture. Koreans really mistrust ambition to the point of stasis. Which is why they need artificial stimulants like degrees and university prestige and other such motivations to get them going.
It's fundamentally a Buddhist/Confucian culture. Not the clown Christianity that has warped the West beyond recognition.

Al said...

"which colleges would people say have fallen in prestige, due to poor leadership, over the last few decades? Brown is an obvious one. I suppose you could say CCNY, due more to a change in admissions policies and city demographics. Not sure who else would be on this list."

Brown? Brown's dropped a bit on US News, but not by much. Their acceptance rates are way lower than they used to be.

Cooper Union and CUNY are the obvious examples of schools that have declined. Perhaps a case can be made that Berkeley has also declined.

Diminishing Power of Professors said...

which colleges would people say have fallen in prestige

Cooper Union and CUNY are the obvious examples of schools that have declined. Perhaps a case can be made that Berkeley has also declined.


One would expect National Public Universities to be hit the hardest economically since the collapse of the Internet (2001) and Real Estate (2008) bubbles in addition to the increasingly winner-take-all effects of hyper credentialism in the US.

Finances are a major indirect measure of USN&WR rankings (faculty pay, faculty:student ratio, financial resources, etc) so much more highly ranked academic public universities have always been ranked below less academic but wealthy private universities in the USN&WR ranking.

The Internet Archive Way Back Machine goes as far back as the 1999 US N&W Report Ranking.
The 2006 Ranking is here. The current ranking can be found at the US News website.

USN&WR Ranking of the Top 5 Public National Universities by pub date:
1999 2006 2011
UC Berkeley 22 21 22
UCLA 25 26 25
Univ VA 22 22-25 25
U Mich 25 22-25 29
UNC ChpHll 24 >26 30

I believe these rankings were similar to when I entered college in just after the USN&WR ranking started in 1983.

There has been virtually no change in the USN&WR ranking of the top two UC schools but a small but measurable drop in the other top national public universities (VA, MI and NC Chapel Hill).

BTW, USC surpassed UCLA for the first time this year.

Diminishing Power of Professors said...

Just found the 1997 published USN&WR rankings - the oldest archived at the Way Back Machine:

21 U of VA
24 U of MI
25 U of NC Chapel Hill
>25 UC Berkeley, UCLA

Which means that UC Berkeley and UCLA have actually risen while the other top public Unis have fallen 1997-2011.

Diminishing Power of Professors said...

Other significant changes 1997-2011 assuming both years are not an aberrations from the trend and ignoring any change <=3 slots:

In 1997, all universities ranked 51-100 are lumped together. Assume any college breaking from this group 51-100 into the top 50 did so from an avg 1997 rank of 66 (at the top 1/3 of the 2nd group of 50):

Risers:
=======
+25 Rensselear Polytech (avg66->41)
+25 UC Irvine (avg66->41) est
+21 UT Austin (avg66->45) est
+20 USC (43->23) CONFIRMED
+19 Penn State (avg66-47) est
+19 U of Miami, Fl (avg66-47) est
+8 U Penn (13->5)
+7 Columbia (11->4)
+7 Boston Col (38->31)
+7 GA Inst of Tech (48->35)
+7 UC Santa Barbara (49->36)
+6 UCLA (31->25)
+5 UC Berkeley (27-22)
+4 WashU StL (17-13)
+5 Carnegie Mellon (28->23)


Fallers:
========
-15 Tulane (36->51)
-15 TX A&M (48->63)
-7 Brown (8->15)
-7 U of Rochester (30->37)
-6 Tufts (22->28)
-5 Duke (4->9)
-5 UMich (24->29)
-5 Brandeis (29->34)
-5 Lehigh (32->37)
-5 UNC Chapel Hill (25->30)
-5 Yeshiva (45->50)
-5 GeoWash U (46-51)
-4 UWisc Madison (41->45)

Generalities among Rising Unis:
------------------------------
* STEM emphasis
* Sunny and/or nice weather
* Near hip urban center
* East or West Coast or Southern Town with Northern/Foreign Dominance