See if you can figure out the answer to the mystery posed in the following blog item by Kevin Drum of Mother Jones. Neither Kevin, nor the physics professor he quotes at length, nor Kevin's first 29 commenters can figure it out. Put your solution in my comments below. Kevin blogs:
Is Our Kids Studying? -- Take 2
After I posted a couple of days ago on the subject of whether or not college students are studying less than they used to, I got a long email on the subject from Paul Camp, a physics professor at Spelman University. This is pretty far outside my wheelhouse of expertise, but his take was so interesting that I wanted to repost it here just so that everyone would have a chance to comment on it. Here's what he told me:
I've been engaged in a few conversations about this in the past couple of years. I can offer the following data that correlates with anecdotal evidence from other professors at a variety of institutions.
Since the early 1990's, I have pre and post tested all of my introductory mechanics classes using a research based diagnostic instrument, the Force and Motion Conceptual Evaluation.
My first job out of graduate school was at an unranked tier 4 institution in Myrtle Beach, South Carolina. Coastal Carolina "University" to be specific. It was the 13th grade. There were a few brilliant students — I've learned that for a variety of reasons you can find exceptional students anywhere — but for the most part the student body was composed of people who were there for financial reasons or because they thought it would be a cool idea to go to school at the beach. The first four pages of our brochure described the beach, not the college. We knew which side our bread was buttered on.
I pretty reliably got 50-60% normalized gains on the FMCE.
Normalized gain is the ratio of how much their scores increased compared to how much they could have increased — (post-pre)/(100-pre). 50-60% is actually pretty stupendous on this particular measure. It means they were typically getting 80-90% of the questions right.
I left that job in a huff. There's a very long story, but the short version is that I was ordered by my dean to give everyone a passing grade and I wouldn't do it. I spent 5.5 years in a research position at Georgia Tech before coming to Spelman.
Spelman is a top 75 liberal arts college, according to US News, and top 10 according to the Washington Monthly. My personal impression of the students is that the average is generally much higher than it was at Coastal. These are students who can think around a few corners. Also, since they are able to cross register in some considerably easier classes at other AUC institutions, I tend to get classes of students who are there because they choose to be there and are therefore more engaged and thoughtful about their efforts.
I think I'm at least as good an instructor as I used to be, and probably a lot better. I know quite a bit more about developmental psychology and cognitive science as a result of my job at Georgia Tech and I think that improves my instruction considerably.
And yet, in a good year I get about 20-30% normalized gains.
I don't really know what is different but something clearly is.
Right now, I'm blaming No Child Left Behind, but that is less because of data than of general suspicion of high stakes testing. In fact, I am also now quite skeptical of pre/post testing (I could send you a research paper on that if you're interested) but not enough that I can account for the difference in the data. ...I can't really say that this is a correct account. I can say that many faculty I have spoken with have expressed similar observations without me prompting them, but the difference between me and them is that I have data. I know what I used to get at a crappy college with surfer students, and I know what I now get at a top tier college with highly engaged students, and it isn't consistent with ought to be happening, all other things being equal.
So that's my data point. I suppose I could always have had some kind of mental excursion and become a bad teacher without knowing it, but I don't think so and my students don't think so either, and neither do my peers in and out of the physics department. So I'm going to provisionally discount that explanation.
I left Coastal in 1998. I started at Spelman in 2004. You tell me what changed during that time frame.
Class, this mystery has baffled the best minds of Mother Jones magazine. Do you have any ideas how to solve this conundrum? Put them in the comments below.