Statistics professor Andrew Gelman has a long, lovely reflection on his late friend Seth Roberts, a genuine American original who pioneered the practice of turning yourself into a human guinea pig for various eccentric self-improvement ideas -- e.g., lose weight by drinking a glass of sugar water one hour before each meal (that actually worked for Roberts) -- and then carefully recording the data on what happens.
The issue is that we need a way to combine the best of Roberts' idiosyncratic self-research and real experimental studies.
Here's a methodological suggestion using a personal example. I strongly believe that echinacea tea helps me head off colds -- if I get run down and develop a sore throat, which for me is always the precursor of a ten-day long cold, several glasses of echinacea tea will make the sore throat go away over 50% of the time. But that doesn't seem to work for most other people, even my sons. I'm not surprised -- immune systems are highly variable from person to person.
Standard studies of echinacea have come up with mixed results. That's hardly surprising. If echinacea works for, say, 2% of the population, you need a massive sample to see a statistically significant result. And maybe it makes 2% of the people worse off, so there is no net effect on the population at all. But, even in that case, it would be good if echinacea were used by the 2% if benefits and avoided by everybody else.
But why not do a traditional experimental study, but only on people who believe echinacea benefits them? I'd sign up for such a study.
If echinacea actually works on a lot of the people who claim it works for them, then you can do another study attempting to find out how generalizable it's effects are. I'd next try to recruit first and second order relatives of those for whom it worked in the first study. This should give us a way to estimate how idiosyncratic the effects are.
Then, look for the genes that make some relatives respond and other relatives don't. In a generation or so, you ought to be able to get your whole genome analyzed and be handed a list of Seth Roberts-type things to try that have a decent possibility of working for you. After all, most of us don't have the energy to be Seth Roberts.
Then, look for the genes that make some relatives respond and other relatives don't. In a generation or so, you ought to be able to get your whole genome analyzed and be handed a list of Seth Roberts-type things to try that have a decent possibility of working for you. After all, most of us don't have the energy to be Seth Roberts.
The simpler protocol would be to use a random sample but repeat the group over multiple trials. Measure the correlation within subject across trials. If the top responders on the first trials tend to be the top responders on subsequent trials, then that tells you that people do tend to vary in individual responses. If there is no subject-specific correlation then that tells you the variance is entirely attributable to noise.
ReplyDeleteBy the way, this concept is already well studied among exercise high and low responders. It's generally well known that about 20% of the population has a very positive physiological response to exercise, while 15% has actually zero or negative response.
http://www.livescience.com/6058-genes-explain-fit.html
You'd probably end up selecting people who have a strong response to placebo, regardless of what actual effects of treatment were.
ReplyDeleteI wonder if you could eliminate that effect by testing for hypnotic potential. Hmmm.
It's a waste of time to run a study
ReplyDeleteHow many people want to waste time out of their lives to do a study to prove what we already know...
It works on some people and it doesn't work on others. Test it out once yourself.
RIP Seth.
ReplyDeleteSeth was a very interesting man. He was far more than just interesting ideas on self-improvement. He had a different way of viewing the world, of questioning and analyzing. A shame that more people did not follow him; most of us end up falling in line, when the world is so much more interest from new vantages.
Diet strikes me as one area where self-experimentation is vital. I've tried numerous diets over the years, but the only one that has really worked for me is one that is high in fat and protein and low in carbs. My sister, in contrast, tried a low carb diet and was miserable. She felt tired and weak. Her brain seemed to be in a fog.
ReplyDelete"How many people want to waste time out of their lives to do a study to prove what we already know..."
ReplyDeleteBecause if it's not yet proven, we don't know it - by the rational rules of evidence. Conviction is not knowledge.
It wasn't just that Seth thought academic research was useless, it was that only useless things were considered academically prestigious. So figuring out how to prevent cancer got zero dollars, but doing research on telomeres was worthy of Nobels, even though the research was useless.
ReplyDeleteYour immune system reaction here might correlate fairly strongly with your cancer history.
ReplyDeleteBTW, apparently they're getting pretty good now at breeding and/or training dogs to SMELL cancer in humans.
Which is pretty fringing awesome on the part of Man's Best Friend.
What happened to him? Was he ill? It's a bit mysterious.
ReplyDeleteDid you realize that he was 61? He looked young for his age.
He was an amazing person and will be missed.
"What happened to him? Was he ill? It's a bit mysterious.
ReplyDeleteDid you realize that he was 61? He looked young for his age. "
That picture is pretty old, and I've seen claims from people claiming to know him that he gained quite a bit of weight the last few years.
Many of the comments about him on Marginal Revolution were rather gormless.
ReplyDeleteIt eventually struck me that there's a strand of opinion, perhaps especially in America, that approves of group diversity but not of individual diversity. Thus, it's good that there a white people and black people. (It's better that there are black, no doubt, but still.) But diversity within, for example, blacks should be deprecated. Hence the scorn and hatred directed at, for instance, blacks who have conservative political views. Thus the hatred of any notion of diverse social or political views among, say, students.
(Seen somewhere this week, a joke that appealed to me. Q: "What's the opposite of diversity?"
A:"University")
Seth was, as it were, individually diverse. Bless him.
It eventually struck me that there's a strand of opinion, perhaps especially in America, that approves of group diversity but not of individual diversity.
ReplyDeleteSort of the Star Trek view of race, which does seem fairly prevalent. Klingons are violent and warlike, Vulcans are logical and controlled, Ferenghi are greedy and conniving, etc. So you can mix them together to get lovely diversity at your school or dinner party, but you still know what to expect when you sit down with an individual. I think you're onto something.