Friday, July 9, 2010

The China Study: Crushed by its Own Data

You may have already seen this outstanding analysis of the data from "The China Study". If you haven't, I highly recommend you give it a read. It's long, but well worth the effort. Readers of this blog know my opinion of T. Colin Campbell and his "scientific" work. Now somebody has taken the time to actually crunch the numbers, using Campbell's own data to demonstrate that his conclusions are baseless (at least when confined to this data), and probably the result of confirmation bias.

I also love the observation that, despite his constant whining about the "dangers of reductionism" in science, Campbell's entire argument against animal protein really hinges on a strongly reductionist experiment, namely the isolated effect of casein fed to rats in large doses. Snap!

Readers know of my criticisms of classical statistics, but it should be noted that I don't really have a problem with the mathematics, but the application. Math is what it is, either right or wrong. My issue is that classical statistics is used incorrectly, to draw inferences about hypotheses, when the underlying mathematical framework has nothing to do with inference. The key problem is that "statistics" are just numbers derived from data, like correlations. They don't say anything about a hypothesis: you will calculate the same correlation between two datasets, regardless of your hypothesis about what causes that correlation. Anyway, I don't want to get off on a rant. My point here is that the author, Denise Minger, does an excellent job of confining her analysis and conclusions within the bounds of what classical statistics can tell you. And along the way, she does a great job of demonstrating how easy it is to fool yourself (as T. Colin Campbell did - repeatedly) by over-interpreting these numbers which, in the end, cannot tell you anything more than what's in the data.

Ms. Minger has also done a great service in providing a concrete example of the issues in observational studies. You've likely read often that epidemiological studies are of little use in distinguishing between competing hypotheses. Now you have an example, replete with numbers. Ms. Minger demonstrates in several cases how a seemingly "obvious" conclusion vanishes once you dig into the large number of uncontrolled variables inherent in all observational studies. It's easy to find correlations in large datasets with many uncontrolled variables. The problem is that people take these correlations to mean more (or less) than they really do in terms of supporting/undermining a particular hypothesis, and the conclusions they draw are essentially ad hoc, not based on any rigorous mathematical analysis, but rather hand-waving about what is "obvious". An oft-quoted example is that men who shave daily have a higher incidence of heart disease. It is "obvious" that heart disease is not caused by shaving, right? Or is it? There's a whole lot of other information that goes into that judgment. We generally take this sort of thing for granted, especially when made in pronouncements from "esteemed" scientists like T. Colin Campbell. But if you dig into the reasoning behind these conclusions, you generally find a tangled web of assumptions, hypotheses assumed to be true, but which have varying (if any) actual evidence to support them. Ms. Minger does a great job of teasing these out of Campbell's reasoning, and demonstrating how the data itself provides little evidence one way or another, precisely because it cannot distinguish between the potential effects of the many intertwined and uncontrolled variables.

Anyway, enough of my babbling. Go read the article, you'll be glad you did (unless you're an uncritical fan of T. Colin Campbell, in which case you've got bigger problems).

30 comments:

praguestepchild said...

I thought it was the men who shaved less frequently that were more likely to have a heart attack? http://aje.oxfordjournals.org/cgi/content/full/157/3/234 Also, countries with more telephone poles tend to have a higher GDP. Obviously we could help out a lot of third world countries by donating telephone polls.

Dave said...

@praguestepchild,

Darn, there goes my excuse for not shaving.

praguestepchild said...

Just double down on the Lipitor and you'll be fine.

durianrider said...

Gday crew,nice blog.
How come NONE of these pro meat bloggers have any real muscle with all that protein talk? :)

Come and see if ANY of you guys can out bench press/dead lift us at
www.veganbodybuilding.com
www.veganstrength.org
www.organicathlete.org

Here is the website for the doubters.
www.pcrm.org

Mike Arnstein ran a 2:28 marathon this year at Boston. He is the FASTEST runner in the raw food movement today. Long time vegan and now powered by sweet fruit. How come there is no competitive athletes eating this 'paleo fat diet?' Please shut me up and show me cos Im sick of seeing cardio and muscle deficient paleo crew trying to debunk the china study that us elite athletes are thriving on.

Can you debunk me with a high fat eating paleo athlete that is a national level runner, cyclist, power lifter, UFC fighter like us vegans clearly have provided.

Didnt think so.. :)

Love, peace and banana grease.

Durianrider

Dave said...

G'day durianrider!

"Can you debunk me with a high fat eating paleo athlete that is a national level runner, cyclist, power lifter, UFC fighter like us vegans clearly have provided.

Didnt think so.. :)"

Keep thinking. Here's one:

http://www.colting.se/

I dropped a line to a friend of mine who is seriously into fitness. He got into power-lifting 5 months ago. According to him, he would be second on your list of top-10 vegan/vegetarian power-lifters - and is about as far away from being vegan as you can get.

Am I right in guessing you're from the Land Down Under? Then you'll appreciate this story of 20,000 year-old footprints found in Australia, indicating this person would have been competitive with today's fastest sprinters:

http://news.nationalgeographic.com/news/2006/08/060803-footprint_2.html

All evidence indicates this ancient human was a hunter of meat.

I have no doubt that you can bench-press more than me, and am sure your impressive musculature more than compensates for any other, ahem, deficiencies. Here's the funny thing about that, though: I can go to the gym and get stronger, but I don't think you're going to get any smarter :)

Since I answered your challenge, perhaps you would care to rise to mine? Debunk me with a vegan who can match wits with this carnivore on the topic of nutrition, metabolism, and health. So far the best your crew seems to be able to muster is T. Colin Campbell. I would dearly, DEARLY love to go brain-on-brain with Dr. Campbell, but will happily settle for any other vegan who feels they are up to the task if Dr. Campbell is lacks the conviction to stand behind what he preaches.

Cheers!

Chris said...
This comment has been removed by the author.
Dave said...

@Chris,

Indeed. durianrider really stuck it out there on this point, considering his own physical attributes:

http://bit.ly/cD0fAv

I'll have a get a picture of myself, 42 years old, and hardly find the time to exercise at all, and compare pretty favorably with durianrider. And I'm even sporting a little extra muffin-top flab from over-carbing during vacation.

Personally, I don't think comparison of physical attributes is terribly relevant to the discussion of nutrition and metabolism. There's plenty of examples of people with tremendous physical strength who are clearly not healthy (e.g. non-natural bodybuilders, who have a tendency to drop dead pretty early in life). But since durianrider chose to engage at the "mine is bigger than yours" level (rather than, say, discussing actual scientific evidence), we should be more than happy to oblige him.

Logan said...

Dave,

We're not dealing with the most rational-minded folk here. Perusing the 30BAD site, I found a question from someone who wanted to know how to field arguments that homo sapiens evolved big brains because of (if my memory serves) cooking and eating meat. Most of the advice she got sounded like: "Tell them that from the dawn of civilization, human cultures have been fueled by fruit and veg, not prime rib" (I'm paraphrasing here)

My favorite answer, and this was serious, was something like- "Tell them: 'so you're saying that my raw vegan diet will revert me into a monkey? I seriously doubt that'" This was one of the chosen answers, not just some crackpot.

Such nonsense abounds in this community, evidently. A handful of raw vegan elite athletes does not constitute proof of a superior human diet. It's asinine on any level, and hardly worth your time. Love your site- keep up the good fight!

Anonymous said...

I was not impressed with the vegan bodybuilders in the photo contest...

The winners wouldn't have made it to the second round of any regular diet bodybuilding contest. Cheeke has an impressive physique he clearly works hard for, but one example does not a rule make.

Durian & Lee's bragged about "Combined 7 years" of veganism probably means whatever muscle they have left will be gone in the next few years, so I will keep in the back of my mind to see what happens to 30bananasaday.com in the next 5 or so.

Durian's Smell said...

Durianrider,

How about almost every single major athlete?

Whether in power sports like football, hockey, basketball, or more skill based sports like tennis/baseball.

ALMOST EVERYONE is not vegan. Seriously, you are one deluded dude.

Anonymous said...

How about Martin from dirtycarnivore.com

http://forum.dirtycarnivore.com/index.php?action=dlattach;attach=779;type=avatar

He eats only of the animal world.

Dave said...

Hi All. Thanks for the comments and links, great stuff. Thanks also for the encouragement. Also take a look at the latest post:

http://sparkofreason.blogspot.com/2010/07/going-bananas.html

Stephan Guyenet said...

You guys should take a look at durianrider's giant arms on his blog. Impressive...

Dave said...

@Stephan,

Yes, durianrider is as impressive physically as he is mentally.

Dave said...

@durianrider,

Here's one:

http://www.70sbig.com/?page_id=128

This guy posted a total 2017 lbs. powerlifting - with three broken ribs, in 1977. Compare to the top on veganfitness:

http://www.veganfitness.net/viewtopic.php?t=17067

715 kg, or about 1500 lbs. Then check out the "Food FAQ" from the first link.

Are you thinking yet?

durianrider said...

I talked with Jonas yesterday on email. This what he said when I asked what % of his diet is carbs/fat in terms of total calories, Jonas said "Hey,
I have no idea what so ever. I don´t concern myself with that but rather
concentrate on getting the best quality possible in my diet.
It would also differ greatly on what kind of day I´m having and what I´m
doing. The more training I´m doing the more carbs I´ll be having but the
actual percentage may still be relative as "more" only relates to some days
when I´ll be having virtually no carbs.

Sorry I can´t give you a more specific answer.

Jonas is a great guy no doubt but he trains a lot and hence eats a lot of carbs.

Any real athletes you can show me that eat a high fat diet? Didnt think so. :)

durianrider said...

Yeah but these guys are still eating carbs...paleo means most of your calories coming from fat and protein and no carbs like fried potato chips like this 70's big is eating..

get my point? show me the athlete that eats paleo and is kick'n ass..

durianrider said...

Thats right, Im an elite cyclist. Im training/racing with some of the best riders on the planet. Show me 1 paleo athlete training with the best in their sport on a low carb diet..JUST ONE!! :)

durianrider said...

Dave, you think you can ride/run as fast as me and you dont even exercise??? :)

durianrider said...

Thats right, we wont be gettting fat, heart disease, depressed, losing cardio/muscle like all the steroid tak'n crew eventually do, osteoporisis etc.

Look after your health gang.

durianrider said...

Thats a good point, and where is even ONE high fat athlete doing good?

Jonas colting aint eating a high fat diet judging by the email he sent me.

Anyone? :)

durianrider said...

and lets remember that Denise Minger is a lovely lady (that is even if she really exists as no one can confirm she is not someone else..)

but Denise posted on our site she was struggling with eating enough, weight loss, teeth issues, fatigue, memory issues, poor digestion and hair loss..these are all trademark symptoms of anorexia.

No wonder she hated life as a vegan! She had anorexia! :(

durianrider said...

Its funny that you crew are so undercarbed you didnt even understand my original post..please read it again..I asked for the HIGH FAT EATING PALEO ELITE ATHLETE..

I didnt ask for elite athletes that eat a high carb, low fat, diet with a bit of animal products. Everyone knows that idiot! :)

durianrider said...

Close but Martin eats cooked grains, fruits and veg these days. And he still aint no elite athlete.

I want to see current youtube vids/race results etc not some bogus photo that could be ANYONE! :)

durianrider said...

Nice gym work champ. But just cos you can pose in shorts like anyone can, doesnt mean your an elite athlete like I am.

Your sport is bodybuilding and you look pretty small for a bodybuilder. You wouldnt be able to train with pro bodybuilders like I train/race with pro cyclists eh?


Thats my point..you's are all lovely people but none of ya's are elite athletes like we are.

Alex said...

Paleo diets are defined by their lack of neolithic foods, not their macronutrient ratio. Not all paleo diets are low-carb high-fat, the Kitavan diet being a prime example. Obviously, an endurance athlete can benefit from higher carb intake.

Personally, as a life-long non-athlete with life-long wonky carb metabolism and blood sugar issues, I have no need or desire for a metabolically hellish diet of 80% carbs. On a starch and sugar heavy, predominantly vegetarian diet I became increasingly fat and lethargic. A paleo-ish diet of 20-25% carbs and 20-25% protein suits me just fine. At 49, I'm i'm in the best shape I've ever been in.

Dustbag said...

I am probably too late to this party, but

@durianrider - Thousands of elite athletes, all eating a Paleo diet: www.crossfit.com

@Dave - not sure if you're familiar with CrossFit, but if not you should check it out just to refute arguments like durianriders. Coach Glassman and his cohorts have done an amazing job of applying the science of fitness. And to this blog postings point, they unequivocally find that elite performance is ONLY obtained through a diet based on Paleo principles. In their words : “eat meat and vegetables, nuts and seeds, some fruit, little starch, and no sugar”. Their site is also full of pictures and videos documenting the performance of their athletes, which I would put up against any of the examples durianrider provides.

durianrider said...

Thanks Dustbag, but if you read my actual request, its for 'elite athletes eating a high fat diet..'

Paleo diet is a broad generic term that can include big macs, beer and coke according to the author of the 'paleo diet for athletes'.

Check out my youtube channel 'durianriders' where I provide more info.

Have a good day all. :)

Anonymous said...

@ dustbag

Greg Glassman. Creator of crossfit. AKA "Couch". No not a typo, looks like he just got off the couch! Loves paleo & whiskey!

http://www.colinmcnulty.com/blog/images/greg-glassman-colin-mcnulty-crossfit.jpg

Mason said...

hi Dave,

I saw one of your posts mentioning Jaynes, over on Densie's blog. I've reproduced some of her analysis and want to share it will all the statisticians interested in the topic, to get more eyes rooting out bugs and to flesh it out: https://bitbucket.org/masonicboom/chinastudy.