The cost of bad research

Ask the average person in the street if a low-fat diet is good for them they would instinctively say 'yes'. Ask them why and they'll almost certainly tell you it's better for your heart, and protects against other diseases they may not even be able to name.

But now, if you believe the headlines, there's no benefit in low-fat. If it didn't reduce the risk of certain cancers and heart disease for 25,000 women in a medical trial, what are the chances of it helping others?

Published in the hugely respected Journal of the American Medical Association, the Women's Health Initiative (WHI) Dietary Modification Trial involved 48,835 post-menopausal women, eight years of follow-up and cost a staggering $415 million dollars of taxpayers' money.

Scientists from such esteemed institutions as Harvard Medical School, UCLA, Ohio State, Brown and Northwestern Universities carried out the research.

The end results created headlines that declared a low-fat, fruit-'n'-veg-rich diet does not change your risk of colorectal or breast cancer of cardiovascular disease (CVD).

The trial set out to compare women eating a normal diet with women eating a diet low in fat and high in fruit, vegetables and whole grains.

Nice idea, so what went wrong?

The design of the study was flawed from the start, which is undoubtedly why there was opposition when the study was first proposed. Leading nutritional and medical researchers said it wasn't worth doing, while the National Institute of Health initially rejected calls for funding.

Strategic lobbying and then a run through congress eventually got the necessary approval and the WHI was off.

The diet aimed to cut down fat intake for the intervention group to less than 20 per cent, eat five servings of fruit and veg, and six portions of grains per day. The studies claim they succeeded in reducing fat intake by 8.2 g, but by year six the average fat intake was still 29 per cent. The normal diet group was eating 35 per cent fat.

The intake of vegetables and fruit did go up as well, but only by about one serving per day.

The researchers themselves admit that very few of the women in the intervention group actually met the targets.

Excuse me for being so bold, but does this not mean that $415 million were spent on an intervention study that didn't really intervene?

Does this not immediately put a massive question mark on any claims?

Apparently, no - at least if you believe the headlines. Many national and local newspapers have picked up on these reports and run with stories taking the conclusions to the consumer.

The San Francisco Chronicle ran with, "OK, so don't hold the fries"​; the UK's Independent chose, "Modern nutrition: Forget all you ever knew about diets"​; CBS News said, "Study: Low-Fat Diets Big Letdown."​

The limitations with this study, of which there are several, do not mean however that it was completely useless: It has given us some big clues as to how not​ to do studies.

Being overweight or obese is a known contributor to diseases such as CVD. Between the test and control groups, the number of overweight or obese women was about the same: 36 per cent of women were overweight and a further 38 per cent were clinically obese.

The researchers claim that weight loss was never a target of the study. I feel the need to ask the question, "Why not?"

With such a body of evidence linking obesity to diseases such as diabetes, CVD and breast cancer, and with over 70 per cent of volunteers with a 'weight issue' surely weight loss should have been a crucial target.

And what about regular exercise? No mention.

Reducing the size of portions? Not a target.

In future, trials should distinguish the type of fat to reduce. Knowledge has expanded to clarify good and bad fats. Effort should be made to reduce trans fatty acids and saturated fat, while omega-3 fatty acids should be increased, for example.

In terms of CVD, no effort was made to reduce salt intake, increase potassium intake, or use the DASH (Dietary Approach to Stop Hypertension) diet. No wonder there was a null result.

And this study was only about post-menopausal women. Maybe this diet could have worked for other groups.

Some experts have rightly spoken out against the studies, encouraging the public to stick with five portions of fruit and vegetables a day and stressing that people take care about fat intake. But are consumers now questioning such advice?

Whether a study costs $1 or $415 million, care should be taken in design, time should be taken to identify the key factors, and methods improved to get maximum benefit from the data.

The public has a right to read about studies like the WHI, but only if journalists take the time to describe the limitations. Many do not, and this is where the problem is exaggerated.

There will undoubtedly be consumers who accept these studies, people who are reluctant to change their diet or lifestyle, despite a mountain of science saying the opposite. The true cost of this study could well exceed the $415 million spent on it.

Stephen Daniells is the Food Science Reporter for NutraIngredients.com and NutraIngredients-USA.com. He has a PhD in Chemistry from Queen's University Belfast and has worked in research in the Netherlands and France.​

If you would like to comment on this article please contact Stephen Daniells​.