Some recent meta analysis in the sector point to some of the difficulties of showing results that are truly persuasive. When these meta analyses include studies done years ago at a time when protocols were perhaps less rigorous than today, they can conclude that there is little evidence for any positive effects for a given ingredient.
Heterogeneity of data quality
A recent meta analysis done on nitrate supplementation illustrates the point. Nitrate supplementation, which can come in several forms including using beetroot products, is generally aimed at creating more nitric oxide in the body which leads to dilated blood vessels and more blood flow. Hence better performance, with more oxygen delivered to the tissues, or so the thinking goes.
After combing through 449 studies on whether nitrate supplementation boosts endurance, and arriving at 79 studies that made the final cut for analysis, the authors gave a qualified ‘yes.’ But they threw in a big caveat by saying the evidence in support of this theory is “very low-to moderate-quality.”
So what gives? With all of the academic sports nutrition programs out there, and all of the learned individuals supervising the research, why is the field unable to provide more definitive answers?
One issue is that sports nutrition studies often tend to be done on small population sizes (sometimes with as few as 10 in each group) and shorter duration. That’s all about funding, said Dr Susan S. Kleiner, PhD, a founding member of the International Society of Sports Nutrition and a practicing nutritionist.
“These kinds of studies might show larger effects if we could study thousands of subjects, but who is going to fund a study like that? It’s hard to fund research on debilitating diseases. This is sports nutrition. The value to society at large might actually come over time, but most funding bodies find the questions much less valuable,” Kleiner told NutraIngredients-USA.
Expert: Careful design can ameliorate small study size
Dr Ralf Jaeger, PhD, principal in the scientific consulting firm Increnovo, said some sports ingredients, like caffeine and ATP, can show acute benefits. And some other endpoints can be arrived at quickly, so the funding limits—which militate toward small, short term studies—can be worked around.
“The duration of a sports study is driven by the ingredient and the target outcome. Very few ingredients show acute performance benefits, e.g., caffeine, ATP, while most other ingredients need to be supplemented chronically to increase their content in the muscle, e.g., creatine or beta-alanine. Looking at changes in muscle mass, strength and power require a controlled multi-week training program accompanying the supplementation. Looking at reduction in muscle damage and recovery can be done through a one-time muscle damaging exercise,” Jaeger said.
“Even a small numerical effect can have significant benefits, considering the fact that the difference between 1st, 2nd, and 3rd place at competitive events is very small,” he added.
“The margin between poor and good is much larger than the margin between very good and great. So often we are measuring changes in subjects who are very healthy, or even super healthy, and looking for improvements. When we study people who are in poor health/sick (or depressed or can’t sleep) and we aim for improvements, it’s not that hard to see measurable improvements,” Kleiner said.
Reporting on ‘trends’ instead of hard results
The danger is that while sorting through these small benefits that are observed over a short time scale, researchers can get lost in the weeds. Big market swings can hang on the results of even a single study, so sometimes research teams could be under some pressure to prove out a given theory. Kleiner said the best research avoids this pitfall, but it would be naïve to assume it never happens.
“It seems obvious to say: Research should not be conducted with a bias toward a particular outcome as the plan of the study. Certainly scientists have theories, but the study design must be as non-biased as possible,” she said.
“Study limitations are often the sticking point of a good vs. a poor study: lack of controls or poor methods that are either never mentioned in limitations, or become excused by researchers and called limitations, reducing the quality of the data collection and the ability to reach meaningful conclusions. Trends are stressed in conclusions because there are often no significant differences discovered,” she added.
Jaeger said knowing with precision the mechanism of action of an ingredient can help researchers get the most bang for their bucks. Also, choosing endpoints that are meaningful to professional and recreational athletes as well as active people can help a study gain wider traction. Still, some studies presented at sports nutrition research forums have enough shortcomings that they might actually serve to confuse the issue, rather than shed additional light on a research question.
“Funding is mostly what constricts the scale of a study. For someone who needs to publish, they design a study based on the available funding. Sometimes that means a study that isn’t really worth doing,” Kleiner said.
“I still see studies submitted for publication that try to create conclusions not based on the data, and meant for creating marketing statements. It’s very irritating as a reviewer. I know that I don’t approve publication of those manuscripts,” she added.
State of art is improving
Both Kleiner and Jaeger have been involved in the sports nutrition field for many years and have seen it all—the good, the bad and the ugly. Both agreed that recently they are seeing much more of the former and much less of the latter.
“Overall, the quality of sports nutrition studies has greatly improved over the years, however, there is still significant room for improvement,” Jaeger said.
“I think we have a growing contingent of passionate young scientists with excellent training, doing very high level work, worldwide. The addition of international communities and communications among labs has been very stimulating to the research outcomes. We are certainly moving in the right direction. Sports nutrition science is very young, but is advancing quickly with great recognition. I’m very optimistic and excited to see where we go from here,” Kleiner said.
