Over the course of my scientific career I have spent over a decade in training (undergrad, masters, doctoral, and now fellowship) and 90% of that training is on how to think and answer questions, the other 10% is content area expertise.
For the last several decades I think most of the nutrition world has lost the forest for the trees and spent entirely too much time focusing on content, and not how to think about dieting and what REALLY are the key things we need to know. I want to try and rectify that today.
Introduction
My training over the course of my scientific career has given me a unique perspective. I have been trained across the translational spectrum; last week in the span of 24 hours I cultured cells, inject animals, talked to a patient, and crunched clinical trial data. This training has taught me a lot about how to think about large problems and how to tie evidence through three key layers into coherent ideas that we can then translate into application for people.
These three layers of evidence are: observational, mechanistic, and intervention.
Observational
Observational studies are simply those that take a large group of people and collect data and attempt to find things are are associated or correlated with one another. For example, in a population of people, does a higher intake of Food X correlate positively with body weight (Figure 1).
Observational studies are often cross-sectional in nature, only showing data from a “snap shot” in time. They can also be longitudinal in nature, wherein you track several data points overtime and follow someone over longer periods of time (Figure 2). Regardless of whether data is cross-sectional or longitudinal, you still have a very substantial limitation of this study design: correlation does not equal causation. The best this type of study can do is find correlation and generate hypotheses about mechanisms.
Correlations are spurious and random things can be correlated. In the example below, it is quite clear that the correlation is present, but there is no mechanism to even begin to support cause and effect (Figure 3).
They are singular snapshots in time, or they look backward, or sometimes even forward. Regardless of their temporal characteristics, they all suffer from the inability to determine causality.
Mechanistic
Mechanistic studies are highly controlled studies designed to elucidate how a phenomena works. For example, if we observe that a lot of people who are consuming a low-carb diet are losing weight, how exactly is that working? Is it due to lower levels of insulin? Is it due to magical properties of eating higher fat by percentage? Is it a magical creature that lives in carbs that prevent you from losing weight? Or is it simply calories?
These studies often employ models (e.g. animal models) and create somewhat artificial scenarios in order to control every variable possible to drill down on that specific mechanism and reduce the noise as much as possible.
A mechanistic study might be something like genetically removing the gene for the protein leptin in a mouse and then examining what the effect on carbohydrate metabolism in skeletal muscle. From this type of experiment you could learn something very fundamental about the role of leptin in muscle metabolism but it would have substantial limitations (e.g. it is not done in humans and doesn’t tell you anything about gradations in leptin levels and how those effect carbohydrate metabolism. This only examines a binary effect.
Another example would be a study like Dr. Kevin Hall’s recent metabolic ward study wherein they took humans, confined them to a hospital and controlled their food intake, exercise, and monitored the rest of their lives (Figure 4). The study compared low-fat versus low-carb ketogenic diets and asked whether the lower insulin response from a low-carb diet increased fat oxidation, weight loss, and fat loss (we will cover this in detail below).
Intervention
Intervention studies are those that directly test a hypothesis in question through isolating and directly manipulating a variable as much as possible in order to achieve a specific effect. An intervention study looks something like this. Take a large group of people, randomize them to intervention X and intervention Y and see which one lowers the risk of developing a disease or which one is the best diet for weight-loss (Figure 5).
These studies can be very mechanistic in nature but are often not. Additionally, the primary outcome is different than mechanistic studies. The goal for these studies is often to assess whether the intervention improves (or worsens) a specific outcome (e.g. weight loss, heart disease, cancer risk, etc.), not designed to determine exactly how a treatment/intervention works (e.g the PCSK9i inhibitor works via reducing LDL receptor degradation).
These studies also have limitations in that they often are done in small samples (but not always), lack complete control over variables, and are sometimes limited in the demographic of their population.
Bonus Step: Meta-Analysis
In addition to the three layers of evidence discussed, we also have a tool at our disposal called meta-analysis. A meta-analysis is essentially a computational tool to “synthesize” the literature. More formally,
“Meta-analysis is a quantitative, formal, epidemiological study design used to systematically assess previous research studies to derive conclusions about that body of research. Outcomes from a meta-analysis may include a more precise estimate of the effect of treatment or risk factor for disease, or other outcomes, than any individual study contributing to the pooled analysis. The examination of variability or heterogeneity in study results is also a critical outcome. The benefits of meta-analysis include a consolidated and quantitative review of a large, and often complex, sometimes apparently conflicting, body of literature. The specification of the outcome and hypotheses that are tested is critical to the conduct of meta-analyses, as is a sensitive literature search. A failure to identify the majority of existing studies can lead to erroneous conclusions; however, there are methods of examining data to identify the potential for studies to be missing; for example, by the use of funnel plots. Rigorously conducted meta-analyses are useful tools in evidence-based medicine. The need to integrate findings from many studies ensures that meta-analytic research is desirable and the large body of research now generated makes the conduct of this research feasible” (1).
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3049418/
Up to this moment, this article has been theory driven and fairly esoteric. Lets use these principles to work through a popular dietary intervention and see what happens when we evaluate it across the levels of evidence.
Low Carb and Ketogenic Diets as a Use Case
The Underlying Hypothesis of Low Carb, Ketogenic Diets
A scientific approach to answer a questions begins with a hypothesis. We have covered the underlying hypothesis of the low-carb or ketogenic diet being a superior diet for fat loss ad-naseum (here, and here). The point of this article is about “how to think about diets and fat loss” so much of this article is simply about how to test hypothesis. Thus I will just lay out the hypothesis and then step through it, each “data level” at a time.
The Hypothesis for Low-Carb, Ketogenic Diets: The accumulation of fat mass is a result of dietary carbohydrates leading to elevated insulin levels which shifts metabolism into fat storage and away from fat oxidation.
Fortunately for us, science is a set of tools designed specifically to test hypotheses. If the hypothesis were true we would see the following things:
- Observational Data: High carbohydrate intakes as a percentage of diet would be associated with higher body weight or body fatness.
- Mechanistic Data: In highly controlled experimental studies, carbohydrate restriction should result in more fat loss than fat restriction
- Interventional Data: In well-controlled experiments in fairly, free living humans, low-carbohydrate diets should be far more effective than higher-carbohydrate diets.
The evidence at all three levels exists, so let us walk through it.
The Observational Data
A simple test of the hypothesis at an observational level would be to sample a large group of people and look at whether higher amounts of dietary carbohydrate intake as a percentage of food intake is associated with higher body weight or body fat. In short, if this hypothesis is correct we should see the red line, if incorrect we should see the blue line.
These type of studies have been done in a wide range of populations and the data is available (2, 3, 4). When we look across the studies done in this area it is the blue line that dominates the results. So the observational data to support the hypothesis is quite scant.
The Mechanistic Data
When we test the hypothesis that carbohydrates are a main regulator of fat mass accumulation in a mechanistic manner, we would design highly controlled experimental studies that show that carbohydrate restriction should lower insulin levels, increase fat oxidation, and result in more fat loss than fat restriction. Fortunately for us, this work has already been done.
We have covered these studies previously here on Science Driven Nutrition so I will just reiterate it.
STUDY 1
The first study was essentially a cross-over design where people were fed either a restrictive carbohydrate diet or a restrictive fat diet for 6 days after 5 days on a baseline diet (5).
To summarize the findings from this study, they found that decreasing carbohydrate intake did lead to increased fatty acid oxidation and decreased carbohydrate oxidation; however, reducing fat intake led to a slightly bigger loss in body fat than the group that restricted carbohydrates (Figure 7).
The other important finding from this study was that an increase in fatty acid oxidation (one of the measurements that is considered a benefit of the ketogenic diet) doesn’t necessarily result in more fat loss, in fact it may be the opposite (this remains to be elucidated) (figure 8).
STUDY 2
This study is one of the most tightly controlled and well-designed dietary studies done to date (6). Metabolic wards, metabolic chambers, DEXA, doubly labeled water, multisite, mutli-investigator, and well done statistics . . . this is the Cadillac of dietary studies.
Briefly, “Seventeen overweight or obese men were admitted to metabolic wards, where they consumed a high-carbohydrate baseline diet (BD) for 4 wk followed by 4 wk of an isocaloric KD with clamped protein”.
In this study the subjects lost about 0.8 kg of body weight (with 0.5 kg from body fat) during the 15 days of the high carbohydrate baseline diet. The ketogenic diet led to a rapid weight loss of 1.6 kg due to water weight loss as only 0.2 kg of body fat were lost during the 15 day ketogenic diet) (Figure 9).
So during this study the participants lost about 1 pound of body fat during the 15 days on the higher carbohydrate diet while they only lost about ½ pound on the ketogenic diet. This all occurred despite lower insulin levels and higher ketone bodies present during the duration of the ketogenic diet phase. This suggests that the ketogenic diet does not offer a metabolic advantage for weight loss during this time frame either.
This study also showed a very rapid “adaptation” wherein insulin secretion dropped, fatty acid and ketone levels rose, and fat oxidation increased substantially. This piece of data is incredibly important as it CLEARLY demonstrates a swift, and robust “adaptation” to a ketogenic diet.
The mechanistic data to support the hypothesis is not even present.
The Intervention Data
Study 1
The first study we will cover is, “Comparison of the Atkins, Ornish, Weight Watchers, and Zone diets for weight loss and heart disease risk reduction: a randomized trial” (7). Essentially, this study took 160 people and randomize them to either an Atkins Diet (Low-Carb) an Ornish Diet (Low-Fat Vegan Diet), a Weight Watchers Diet (IIFYM of the 90s), and a Zone Diet (Macronutrient Balanced Diet). They followed these people for a year and here is what the data shows (Figure 11).
In the left figure, they show that each diet had very similar weight loss profiles and that there was no difference in weight loss between diets. In the right figure they show that adherence is what determined weight loss more so than any specific diet when it comes to weight loss.
Study 2
The second study, “Comparison of the Atkins, Zone, Ornish, and LEARN diets for change in weight and related risk factors among overweight premenopausal women: the A TO Z Weight Loss Study: a randomized trial” is virtually identical in design, but consisted of overweight menopausal women (8). In this study, they took approximately 76-79 people and randomized them to either an Atkins Diet (Low-Carb) an Ornish Diet (Low-Fat Vegan Diet), a LEARN Diet (IIFYM of the 90s), and a Zone Diet (Macronutrient Balanced Diet) (Figure 12).
This study also demonstrated that each diet had very similar weight loss profiles and that there was no difference in weight loss between diets. They also demonstrated that adherence was substantially more important for weight loss than which diet people we on.
Study 3
The third study, “Comparative study of the effects of a 1-year dietary intervention of a low-carbohydrate diet versus a low-fat diet on weight and glycemic control in type 2 diabetes”, examined the effects of a low-carb versus and low-fat diet in people with type 2 diabetes (9).
In this study they demonstrated that a low-carb and a low-fat diet had virtually identical results on changes in HbA1c, weight, and blood pressure. This effectively demonstrates that if there is a benefit to low-carb diets, the benefit is likely to be small and that weight loss still determines most of the health benefits of dieting in people with type 2 diabetes (Figure 13).
Meta-analysis
We could continue ad nasuem with the data, or we can simply use a powerful tool called a meta-analysis where we pool all the data from these types of studies together. When you compile the intervention based research into a meta-analysis for the comparison of Low-carb ketogenic diets to Low fat diets you see this picture emerge (Figure 14) (10).
This picture tells us three critical pieces of information: 1) There is high heterogeneity between the results, 2) there appears to be a slight benefit of about 2 pounds of a ketogenic diet over a low fat diet over a 12 month time frame, 3) This is regardless of adherence.
This lends us to several critical observations. The clear, repeatable data from the intervention studies that showed the variable that most explained meaningful superiority for weight loss was adherence (e.g. a ~15 pound difference between low and high adherers over 12 months). The superiority of low-carb versus low fat is ~2 pounds over 12 months. The juxtaposition of those two data points should show you what “battle” you should really be fighting.
The cumulative data from myriad intervention trials also do not support the hypothesis.
What Do We Know?
What does all this data from low-carb versus low-fat studies tell us? It tells us that calorie balance is more important than a specific diet, and that adherence matters more than any macronutrient feature of any diet.
This means that adherence needs to be the most important factor you consider when determining a diet for yourself or a client and that diets are tools, they are not the one size fits all answer to all your problems like your Swiss army knife was when you were 11.
P.S. This article is based on a lecture I gave at the Inland Empire Fitness Conference.