Two Thirds of Keto Studies Did NOT Provide a Ketogenic Diet

The single most overlooked issue in keto-related discussions

TLDR:

• There are good reasons why the ketogenic diet has grown in popularity

• Health authorities tend to recommend against the use of the diet, doing so with some shaky rationale

• Arguments about whether keto is ‘good’ or ‘bad’ cannot be resolved by the literature, as two-thirds of studies are not providing a ketogenic diet

• Even if we filter out the studies that aren’t inherently flawed, they can’t say if keto is right for you

As more and more research avenues yield to the tenet that energy metabolism is key in all health disorders, interest in the ketogenic diet continues to climb. Regularly dismissed as ‘a fad’ or ‘risky’ by authorities, the ketogenic diet  remains a go-to option in the toolbox of many practitioners precisely because of how it can both support and rehabilitate energy metabolism.

Recent decades have seen our knowledgebase grow from personal anecdotes of success to a more complete body of evidence, with an assortment of studies showing benefits of the ketogenic diet in a variety of conditions.

A 12-month RCT on individuals with type II diabetes, 60% of patients reversed their diabetes (and insulin therapy was reduced or eliminated in 94%). This echoed the findings of a previous trial, where 86% of participants were able to reduce or eliminate their medication. Advocates of the diet point out that, in around 150 years of dietary research, no other dietary approach has come close to this. 

The potential of the ketogenic diet was further underscored in a trial on Type II diabetics that tracked their HbA1c levels (a marker of glycaemic control). They found that those on the ketogenic diet saw a dramatic drop in these levels, plummeting from an average of 8.9% to just 5.6%. The fact that this was an average response and that this drop put participants within the normal range (ie. no longer diabetic) underscores the benefits of keto. However, the most impressive aspect of this study was the speed at which this resolution occurred; this was achieved in just 90 days.

Another RCT found that Type II diabetics using the ketogenic diet for a year saw improvements in 23/26 markers for cardiovascular health.  A pharma sales exec’s wet dream, only not achieved with fistfuls of medications but instead with a change in diet.

A 2021 trial (on obese women with fatty liver and polycystic ovary disease) compared the responses of a ketogenic diet to conventional pharmaceutical treatment. They found that 6/7 women using the dietary approach reversed fatty liver, whereas only 1/10 of the conventional arm did so. Emphatic stuff.

A 2010 RCT found that those following a  low-carbohydrate, high-protein diet (without caloric restriction) recorded an average weight loss of 13.2kg in 13 weeks, doing so without restricting calories (this compared to 7.3kg recorded by those controlling calories on a ‘normal’ diet).

On the subject of weight loss, there have been many trials that have pit low-carb diets against low-fat or ‘standard’ diets. To date, no conventional plan has ever been shown to beat out ketogenic approaches but, conversely, there are 35 trials that show low-carb diets to be superior to weight loss. A 35-0 scoreline indicates that there may be something remarkable occurring in the metabolism.

Naturally, if we accept the principle that energy metabolism governs everything, we may also expect the ketogenic diet to have impacts outside of glucose management, cardiovascular risk and weight loss. And we do indeed see just that; a meta-analysis on keto and cognition found that 80% of the studies showed a benefit (and the other 20% showed no harm). A further study found that the ketogenic diet improved IVF outcomes. A pilot study found clear benefits in autism.

So why the pushback?

No doubt that emotion plays a role, with tribalism one obvious element but certainly not the only one. It’s lack of compatibility with grab-and-go eating culture causes hesitation for some. However, in my experience, ‘I tried it a couple of years ago and felt rough for two days’ also goes a long way in understanding resistance to using this tool. So too the ubiquitous messaging about the ‘dangers’ of the ketogenic diet.

The Science(TM): Keto is Dangerous

A 2003 review is often quoted when discussing the nutritional inadequacy. The paper itself claims that ‘low-carbohydrate diets are at greater risk of being nutritionally inadequate as they enforce restriction of food choices’ (they do not point out that the only item that is universally restricted is grains and sugar, the two foods groups with the lowest nutritional value). They go onto claim that ‘losses of protein and fat are the same during a ketogenic diet as during a hypoenergetic, non-ketogenic diet hence no diet is superior to another in terms of preservation of lean body mass’ (even though there is plenty of evidence that says the opposite).

A 2024 ‘state of the art’ review concluded that keto ‘does not fulfill the criteria of a healthy diet’. The body of the text does not make it clear what they define this as; however, it did lay out a whole host of positive outcomes shown in the literature before quickly pooh-poohing the benefits of such an approach, pointing out that many studies on the ketogenic diet did not employ control diets (they did not offer comments on the many studies that were controlled and came to the same conclusion). They also urge caution on the basis that the trials on the ketogenic diet were not of a uniform length (without noting that this applies to the literature we have for every single dietary approach ever studied). A further (unsupported and overgeneralized) criticism the researchers made is that ‘extreme carbohydrate intake reduction is executed by limitation in wholegrain products, fruits, and vegetable consumption’ before immediately contradicting this claim by pointing to the fact that most individuals on a ketogenic diet will consume large amounts of leafy greens. The problem, you ask? That the Vitamin K in the leafy greens ‘can potentially interact with pharmacological treatment’. Hmm.

Another oft-quoted paper is a 2021 study found an association between those consuming less carbohydrates and increased overall mortality. The coverage that followed failed to note the lack of causal links, failed to point out that the dietary information was incomplete (around 750 kcals per day per person was unaccounted for) and that ‘low carbohydrate’ was defined as getting 37% of total energy from this source (I trust I need not provide comment on research that classifies 230g of carbohydrates per day as ‘low carb’).

A more sober criticism of the ketogenic diet is that it is not sustainable; indeed, the 2010 study mentioned above is particularly pertinent to this discussion (due to both the striking weight loss recorded in such a short time frame, but also how the relative benefits faded in the 25 weeks that followed). A 2018 review pointed to a trend in the results of keto trials, finding that weight loss tends to peak at 5 months in.

It is understandable that anyone curious about this approach may feel confused. The confusion from these studies can be magnified as a result of the messaging from the usual institutions.

Only last year, the American Heart Association published dietary advice warning that the keto diet was bad for heart health. The justification for this assertion mainly focuses on offering ‘concerns about nutrient adequacy’ and pointing that such diets ‘align poorly with AHA Dietary Guidance‘ (shock: they don’t recommend it because, erm, they don’t recommend it). They do acknowledge that there have been benefits recorded but cite one particular review, a 2019 publication that conducted a meandering discussion on topics loosely related to the ketogenic diet, jump in heavily on concerns as to cholesterol levels and then half-conclude that erm-its-an-option-but-probably-not-necessary. The review, for which food manufacturers and drug companies were listed in the financial disclosure section, just so happened to be authored by the National Lipid Association (NLA) (previously been described as having a ‘dizzying number of industry relationships’).

The establishment position is largely consistent. As usual, Harvard University’s educational page on the ketogenic diet perfectly exemplifies the singular position on ketogenic diets that have seemingly been adopted by all institutions that are a) considered prestigious and b) receive generous industry funding.

They begrudging acknowledge that it may have a role in weight loss and type II diabetes, but provide little additional details and make no mention at all of other conditions that have responded well to keto in trials. However, they place major focus on the dangers of ketoacidosis, a complication of diabetes and something that a ketogenic diet cannot reasonably be expected to induce (something known by anyone familiar with the mechanisms involved). They also hone in on potential side-effects but never speak to the rate at which these have appeared in studies. Instead, they focus on those that have been ‘suggested’ as a concern (without saying who has suggested them), outlining concerns of gout, kidney stones and osteoporosis.

And, in keeping to the expected script, the provide the customary phrase: “Available research on the ketogenic diet for weight loss is still limited”. This tends to be a shortcode for ‘let’s not talk about the science’.

Of course, it may seem a bit of a stretch to say that the research is limited when there are over 10,000 studies on the subject. However, the real shocker is that this statement goes nowhere near to conveying the real issue with the available literature on keto.

Conflicting meta-studies on the ketogenic diet: its good for weight loss in diabetics, but also not good for weight loss in diabetics

Meta-studies are essentially ‘studies of studies’ and assess the full body of literature that addresses the same question; the idea being that they calculate a total effect size and therefore allow us to form a more confident picture of the true effect (in short, that we can know whether the results of any paper were anomalies, or not).  In the Hierarchy of Evidence-Based Medicine, they are considered to be the highest form of evidence.

Let’s take a look at the evidence provided by the meta-studies. To get a fair sample, I ran a Pubmed search for meta-studies on the ketogenic diet that relate to the metabolism (ie. weight loss, lipid profiles or blood sugar level control) and were published in the last five years, conducted on adults and exclude the ‘very low calorie ketogenic diet’ (which provides less than 1000 kcals per day). You can bring up this search yourself here, which returns 4 hits. One relates to cognitive effects in Alzheimers, leaving three meta-analyses to examine:

An executive summary here:

Yuan et al (2020) assessed trials that had studied the impact of the ketogenic diet on metabolic markers in Type II Diabetes, and found an impressive effect on glucose control, lipid profiles and weight.

Choi et al (2020) looked into similar markers in those who were overweight or obese (some of which were diabetic) and found that, in the 14 trials they included, ketogenic diets were more effective than low-fat diets in all metabolic parameters measured (and even more so in those with Type II diabetes).

Meanwhile, Choy & Louie (2023) presented their findings on 11 trials on Type II diabetes and concluded that, compared to control diets, keto improved HDL figures and reduced triglycerides but provided no benefit in glycaemic control or weight loss.

In short, the available meta-studies tell us that a) the ketogenic diet has impressive effects on weight loss in Type II diabetes but also b) the ketogenic diet has no effects on weight loss in Type II diabetes.

As is often the case, we are faced with a need to reconcile these opposing conclusions. This is generally a much better option than picking sides, albeit one that goes against the human categorisation instinct and takes a lot longer; it involves looking at the populations studied, the inclusion/exclusion criteria (length of study), baseline diet and additional interventions (such as medications, exercise, etc).

But before we do this, we must ask a much simpler question: how many of these studies actually assessed the ketogenic diet? Let’s take a look.

Analysis: how many studies are even providing a ketogenic diet?

Here I did an analysis of the individual studies that were included in the three meta-analyses, and looked into the methodology for both the intended dietary plan and, where published, the dietary plan that the participants actually received (which is only sometimes the same thing). The intentions ring enough alarm bells – many simply tell participants to follow the Atkins Diet (a low carbohydrate diet, but normally ketogenic). Many did not even assess compliance with this faulty recommendation; those that did showed macronutrients all over the shop, with a number reporting carbohydrate intake above 70g per day (and one ‘ketogenic’ study reporting a carbohydrate intake of 225g per day).

I then looked into the ketone levels reported. Many provided no such checks. Some used urinary ketone measurements; this as clear a sign you will find that the researchers do not understand ketosis (urinary test strips only tell you the net sum of what was made available minus what was used; this makes their relationship to blood ketones highly unreliable).

A core principle here: a ketogenic diet is one that generates ketones. But only one paper achieved respectable ketone levels, with a total of 7 recording some ketone production. From the dietary (and exercise) details provided, I would expect a further two studies to have a fair chance of achieving this (despite not undertaking any measurement).

Here's the breakdown…

Yuan et al, 2020

Total number of studies included: 13

Studies that achieved nutritional ketosis in the majority (or <1.0nmol/L): 1/13 (8%)

Studies with borderline nutritional ketosis, or not in the majority (or 0.5-1.0 nmol/L): 3/13 (23%)

Additional studies where recorded dietary intake could be expected to induce with nutritional ketosis: 1/13 (8%)

Choi et al, 2020

Total number of studies included: 14

Studies that achieved nutritional ketosis in the majority (or <1.0nmol/L): 1/14 (7%)

Studies with borderline nutritional ketosis, or not in the majority (or 0.5-1.0 nmol/L): 2/14 (14%)

Additional studies where dietary intake could be expected to induce with nutritional ketosis: 2/14 (14%)

Choy & Louie, 2023

Total number of studies: 11

Studies that achieved nutritional ketosis in the majority (or <1.0nmol/L): 0

Studies with borderline nutritional ketosis, or not in the majority (or 0.5-1.0 nmol/L): 2/11 (18%)

Additional studies where dietary intake could be expected to induce with nutritional ketosis: 1/11 (9%)

So, to round up…

Total number of studies: 25 (accounting for crossover between the meta-studies)

Studies that achieved nutritional ketosis in the majority (or <1.0nmol/L): 2/25 (8%), Watton & Vazquez

Studies with borderline nutritional ketosis, or not in the majority (or 0.5-1.0 nmol/L): 3/25 (12%), Makenzie, Goday & Saslow

Additional studies where dietary intake could be expected to induce with nutritional ketosis: 3/25 (12%), Jabek, Westman & Sun…

…which makes a total of 8/25 studies (32%) for which inclusion can be justified, and 17/25 studies (68%) that did not (or had no fair chance to) produce ketones. That’s 68% of studies that did not provide ketogenic diets. I trust I need not explain why such papers should not ever have made it into a meta-analysis that studies the effects of the, ahem, ketogenic diet.

In short, these meta-analyses can’t expect to provide ‘the one true effect’ of ketogenic diets because three-quarters of the studies did not provide a ketogenic diet.

How can Evidence-Based Medicine get it so wrong?

The biggest hindrance within the EBM paradigm is the baked-in insistence that we must exclude all evidence that did not come from Randomized Controlled Trials, particularly expert knowledge. While such a paradigm comes from a well-meaning place – to protect patients from harmful and ineffective methods that are based on superstition – it is dogmatic and results in the exclusion of both clinical experience and so-called Basic Science (which speaks of how things work, important in outlining the mechanisms behind how and why each intervention works). In short, if you exclude all expert input, you should not be surprised if the science you are left with is lacking expertise.

In applying the RCT-only approach, we are left with the ‘Black Box’ notion and the ‘One True Effect’ aberration, which determine that Intervention X = Outcome Y  and that there is one true effect (something that can be accurately determined if we have enough trials that measure this).

This is why we so often hear things like ‘the ketogenic diet reduces weight by 12.7kg over 8 months’. And we hear this even though such conclusions are made on trial data that shows a wide range of responses, from 4-28kg; in short, that there was not a single participant in the trial that showed this particular response. Examples are from a real trial, here.

These problems are endemic to all EBM research. However, the issue is compounded in the case of ketogenic diets in that laboratory bods – those planning and conducting the research - treat low carb diets as akin to ketogenic diets (in particular, the Atkins plan). The award of research funding is entirely independent of whether those running the studies know anything about ketosis (or have even undertaken such a diet themselves).

Therefore, any discussing of what ‘the science days about keto’ is tarnished by the usual limitations, in that conclusions are fashioned only on what the RCTs tell us and not what we know from good scientific studies or from the frontline. It is also tarnished by the total lack of quality control on the RCTs that are done, in that the majority of studies are measuring responses to a diet that is clearly not ketogenic.

EBM is therefore unable to offer any useful insight on the role of a ketogenic diet, and will not be able to do so until a) those that that conduct research on the ketogenic diet only make conclusions on those that are actually received such an intervention and b) that there are sufficient meta-studies undertaken in various conditions to form a verdict on its role in these circumstances.

Of course, the journey to this point is inseparable from corporate interests. The EBM sect demand that claims are backed by ‘good science’. They emphatically define this as only large, randomised controlled trials and the meta-studies that appraise them. Such RCTs cost anywhere between $40-500m a pop. No-one outside the industry-government web is likely to pony up this cash.

So where does this leave us? At the same place we normally end up… dogma and money.

It is naïve to expect any science bods to suddenly start questioning ‘the way things are done’ and then embedding the required nuanced into their methodology. While it is unfair to generalise, there are few exceptions to the stereotype of over-caffeinated, formula-driven science bods who have no frontline experience of the dietary approach they are studying. And if all you learn on a subject is from the literature, how could you be expected to know what the literature is missing?

It’s also naïve to expect corporations, or the government departments they partner with, to stump up billions of dollars to fund research that endangers their business model.

Research on schizophrenia and brain health is in its infancy but looks promising, while the research on weight loss is already compelling. Worldwide sales of schizophrenia medications total $9.8bn in 2023 while those of ADHD medications came in at $14.3bn and the market for Ozempic is anticipated to hit $100bn by 2035. Expect to see more mysterious cancellations of trials that study the effect of ketogenic diets, especially any head-to-head comparison against existing drugs, as we saw with the recent Maryland trial for the ketogenic diet in schizophrenia.

Coming Up…

This is Part 1/2. In the next part, I will lay out what we can actually conclude about the ketogenic diet from the available literature but also what frontline experience can tell us about the approach; why a ketogenic diet is not superior but why it regularly delivers better results, how it works, and who is likely/unlikely to see benefits from using it.