Hi, author of the blog post here! Thank you for writing in with your concerns. First:
> Please be very careful when someone tries to tell you that supplements are miraculous and pharmaceutical drugs don’t work at all.
I'll concede I unintentionally gave the tone that one should replace antidepressants with supplements, even though the conclusion specifically writes: "(Don't quit your existing antidepressants if they're net-positive for you!) you may also want to ask your doctor about Amitriptyline, or those other best-effect-size antidepressants."
I have now edited the intro to more explicitly say "you can take these supplements alongside traditional antidepressants! You can stack interventions!"
===
> and nobody noticed this massive discrepancy until now?
> Several meta-analyses of epidemiological studies have suggested a positive relationship between vitamin D deficiency and risk of developing depression (Anglin et al., 2013; Ju, Lee, & Jeong, 2013).
> Although some review studies have presented suggestions of a beneficial effect of vitamin D supplementation on depressive symptoms (Anglin et al., 2013; Cheng, Huang, & Huang, 2020; Mikola et al., 2023; Shaffer et al., 2014; Xie et al., 2022), none of these reviews have examined the potential dose-dependent effects of vitamin D supplementation on depressive symptoms to determine the optimum dose of intervention. Some of the available reviews, owing to the limited number of trials and methodological biases, were of low quality (Anglin et al., 2013; Cheng et al., 2020; Li et al., 2014; Shaffer et al., 2014). Considering these uncertainties, we aimed to fill this gap by conducting a systematic review and dose–response meta-analysis of randomized control trials (RCTs) to determine the optimum dose and shape of the effects of vitamin D supplementation on depression and anxiety symptoms in adults regardless of their health status.
===
> even common OTC pain meds can have effect sizes lower than 0.4 depending on the study. Have you ever taken Tylenol or Ibuprofen and had a headache or other pain reduced? Well you’ve experience what a drug with a small effect size on paper can do for you.
I must push back: that's an effect of 0.4 plus placebo effect and time.
There's now RCTs of open-label placebos (where subjects are told it's placebo), which show even open-label placebos are still powerful for pain management. So, I stand by 0.4 being a small effect; even if you took a placebo you know to be placebo, you'd feel a noticeable reduction in pain/headache.
> We found a significant overall effect (standardized mean difference = 0.72, 95% Cl 0.39–1.05, p < 0.0001, I2 = 76%)
of OLP.
In other words, if the effect on antidepressants vs placebo is ~0.4, and the effect of a placebo vs no placebo (just time) is ~0.7, that means the majority of the effect of antidepressants & OTC pain meds is due to placebo.
(I don't mean this in an insulting way; the fact that placebo alone has a "large" effect is a big deal, still under-valued, and means something important for how mood/cognition can directly impact physical health!)
> Researchers have noticed it for 13 years! From the linked Ghaemi et al 2024 meta-analysis
You’re cherry picking papers. Others have already shared other studies showing no significant effects of Vitamin D intervention.
For any popular supplement you can find someone publishing papers with miraculous results, showing huge effect sizes and significant outcomes. This has been going on for decades.
With Omega-3s the larger the trial size, the smaller the outcome. The largest trials have shown very little to no detectable effect.
I think a lot of people are skeptical about pharmaceuticals because they see the profit motive, but they let their guard down when researchers and supplement pushers who have their own motives start pushing flawed studies and cherry picked results.
> In other words, if the effect on antidepressants vs placebo is ~0.4, and the effect of a placebo vs no placebo (just time) is ~0.7, that means the majority of the effect of antidepressants & OTC pain meds is due to placebo.
You keep getting closer to understanding why these effect size studies are so popular with alternative medicine and supplement sellers: They’re so easy to misinterpret or to take out of context.
According your numbers, taking Tylenol would be worse than placebo alone! 0.4 vs 0.7
Does this make any sense to you? It should make you pause and think that maybe this is more complicated than picking singular numbers and comparing them.
In this domain of cherry picking studies and comparing effect sizes, you’ve reached a conclusion where Vitamin D is far and away more effective than anything, placebo is better than OTC pain medicines, and OTC pain meds are worse than placebo.
It’s time for a reality check that maybe this methodology isn’t actually representative of reality. You’re writing at length as if these studies you picked are definitive and your numeric comparisons tell the whole story, but I don’t think you’ve stopped to consider if this is even realistic.
I just picked the most recent meta-analysis I could find, which also specifically estimates the dose-response curve. (Since averaging the effect at 400 IU and 4000 IU doesn't make sense.)
> Others have already shared other studies showing no significant effects of Vitamin D intervention.
Yes, and the Ghaemi et al 2024 meta-analysis addresses the methodological problems in those earlier meta-analyses. (For example, they average the effects at vastly varying doses from 400 IU and 4000 IU)
> According your numbers, taking Tylenol would be worse than placebo alone! 0.4 vs 0.7
No, I understand this fine. Taking Tylenol would give you active medication + placebo + time, which is 0.4 + 0.7 + X > *1.1.* Taking open-label placebo is just placebo + time = *0.7* + X.
(Edit: Also, these aren't "my" numbers. They're from a major peer-reviewed study published in Nature, the highest-impact journal. I don't like "hey look at the credentials here", but I bring it up to note I'm not anti-science, see below paragraph)
===
Stepping back, I suspect the broader concern you have is you (correctly!) see that supplement/nutrition research is sketchy & full of grifters. And at the current moment, it seems to play into the hands of anti-establishment anti-science types. I agree, and I'll try to edit the tone of the article to avoid that.
That said, there still is some good science (among the crap), and I think the better evidence is accumulating (at least for Vitamin D) that it's on par with traditional antidepressants, possibly more. I agree that much larger trials are required.
> They're from a major peer-reviewed study published in Nature, the highest-impact journal.
No, the domain name is nature.com because it's in a Nature Publishing Group journal, Scientific Reports, which is their least prestigious journal.
It's a common mistake, and they do that on purpose, of course, to leverage the Nature brand.
It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously.
> It's also a mistake that implies a complete lack of familiarity with scientific publishing, unfortunately, which makes it a bit difficult to take your judgements regarding plausibility very seriously.
It's still peer reviewed, and as the sibling comment said, more applicable to this type of research. Also you now went from raising understandable objections to refusing the argument because it comes from a specific journal, which doesn't sound very scientific to me
You're right it isn't fair to reject someone's scientific argument just because they seem unfamiliar with how professional science works.
We shouldn't have believed the study more if it actually had been in Nature.
I don't think that's what I was saying, though.
The issue in this thread was about taking a step back and looking at the overall plausibility of the conclusions, taking together multiple studies.
I agree with the GP that the argument doesn't really pass the smell test.
That's still the main issue, and it is something that people who don't understand scientific publishing struggle understanding/doing, because they lack the intuition for how certain results came about.
It’s less prestigious because it doesn’t judge papers on novelty, only on technical accuracy. For incremental research like this, it is an appropriate choice. The lower prestige has no bearing on the accuracy of their findings.
> A statistical error in the estimation of the recommended dietary allowance (RDA) for vitamin D was recently discovered
> ... This could lead to a recommendation of 1000 IU for children <1 year on enriched formula and 1500 IU for breastfed children older than 6 months, 3000 IU for children >1 year of age, and around 8000 IU for young adults and thereafter.
the larger the trial size, the smaller the outcome
I find this a bit surprising. Could there be something else affecting the accuracy of larger trials? Perhaps they are not as careful, or cutting corners somewhere?
Maybe. Those could be the case. But ignoring all confounding factors, this phenomenon is possible with numerical experiments alone. One of the meanings of "the Law of Small Numbers".
Sure, could be just lucky. But if there are several successful small studies, and several unsuccessful large ones (no idea if this is the case here), we should probably look for a better explanation.
It does not require more explanation: publication bias means null results aren't in the literature; do enough small low quality trials and you'll find a big effect sooner or later.
Then the supposed big effect attracts attention and ultimately properly designed studies which show no effect.
Just my hypothesis, but I wonder if larger sample sizes provide a more diverse population.
A study with 1000 individuals is likely a poor representation of a species of 8.2 billion. I understand that studies try to their best to use a diverse population, but I often question how successful many studies are at this endeavor.
If that's the case, we should question whether different homogeneous population groups respond differently to the substance under test. After all, we don't want to know the "average temperature of patients in a hospital", do we?
No, the other way around. It's the combination of two well known effects. Well, three if you're uncharitable.
1. Small studies are more likely to give anomalous results by chance. If I pick three people at random, it's not that surprising if I happened to get three women. It would be a lot different if I sampled 1,000 people.
2. Studies that show any positive result tend to get published, and ones that don't tend to get binned.
Put those together, and you see a lot of tiny studies with small positive results. When you do a proper study, the effect goes away. Exactly as you would expect.
The less charitable effect is "they made it up". It happens.
A point I think is crucial to mention is that “effect size” is just standardized mean difference.
If a minority of patients benefit hugely and most get no benefit, then you get a modest effect size.
This is probably why this discussion always has a lot of people saying “yeah, it didn’t help me at all” and a few saying “it changed my life.”
I believe we should be focusing on more relevant statistical methods for assessing this hypothesis formally. Basically, using mean differences is GIGO if you assume you’re comparing a bimodal or highly skewed distribution to a bell curve.
Hi, I'm the author of the main blog post. Just wanted to say that's a fascinating experience, 160,000 IU a day! I mean, I'm not going to try that, but that's good to hear that 5,000 IU/day for years has been working fine for you. Thanks for sharing!
Hi, author of the blog post here! Thanks for your concern. I do still stand by my claim, since more recent peer-reviewed studies have shown that up-to-10,000 IU is safe. As written in the post:
> McCullough et al 2019 gave over thousands of patients 5,000 to 10,000 IU/day, for seven years, and there were zero cases of serious side effects. This is in line with Billington et al 2020, a 3-year-long double-blinded randomized controlled trial, where they found "the safety profile of vitamin D supplementation is similar for doses of 400, 4000, and 10,000 IU/day." (though "mild hypercalcemia" increased from 3% to 9%. IMHO, that's a small cost for reducing the risk of major depression & suicide.)
So why then does Mayoclinic, etc, all say 4000 IU is the limit? I think because policy is decades behind science (this happened with trans fats), and also policymakers are much more risk-averse. (this is why in California, thanks to Prop 65, up until ~2018, there used to be a warning in every coffeehouse that coffee causes cancer.)
But thanks to your comment, I will edit the intro to note what the official max safe dose is, and that more recent peer-reviewed research shows it's too low!
Hi, author of the blog post here! Thanks for bringing this up -- it wasn't my intention to say one should replace antidepressants with vitamins (the conclusion even says "(Don't quit your existing antidepressants if they're net-positive for you!)", but you're right that the intro may give that impression. I'll edit the intro to say you can stack, not substitute, regular antidepressants.
> many studies in the Vitamin D meta-analysis enrolled patients already taking antidepressants.
Yes, and that's even more encouraging, that there's still effects of Vitamin D on major depression even if already on antidepressants! This suggests we can "stack" the interventions.
Table 1 of the meta-analysis ( https://pmc.ncbi.nlm.nih.gov/articles/PMC11650176/ ) shows the raw sub-group analysis. There were 9 studies on patients using antidepressants, 13 on patients who weren't, the rest were Mixed or Not Reported (...how do 6 studies just not report that?) Anyway,
Effect size of Vit D for people on antidepressants:
−0.54 (−0.85, −0.23)
Effect size of Vit D for people NOT on antidepressants:
−0.28 (−0.40, −0.16)
Both negative. Weirdly, the effect of Vit D seems to be a bit stronger for people on antidepressants, but the difference isn't statistically significant at the p<0.05 level (P subgroup difference is 0.23)
(As for why those effect sizes, -0.54 & -0.28, are lower than what I (and that meta-analysis itself) report, -1.82, that's because the majority of RCTs for any group used far less than 5000 IU. Table 2 in that paper shows the effect (with 95% CI) for various dosages.)
I'll lightly edit my blog post to emphasize stack them, don't substitute. Thanks again for your comment!
> Yes, and that's even more encouraging, that there's still effects of Vitamin D on major depression even if already on antidepressants! This suggests we can "stack" the interventions.
I think the point here is that such a study selects for people where their antidepressants are already known not to work very well for them, or they wouldn't be interested in participating.
Hi, author of the blog post here! Yes thank you for catching this awful typo, it's fixed now! I did write "4000 or 5000 IU of Vitamin D" everywhere else in the article -- main text, conclusion -- just my luck that the one place I mess up is right at the very start.
(Do not take 5000 mg, that's 200,000,000 IU. You'd have to chug dozens of bottles per day)
Colon Blow: "It would take over 30,000 bowls. [ a giant pyramid of cereal bowls shoots up from under the man, who yells in terror as it rises ] To eat that much oat bran, you’d have to eat ten bowls a day, every day for eight and a half years."
Hi, author of the blog post here! Thank you for sharing your experience with antidepressants, I'm really glad it worked for you & made your life better.
I did mention the following at the end of the "antidepressants" section, but reading your comment convinced me to move it further up. The intro now reads:
> The "standardised effect size" of antidepressants on depression, vs placebo, is around 0.4. (On average; some people respond much better or much worse.)
Also, I wasn't expecting my article to do well on Hacker News; thank you everyone for the comments & critiques! I'll edit the blog post as I go along, to refine it in response to your comments.
My personerino let me be ridiculous and fawn a bit, and tell you that you're one of my internet heroes.
Don't take it as criticism, more of a personal take on figuring out what antidepressants do for me. Furthermore, since posting that parent comment I've converted my vit. D dose to IUs and I realised i'm only taking 800 IUs daily. So a thank you for clueing me in on that, and who knows what happens if I up that. Maybe you were right all along and all i DID need was a heroic dose of vitamin D. (... thats what she said)
Thank you for the blog post! I live in New England and always had the winter blues, always just assumed it was because of the weather but never acted on it.
About a week ago, there was a reddit post claiming it's actually geographically impossible for anyone where I live to produce enough Vitamin D naturally from the sun alone, due to the shorter days and lower angles throughout the day. I had no idea.
Thank you! I relate; I live in Montréal, close to New England, with similar climate. The current UV Index for Montréal is... 0. And the current UV Index for Boston is... 0.6. (1.6 later today)
I can't find a rigorous academic source right now, but the top web results all say we need at least UV Index 3 for our skin to be able to make enough Vitamin D. I guess summer may work for us, in the Montreal/New England area, but other than that, yeah, you and I will need to get Vitamin D from diet and/or supplements. And fish is expensive, so supplements it is.
The devs are also running a crowdfunding campaign for this,
with 10% of the donations going to the EFF, Demand Progress, Mozilla, etc:
https://back.nothingtohide.cc
This is the url of the video on that page--just in case you've disabled JavaScript like me and download it with youtubedown[1]: http://www.youtube.com/watch?v=hxvXdwRNDUM
Thanks for that. As noble as I believe the aims, knowing that it's funding Demand Progress has saved me from making a donation I would have otherwise regretted.
At its core, absolutely nothing. In its execution, it's extremely partisan. I'm neither a Democrat nor a Republican, but I've found that, for me at least, I despise when either side slings mud on the other.
Demand Progress isn't quite so blatant, in that they "question" Democrats, or "suspect" that they might be wrong, but similar judgements of the Republicans are downright hateful.
I don't begrudge their right to engage in "my party's right and your party's wrong" sorts of politics, but I will make sure that none of my money gets spent in furtherance of it.
That's a great idea! Perhaps the gamedevs who participated in The Bitcoin Bundle [1] might be the most receptive to the idea of open-sourcing their games.
Not art, but Blender is the biggest example of "buying out for the public" that comes to mind.
Blender used to be proprietary software, and when their company went bankrupt, they crowdfunded 100,000 euros to release their code as open-source. This was a decade ago. Blender is now the most popular open-source 3D animation software today.
Hi, organizer of The Open Bundle here! Thanks for submitting our campaign to HN. We just reached our "public domain release" goal half an hour ago, hopefully this will lead to more indies aiming to earn money by opening their works.
I had some problems when I wanted to pledge earlier in that I just went to the bottom, put in my number and hit credit card and nothing happened. ( safari, with no obvious js errors in console. )
Thanks! As for the form, I'm guessing it was because the required email field (to send you your download link) was left unfilled. It's my bad, I didn't make the "please fill in this field" prompt obvious enough. I just left it as the HTML5 form validation default.
> Please be very careful when someone tries to tell you that supplements are miraculous and pharmaceutical drugs don’t work at all.
I'll concede I unintentionally gave the tone that one should replace antidepressants with supplements, even though the conclusion specifically writes: "(Don't quit your existing antidepressants if they're net-positive for you!) you may also want to ask your doctor about Amitriptyline, or those other best-effect-size antidepressants."
I have now edited the intro to more explicitly say "you can take these supplements alongside traditional antidepressants! You can stack interventions!"
===
> and nobody noticed this massive discrepancy until now?
Researchers have noticed it for 13 years! From the linked Ghaemi et al 2024 meta-analysis ( https://pmc.ncbi.nlm.nih.gov/articles/PMC11650176/ ):
> Several meta-analyses of epidemiological studies have suggested a positive relationship between vitamin D deficiency and risk of developing depression (Anglin et al., 2013; Ju, Lee, & Jeong, 2013).
> Although some review studies have presented suggestions of a beneficial effect of vitamin D supplementation on depressive symptoms (Anglin et al., 2013; Cheng, Huang, & Huang, 2020; Mikola et al., 2023; Shaffer et al., 2014; Xie et al., 2022), none of these reviews have examined the potential dose-dependent effects of vitamin D supplementation on depressive symptoms to determine the optimum dose of intervention. Some of the available reviews, owing to the limited number of trials and methodological biases, were of low quality (Anglin et al., 2013; Cheng et al., 2020; Li et al., 2014; Shaffer et al., 2014). Considering these uncertainties, we aimed to fill this gap by conducting a systematic review and dose–response meta-analysis of randomized control trials (RCTs) to determine the optimum dose and shape of the effects of vitamin D supplementation on depression and anxiety symptoms in adults regardless of their health status.
===
> even common OTC pain meds can have effect sizes lower than 0.4 depending on the study. Have you ever taken Tylenol or Ibuprofen and had a headache or other pain reduced? Well you’ve experience what a drug with a small effect size on paper can do for you.
I must push back: that's an effect of 0.4 plus placebo effect and time.
There's now RCTs of open-label placebos (where subjects are told it's placebo), which show even open-label placebos are still powerful for pain management. So, I stand by 0.4 being a small effect; even if you took a placebo you know to be placebo, you'd feel a noticeable reduction in pain/headache.
EDIT: Here's a systematic review of Open-Label Placebos, published in Nature in 2021: https://www.nature.com/articles/s41598-021-83148-6.pdf
> We found a significant overall effect (standardized mean difference = 0.72, 95% Cl 0.39–1.05, p < 0.0001, I2 = 76%) of OLP.
In other words, if the effect on antidepressants vs placebo is ~0.4, and the effect of a placebo vs no placebo (just time) is ~0.7, that means the majority of the effect of antidepressants & OTC pain meds is due to placebo.
(I don't mean this in an insulting way; the fact that placebo alone has a "large" effect is a big deal, still under-valued, and means something important for how mood/cognition can directly impact physical health!)