> One who grasps statistical significance can better interpret health studies ...
Ok, that's where I draw the line. Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston. Its biggest failure is that it gives a false sense of security. It's what lead us to the reproducibility crisis. This should not have a place in an article that purports to emphasize the importance of statistics in the school curriculum.
Unfortunately we will not get rid of statistical significance anytime soon. Teaching it and explaining its shortcomings is much better than sweeping it under the rug, and perhaps it will help the next generations of scientists not falling into the same trap.
>Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.
The term-of-art "statistical significance" isn't bs because we ultimately have to choose what to pay attention to. Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
The observed effect of <something> is either:
(1) appearance of cause & effect is actually not there and just random chance or coincidence
That's true but the abuses still doesn't eliminate the need to name the concept of real cause & effect vs random chance. Whatever alternative mental framework one uses to decide which of those scenarios is happening, you will arrive at something that looks like "statistical significance" even if you don't call it that.
As an analogy, even though "averages" in math is misused, it doesn't mean "averages are bs".
Sorry, but statistical significance has a pretty narrow meaning, and it's wrong.
> Removing "statistical significance" from the vocabulary doesn't change the underlying reality about people deciding what to do based on a threshold of a number.
I'm not arguing to remove it from the vocabulary. I'm arguing that that bloomin' threshold is the source of many problems.
The article also isn't about methodology, it's about the secondary school curriculum. To provide pupils with a false understanding of statistics is just bad. And to teach them why it's bad, is hard.
There's a lot of technical stuff, but you could look into Ioanides articles [1], and the book Bernoulli's Falacy by Clayton [2] is a good introduction, also for people less involved in statistics.
> Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston.
Ok, that's where I draw the line - statistical significance is not "bullshit" - however as you say, leaning on it too hard can cause things to break quite badly. Scientists misusing it do not negate all the medical advances we have made from moving to a significance-based system. It is an absolutely essential tool for people using statistics to understand, but its limitations must be emphasised when taught, and it must be understood that it is a tool, not a conclusion. Also other alternatives should be taught more widely (e.g. Bayesian inference).
We would have made better progress if we had skipped NHST. Mind you, nor Fisher, nor Pearson were in favor of it. It is a tool that should be taught after better ways, and then only to understand the past. Like phlogiston.
I don't even know what you mean by "under-powered statistical training."
But about the harder sciences: when this ball got rolling, I attended a lecture of a statistician, who explained that basically all genetic results preceding were likely wrong, unless they showed something like 6 sigma significance. That's because H0 is so easy to reject when you base H0 and H1 on different measurements. The result is that every theory is true.
Many scientists in non-mathematical fields are wrote taught "how to write a paper" in undergrad. P-values and statistical significance are taught completely devoid of context, essentially just a step in your final analysis. Many scientists perpetrating p-hacking or data dredging thought this was a process of good science, and didn't understand the axioms for which these metrics depend. This is something learning fixes, and ignorance makes worse.
It comes from the fact that soft sciences deal with vast numbers of variables, far too many to control. They actually get a ton of statistics, trying to find a signal in so much noise.
It would be great if human beings were more amenable to rigorous experiment. Failing that, we at least need to understand what these things do and don't actually mean. It's either that or give up trying to study people entirely.
Statistics are not values you collect - but the analysis you perform. In data with many confounding variables or many degrees of freedom, the only way to be honest about what the data shows is through statistics. Statistics is what allows you to filter signal from noise. Statistical significance is a very useful tool, you just have to be honest about what it represents and under what conditions this breaks down.
Ok, that's where I draw the line. Statistical significance is bullshit. Learning about it is as useful as learning about phlogiston. Its biggest failure is that it gives a false sense of security. It's what lead us to the reproducibility crisis. This should not have a place in an article that purports to emphasize the importance of statistics in the school curriculum.