News

Scientists should ditch ‘statistical significance’, say experts

A better more contextual way of evaluating scientific results is needed, it is claimed.
A better more contextual way of evaluating scientific results is needed, it is claimed. A better more contextual way of evaluating scientific results is needed, it is claimed.

A central tenet of science, the concept of “statistical significance”, should be abandoned, a group of influential experts claim.

Statistical significance is what gives scientific findings weight.

Without it, results are often not taken seriously.

This has serious implications for studies of the effectiveness or potential hazards of new medicines, for instance.

If a drug’s effectiveness falls below the threshold of statistical significance it tends to mean going back to the drawing board.

A computer screen displaying statistical information (Pixabay/PA)

Statistical significance is based on a mathematically-derived probability value, or p-value, a number between zero and one.

Generally, a p-value of less than 0.05 corresponds to a statistically significant effect or association.

It is this fundamental idea that has been rejected by experts in a special issue of The American Statistician journal.

The review of 43 international papers is expected to lead to a major re-think of the “significance” concept.

According to a press release accompanying the article, it could move science into “a new age” in which more attention is paid to context.

The three American authors of the editorial write: “Based on our review of the articles in this special issue and the broader literature, we conclude that it is time to stop using the term ‘statistically significant’ entirely.”

They add: “No p-value can reveal the plausibility, presence, truth, or importance of an association or effect.

“Therefore, a label of statistical significance does not mean or imply that an association or effect is highly probable, real, true, or important.

“Nor does a label of statistical non-significance lead to the association or effect being improbable, absent, false or unimportant.”

The review authors are Dr Ronald Wasserstein, executive director of the American Statistical Association (ASA), Dr Allen Schirm, retired from the US policy research institute Mathematica Policy Research, and Professor Nicole Lazar, from the University of Georgia.

They argue that “for the integrity of scientific publishing and research dissemination” the fact that a p-value passes or fails to pass an “arbitrary threshold” should “not be considered all when deciding which results to present or highlight”.

The authors recommend that in place of “one-size-fits-all” p-values the scientific community should adopt the “Atom” model.

The acronym stands for “Accept uncertainty, be Thoughtful, be Open, be Modest”.