What is medicine's 5 sigma?

Another piece exploring the same theme as the previous post - that is the poor quality and high risk of of false positive results in much of the scientific literature. This time my source is an editorial from the Lancet a leading general medical journal. You can read it all here

Richard Horton the editor of the Lancet attended a meeting at the Welcome Trust in London. The theme of the meeting was the current state of the biomedical research literature and the endemicity of bad research. As one speaker put it - poor methods get results. Another stated that too any scientists sculpt their data to fit their preferred theory of the world or retrofit their hypotheses to fit the data (see the previous post about arrows and targets)
The current estimate is that about half of studies contain false positive results. So when proponents of the use of colour to treat visual stress ( and any number of other conditions) say that research in the peer reviewed literature has shown that ............ ( you can fill in the blank) that is not enough.These studies have to be looked at in detail.

A number of factors were identified that put studies at high risk of producing false positive associations. These were

1) Small sample sizes
2) Tiny effects
3) Invalid exploratory analysis
4) Flagrant conflicts of interest
5) Pursuing fashionable trends of dubious importance

Much of the research on visual stress in dyslexia ticks all of these boxes. While this does not remove the need to read each study in detail and with  crtitical eye the statement that research in the peer reviewed literature has shown that........... is not nearly as impressive as it sounds.
Onwards with more reviews