Holy trinity number one -

This is the first of the three papers that are frequently cited together in support of the treatment of visual stress in poor readers.

Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt 1994 Oct;14(4):365–70.

Pubmed

Arnold Wilkins web page where you can download this reference if you wish

At least this paper was a big step up
from this…..

It would be a good idea to start with the good things about this paper which  was  massive step forward from what went before – the opinions of a self proclaimed revolutionary. At least the authors believed that this subject was amenable to the scientific method and should be tested according to the standards that apply in other areas of science.

It is also important to state that these kind of studies require a prodigious amount of work and studies on dyslexia are particularly difficult. Furthermore, the first randomised controlled trial is often problematic. Unexpected problems and sources of bias are frequently encountered.

So rather than be critical of the study itself I am critical of the ‘spin’ that has been put on the data in the discussion and the way this study has been selectively cited as 'proving' that precision tinted lens can ameliorate visual discomfort in individuals with ‘visual stress’. This study produced negative results that do not support the use of precision tinted lenses to treat  ‘visual stress’ in poor readers.

So what about the paper.

The subjects were 68 poor readers drawn from a school for dyslexics and two state schools.

The criteria for diagnosing visual stress was voluntary sustained used of overlays.

The study design was a crossover one so that subjects received the optimum tint determined with the ‘intuitive colorimeter’ and another tint which was found not to ameliorate their visual distortions. They were randomised to either receive experimental lens or control lens first. As far as we can tell allocation concealment was good. The subjects wore one lens for one month followed by two weeks with no lens, then one month with the other lens

The intuitive colorimeter was used to determine the 

optimum tint and a closely related tint which did not

ameliorate their visual distortions

The two outcome measures were reading accuracy speed and comprehension measured with the Neale Analysis of Reading test and the symptom diaries that participants in the trial kept.

The principle problem with this study was the high rate of dropout and failure to analyse the data on an intention to treat basis. High drop out rate can be a problem with cross-over studies, which of necessity last longer than parallel trials. The latter compares two groups randomised to receive one treatment or the other.

Neale Analysis of Reading Test

Data were available for 45 of 68 subjects. Both placebo group and experimental group improved over the course of the study but there was no statistically significant difference for reading speed, accuracy or comprehension between the two groups.

Symptom diaries

This is where the data starts to be spun. Unfortunately, data was only available for 36 of 68 participants. That is, nearly 50% of the data is missing. While some missing data is the norm; a study with this rate of drop out is 'pretty fatally holed' below the water line. 19 subjects had more good days with the experimental lens and 17 subjects had the same number of good days or more good days with the placebo lens. Without data from more of the other 32 subjects, no meaningful conclusions are possible.

The authors claim that their 'positive result' should not be seen as an artefact of the high rate of attrition. They have no reason for that belief.  It is more likely that those who drop out of a study are obtaining no benefit. This claim constitutes the sort of wishful thinking that good peer reviewing should have eliminated. The study also contains conflicting data. When asked at the end of the study which lens they preferred 22 children preferred the experimental lens and 26 preferred the control lens. 

The strange thing is that the most robust part of the study looking at the Neale Analysis of Reading, with a follow up of 45/68 = 66% is seldom mentioned. Yet the least robust part of the study, looking at symptom diaries, with a follow up of just 36/68 = 53% has been hyped to the point of absurdity. Particularly when you remember that the same study produced conflicting evidence. When asked, a small majority of children preferred the control lens. 

This was really a pilot study (nothing wrong with that) which should have pointed to the way to further better studies.

The holy trinity: three papers commonly used to justify the treatment of visual stress in dyslexia

When people are trying to make a case for treating visual stress to ameliorate reading difficulties they frequently cite three papers. For example, this power point file was taken from a lecture given to optometry students by a well known proponent of coloured lenses and overlays. The first three papers cited (listed below)  are said to validate treatment with coloured lenses and overlays. I will be reviewing these in some detail in subsequent posts. Ignoring any issues of cherry picking, taken individually or as a whole these papers do not come close to forming a credible evidence base for the treatment of visual stress in dyslexia.
Another example comes from a publication in the March 2015 issue of the Scandinavian Journal of Occupational Therapy - Using coloured filters to reduce the symptoms of visual stress in children with reading delay. The authors state 'that this type of intervention has been found to ease the discomfort experienced and improve reading accuracy' - citing references 1 and 3 below. Subsequently they claim that randomised controlled trials show that coloured filters can improve reading speed citing reference 2 below.

So, over the next few weeks I will be reviewing these papers in some detail. In my experience they are frequently cited but I suspect some people a only read the title or the abstract or worse still, copy the citation from other papers without going back to basics and critically appraising the text. 
This will be a somewhat dry process but, as ever, the devil lies in the detail. Sometimes those who produce superficially impressive lists of references to back up their claims rely on the fact that most people do not have have time or inclination to carefully read and appraise the original papers.

The first paper is
1.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70. (pubmed link)

The second
2.         Bouldoukian J, Wilkins AJ, Evans BJW. Randomised controlled trial of the effect of coloured overlays on the rate of reading of people with specific learning difficulties. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 2002 Jan;22(1):55–60. (pubmed link)

And third
3.         Robinson GL, Foreman PJ. Scotopic sensitivity/Irlen syndrome and the use of coloured filters: a long-term placebo controlled and masked study of reading achievement and perception of ability. Percept Mot Skills. 1999 Aug;89(1):83–113. (pubmed link)

You can obtain the first two publications from Arnold Wilkins' web page unfortunately number 3 is hidden behind a pay-wall in a pretty obscure, low impact factor, journal. Most university libraries do not have it.