Holy trinity number 3 - a missed opportunity and a seriously misleading title

This is the third of three papers that are frequently cited together in support of the treatment of visual stress in poor readers.

Robinson GL, Foreman PJ. Scotopic sensitivity/Irlen syndrome and the use of coloured filters: a long-term placebo controlled and masked study of reading achievement and perception of ability. Percept Mot Skills. 1999 Aug;89(1):83–113
Pubmed link

Unfortunately, this paper is hard to obtain and the journal is not stocked by most University Libraries. The cost of downloading a copy from the publisher's website is quite unreasonable. However, two systematic reviews contain useful summaries and critical reviews of the findings.
First, a systematic review by the West Midlands Health Technology Assessment Board that you can download here. Second, an excellent literature review commissioned by the New Zealand Ministry for Health, written by Christine Malins, that you can download here. I can not recommend these two reviews highly enough.
The original paper is heavy going and would have benefited from a firm editor.
Like much research in this area, this study was hampered by an a-priori assumption that treatment works and that it would, therefore, be unethical with treat with placebo filters for prolonged periods. In fact, studies of tinted lenses and overlays have consistently shown that the placebo group improves as much the experimental group. Furthermore, a study design that is unlikely to yield meaningful results is also unethical.
A good thing about the study was the large sample size of 113 students with reading difficulties, ranging from 9-13 years of age. All individuals had moderate to high levels of visual stress according to Irlen methodology.
The 113 students were split into three groups. Group 3 initially received an optimum tint that was determined according to Irlen protocols. Group 2 a placebo tint of a closely related colour and group 1 a blue tint. Reading was assessed using the Neale Analysis of Reading Test.
On the surface, it appears that there was a prolonged follow up of 18 months. In fact, both the placebo tint group and blue tint group was changed to diagnosed tint between 3 and 4 months.  The title is misleading, it was a short term placebo controlled trial followed by a long term observational study.

Only the first 3-4 months of this trial was placebo controlled and masked

A second problem was the 'control' group which was not really a control group at all. It consisted of 35 poor readers without visual stress who received no tint at any stage of the study (see diagram below) This means any comparisons between groups one, two and three and the control group can not be described as placebo controlled and masked. This part of the study was purely observational with all the opportunities for bias that entails.

The authors also state that it was a double blind cross over design. Not so, or at least it was a very badly designed crossover. It is true that subjects crossed over from placebo to diagnosed tint but a proper crossover design would have had some participants crossing from diagnosed tint to placebo. The problem is that you often get improvements in performance on changing test conditions purely because of novelty effects. This study failed to control for that bias.

The control group did not have visual stress and did not receive treatment

Results
Looking at the only part of the study that matters - what happened during the first 3-4 months the three treatment groups improved faster than the untreated control group but crucially there was a significant difference in reading speed, accuracy, or comprehension between placebo tint, blue tint or diagnosed tint.

Conclusion
In my opinion, the title and the abstract are misleading. It is a negative study that failed to demonstrate that those receiving diagnosed tint, improved more than those receiving blue or placebo tint. Despite this, it is frequently cited in support of treatment of visual stress with colour in the overlap group who also have reading impairment.