Placebo effects

To suggest that the reported beneficial effects of coloured lenses and overlays could be due to the placebo effect or more properly placebo effects causes offence. Replies of the ‘I have worn overlays for x years and I could not possibly be subject to the placebo effect’ sort usually come thick and fast. Most people acknowledge the existence of the placebo effect but believe themselves to be too sophisticated to be affected themselves. The truth is that we are all vulnerable and it’s no bad thing.

I have experienced the placebo effect myself when I obtained pain relief from a morphine infusion that wasn’t even working. At first, I felt humiliated but on reflection, it was great to get pain relief without the small but significant risks of intravenous morphine. It is not only medical interventions that provoke placebo effects.  I like playing tennis socially and badly. From time to time it's good to try something new to boost my position on the tennis ladder; a new racket perhaps with a novel stringing pattern. Recently I tried some fancy new strings -‘biphase multifilament’ which also happened to be red. I did play a little bit better for a while although more so in my own estimation than my opponents. However, in spite of my subjective opinion, my position near the bottom of the tennis ladder didn’t change and what improvements I have made have come about through practice and one to one coaching.

What I am trying to say is this. When I discuss the importance of placebo effects I do not do so from a position of superiority I am just as vulnerable  (or blessed) as the next person in this regard. However, in appraising clinical trials of coloured overlays you can’t get away from inadequate attempts to control for research participation effects.

The components of the placebo effect, which need to be considered in relation to trials of coloured overlays and lenses are
1) Novelty effects, that is, the use of an intervention that is new and exciting may enhance motivation and performance.
2) The effect of being observed and tested, which is sometimes called the Hawthorne effect.
3) The therapeutic ritual of the intervention itself.
4) The relationship with the carer.

The usual way around this problem is to have a placebo control group and ensure that both research participants and experimenters are blinded to which intervention the subject is receiving - a so called double blind trial. I acknowledge that this is not easy in studies of coloured overlays and lenses but it is not impossible.

In Arnold Wilkins’1994 study (1) this was achieved by assessing subjects with the intuitive colorimeter so that they did not see the actual lenses they are going to receive and by leaving an interval between the assessment and being given the experimental and control lenses. By this means 23 subjects did not know which lens matched the colorimeter setting 10 thought the experimental lens matched and 10 though the placebo lens matched. Unfortunately, this trial was severely compromised by attrition of participants which is a common problem in studies of coloured overlays and lenses. The study failed to show any difference in reading rate, accuracy or comprehension between optimum filter and control filter for the 45 of 68 subjects for whom data were available. This study also provides some evidence for novelty effects. 31 subjects preferred the first lens they were tested with 17 the second and 4 expressed no preference. This is by far the most statistically significant result of the whole study!

Some studies made no attempt to mask subjects or experimenters to the intervention being used but claimed that they were able to control for this by enhancing the placebo effect of the control intervention. For example, in the trial reported by Bouldoukian and colleagues (2) the placebo overlay consisted of a pale yellow filter which would have been distinct from the more saturated colours of Intuitive Overlays^TM.  The placebo filter was described to the subjects as ‘a new filter from the United States where it was thought to be a wonderful discovery’. It was labelled ‘Research Model A16 Anti UV IR Filter. Made in the USA’. The authors argued that, as a result of the enhancement of the placebo overlay, the modest increase in reading speed on the Wilkins Rate of Reading Test (WRRT) that was observed (4% on average) was unlikely to be accounted for by placebo effects. The unstated assumption, that it is possible to measure the placebo effect of the experimental intervention and balance it, by enhancing the placebo effect of the control intervention, is scientifically unfounded. The act of selecting the optimum tint from a set of overlays is likely to involve a more prolonged relationship with the experimenter and a richer therapeutic ritual, both of which are powerful drivers of the placebo effect (3).

Overall there is strong evidence in the literature that placebo effects play a big role in the reported benefits of coloured lenses and overlays to improve reading in subjects with visual stress.

Those trials that do have a proper placebo control report no improvements in reading. For example Wilkins 1994, Robinson Foreman 1999 (4) (for the three months it was a placebo controlled study), Mitchell 2008 (5)  and Ritchie 2012 (6). Those trials with enhanced placebo report small effects and those trials with no placebo control report significant effects. This gradient points strongly to a placebo effect.

Sometimes this gradient can even be seen within trials. The Robinson Foreman study of 1999 (4) and Mitchell 2008(5)  contain a placebo control and an untreated control group. There was a significant difference between the experimental lens group and the no treatment control group but again no significant difference between experimental lens and placebo lens groups.

Conclusion 
There are no studies that demonstrate improvements in reading with overlays that can not be more plausibly explained by placebo effects.

1.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70. 

2.         Bouldoukian J, Wilkins AJ, Evans BJW. Randomised controlled trial of the effect of coloured overlays on the rate of reading of people with specific learning difficulties. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 2002 Jan;22(1):55–60. 

3.         Kaptchuk TJ, Kelley JM, Conboy LA, Davis RB, Kerr CE, Jacobson EE, et al. Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome. BMJ. 2008 May 3;336(7651):999–1003. 

4.         Robinson GL, Foreman PJ. Scotopic sensitivity/Irlen syndrome and the use of coloured filters: a long-term placebo controlled and masked study of reading achievement and perception of ability. Percept Mot Skills. 1999 Aug;89(1):83–113. 

5.         Mitchell C, Mansfield D, Rautenbach S. Coloured filters and reading accuracy, comprehension and rate: a placebo-controlled study. Percept Mot Skills. 2008 Apr;106(2):517–32. 

6.         Ritchie SJ, Della Sala S, McIntosh RD. Irlen colored overlays do not alleviate reading difficulties. Pediatrics. 2011 Oct;128(4):e932–8.

'Prolonged use of coloured overlays for classroom reading'

Jeanes R, Busby A, Martin J, Lewis E, Stevenson N, Pointon D, et al. Prolonged use of coloured overlays for classroom reading. Br J Psychol. 1997 Nov;88(4):541–8. 

You can download this paper from Arnold Wilkins webpage here

The paper starts with some background information about Intuitive Overlays vs Irlen Overlays and the authors make their point by plotting the chromaticity coordinates of Irlen overlays and Intuitive overlays. However, they exaggerate the difference between the two systems by plotting both the x and y coordinates on different scales. See the figure below. In my opinion, this is poor practice in scientific reporting.

The authors exaggerate the difference between Irlen and Intuitive overlays by plotting the chromaticity coordinates on different scales and with different symbols. This is poor practice in representing data and it is noteworthy that one of the authors has a financial interest in Intuitive Overlays; the system that benefits from this distorted representation.

Prevalence of 'visual stress'
In the first part of the paper, the authors estimate the prevalence of visual stress in 93 children attending a primary school and 59 children attending the first year of secondary school. When asked to look at random letters (not naturalistic text) through overlays 51% of the former group and 54% of the latter chose an overlay to ameliorate perceptual distortions. When the group was reassessed at two months 35% of the primary school group said they were still wearing lenses and wished to continue (note this is based on self-report without independent verification). Similarly, 36% or secondary school children were still using and wished to continue with overlays. By ten months this had dropped even further so that 14% of the primary school group were still using overlays or glasses and only 5% of the secondary school group. By the criterion of voluntary sustained use, the prevalence of visual stress in a mixed ability group of children should be between 5-14%. What is remarkable about this and other papers looking at coloured overlays and lenses to treat visual stress is the astonishing rate of attrition over time.

Clinical trials
The paper also contains two trials of sorts. Both are at high risk of bias and use a non-standardised reading test of unknown relevance to real world reading. The Wilkins Rates of Reading Test (WRRT).
Those who continued using overlays for ten months were tested with and without overlays. There was no placebo group and the study was not masked. The authors found a small difference in words per minute on the WRRT.
However, another part of the paper describes a small study which did have a placebo control overlays of a complementary colour. This time there was no statistically significant difference between the two groups.

Conclusion
Overall some questionable reporting and what appears to be a posthoc data trawl. The data does not provide compelling evidence for the treatment of visual stress with colour.