This is the second of the three papers that are frequently cited together in support of the treatment of visual stress in poor readers.
Bouldoukian J, Wilkins AJ, Evans BJW. Randomised controlled trial of the effect of coloured overlays on the rate of reading of people with specific learning difficulties. Ophthalmic Physiol Opt. 2002 Jan;22(1):55–60.
pubmed link
Download from Arnold Wilkin's web-page
 |
The IOO markets the products used in this study-
Intuitive Overlays and the Wilkins Rate of Reading Test
|
The paper gets off to bad start in the abstract with the claim that ' a randomised controlled trial has demonstrated that, for selected children with reading difficulties individually prescribed coloured filters reduce the symptoms of asthenopia'. As you will recollect from the previous post, this part of the study was severely hampered by a high drop out rate of just under 50%. As a result it is not possible to draw meaningful conclusions. The authors neglect to mention that the more statistically robust part of the study showed no improvement in reading speed, accuracy or comprehension using experimental tint compared to control tint.
The subjects were 4 adults and 29 children already attending the specific learning difficulties clinic at the Institute of Optometry (IOO). The IOO a self financing charity that sells overlays.
The criteria for diagnosing visual stress was voluntary sustained use of overlays for some subjects and immediate amelioration of symptoms for the others.
The study was of a crossover type so that each subject received an experimental overlay determined using the Intuitive Overlays testing schedule (to be described in a future post) and a placebo overlay which was a straw coloured UV blocking filter marked Research Model A16 Anti UV/IR Filter. Made in the USA. The filter was described to the subjects as a wonderful discovery to help patients with reading difficulties. It was argued that this counteracted problems with the placebo effect arising from what was a non masked study. More of this unfounded assumption later.
Another important feature of this study was the use of the Wilkin's Rate of Reading test (WRRT). Having obtained a negative result with the widely used Neale Analysis of Reading Test the authors argued that it was dependent on higher level linguistic skills (not true as the test measures reading speed and accuracy as well as comprehension. So they developed their own test which consists of common words but no structure or syntax. I have never met anyone who doesn't find it unpleasant to read and we do not really know what, if anything, it actually measures.
Results
The results showed a modest increase in reading speed which averaged 4% using the experimental overlay, using a surrogate outcome measure - the WRRT see left. In fact, these results are very unimpressive.
There are five main problems with this paper, which will be discussed in turn.
1) Selection bias
2) The placebo effect,
3) The significance of reading jumbled text faster
4) The criteria used to diagnose visual stress
5) Conflicts of interest.
Problem One - selection bias
The subjects were already attending the IOO special learning difficulties clinic. We need to know more about how they got there. Where they brought by their parent specifically seeking treatment with overlays? In which case they could be described as 'believers' - as well as the workers as at the IOO.
It is important to note that this was not a sample taken from a classroom situation so even if you think the results of this study are significant, it is not clear that they can be generalised to the classroom .
Problem Two - controlling for placebo effect(s)
This study was not masked and both experimenters and subjects knew whether control lens or experimental lens was being used. The authors argue that they were able to control for this by calibrating the placebo effect for each intervention and ensuring that they were matched so that any difference observed was due to the experimental overlay. This is a completely unfounded assumption and there is good reason for believing that the placebo effect would have been more pronounced in the experimental overlay group. An enhanced relationship with the practitioner is one the most powerful drivers of the placebo effect(1). Choosing the experimental overlay would have involved much more contact and dialogue with the experimenters and since they were based at the IOO it is likely that they were 'believers'. Furthermore, they were not attentional controls and participants were more likely to perceive value in overlays they had spent time choosing themselves compared to an 'off the shelf' model that was just given to them.
Most children I know would have seen straight through the 'Research Model A16 Anti UV/IR Filter. Made in the USA' as a shallow attempt to mislead them. So the authors claim that they could, in effect, balance the placebo effects of these two interventions is not credible and should not a have got past the peer reviewers.
Problem Three - relevance of the WRRT
The Wilkins rate of Reading test consists of jumbled text. The relevance of increase speed of reading jumbled text to real world reading was unknown at the time of this paper. Subsequent research has shown that improvements of reading of the WRRT with coloured overlays do not generalise to naturalistic text nor are they sustained over time(2). More on the WRRT in the future.
Problem Four
This study has no relevance to current diagnosis and management of visual stress where a commonly used criterion is reading 10% faster on the WRRT with a coloured overlay. The average increase in reading speed in this study was 4% compared to placebo and very few of the subjects would be diagnosed with visual stress according to current criteria.
Results
There are five main problems with this paper, which will be discussed in turn.
1) Selection bias
2) The placebo effect,
3) The significance of reading jumbled text faster
4) The criteria used to diagnose visual stress
5) Conflicts of interest.
Problem One - selection bias
The subjects were already attending the IOO special learning difficulties clinic. We need to know more about how they got there. Where they brought by their parent specifically seeking treatment with overlays? In which case they could be described as 'believers' - as well as the workers as at the IOO.
It is important to note that this was not a sample taken from a classroom situation so even if you think the results of this study are significant, it is not clear that they can be generalised to the classroom .
Problem Two - controlling for placebo effect(s)
This study was not masked and both experimenters and subjects knew whether control lens or experimental lens was being used. The authors argue that they were able to control for this by calibrating the placebo effect for each intervention and ensuring that they were matched so that any difference observed was due to the experimental overlay. This is a completely unfounded assumption and there is good reason for believing that the placebo effect would have been more pronounced in the experimental overlay group. An enhanced relationship with the practitioner is one the most powerful drivers of the placebo effect(1). Choosing the experimental overlay would have involved much more contact and dialogue with the experimenters and since they were based at the IOO it is likely that they were 'believers'. Furthermore, they were not attentional controls and participants were more likely to perceive value in overlays they had spent time choosing themselves compared to an 'off the shelf' model that was just given to them.
Most children I know would have seen straight through the 'Research Model A16 Anti UV/IR Filter. Made in the USA' as a shallow attempt to mislead them. So the authors claim that they could, in effect, balance the placebo effects of these two interventions is not credible and should not a have got past the peer reviewers.
Problem Three - relevance of the WRRT
The Wilkins rate of Reading test consists of jumbled text. The relevance of increase speed of reading jumbled text to real world reading was unknown at the time of this paper. Subsequent research has shown that improvements of reading of the WRRT with coloured overlays do not generalise to naturalistic text nor are they sustained over time(2). More on the WRRT in the future.
Problem Four
In the figure on the left I have attempted to add a 10% faster line and it can be seen that only two or at most three of the subjects of this study would actually be diagnosed with visual stress according to current criteria.
Problem Five - conflict of interest
The final paragraph of the study states the 'The medical Research Council (MRC) owns the rights to the Intuitive overlays and Rate of Reading Test. Arnold Wilkins receives an 'Award to Investors' from the MRC. These products are available from IOO Marketing Ltd which raises funds for the Institute of Optometry, a registered charity.
Conclusion
The most generous possible interpretation of this study is that if you have to read jumbled text faster over the short term, overlays might be useful.
Problem Five - conflict of interest
The final paragraph of the study states the 'The medical Research Council (MRC) owns the rights to the Intuitive overlays and Rate of Reading Test. Arnold Wilkins receives an 'Award to Investors' from the MRC. These products are available from IOO Marketing Ltd which raises funds for the Institute of Optometry, a registered charity.
Conclusion
The most generous possible interpretation of this study is that if you have to read jumbled text faster over the short term, overlays might be useful.
1. KaptchukTJ, Kelley JM, Conboy LA, Davis RB, Kerr CE, Jacobson EE, et al. Components of placebo effect: randomised controlled trial in patients with irritable bowelsyndrome. BMJ. 2008 May 3;336(7651):999–1003.pubmed link
2. Henderson
LM, Tsogka N, Snowling MJ. Questioning the benefits that coloured overlays can
have for reading in students with and without dyslexia:. J Res Spec Educ Needs.
2013 Jan;13(1):57–65. Link to Wiley online where you can download this paper
No comments:
Post a Comment