Bruce Evans Lecture to the British Dyslexia Association 2016 (part one)

Bruce Evans is one of the leading proponents of visual stress treatment to ameliorate reading difficulties.   Professor Evans has even given a lecture of the subject of evidence-based optometry to the college of optometrists in which he extols the virtues of randomised controlled trials and systematic reviews. However, in his promotion of the coloured lens and overlays I think greater attention could be paid to these principles.
It is to his great credit that he makes his lecture handouts available online where they can be downloaded and reviewed by interested parties. I regret that I could not be at his lecture but perhaps having the handout is the next best thing. I acknowledge that some nuances of the lecture may be lost by only viewing the power point files. Nonetheless in the same spirit of openness I propose to argue against some of the points he makes.

Background (slide 4) 

This presents what is an essentially a Tony Blair  'new labour' style of argument. By presenting a false dichotomy Bruce Evans places himself in what he sees as the centre-ground. The implication  seems to be that skepticism is a dirty word rather than a virtue. There are two types of zealots those who unquestioningly believe the evidence for the treatment of visual stress and those who unquestioningly dismiss it. It is the skeptic who occupies the middle ground reviewing the evidence as it comes and forming opinions based on that evidence.
Bruce Evans then goes on to ask why is the subject still controversial? I would argue that in scientific circles it is not controversial - most scientists do not take the concept of treating visual stress to ameliorate reading disorders seriously. A recent review in Nature Reviews in Neuroscience listed it among neuromyths. Reports prepared by the Royal College of Ophthalmologists in the UK and by the American Academy for Pediatrics, American Academy of Ophthalmology, Council for Children for Disabilities. American Association for Pediatric Ophthalmology and Strabismus and the American Association of Certified Orthoptists in the USA have argued against this Treatment. Reviews prepared for the New Zealand Ministry for Health, by the West Midlands Health Technology Assessment Board and for a number of American Health Maintenance Organisations (HMOs) have concluded that the evidence does not support the use of this treatment. What evidence there is comes form a small handful of individuals many of whom have a financial interest in coloured lenses and overlays. In my opinion this is not a controversial treatment; rather, it is a fringe activity.

 Pitfalls in researching VS (slide 5)

This slide contains a number of statements that are in my opinion of questionable accuracy.
1) 'Mitchell et al (2008) did not study people with VS' This is in fact a grey area. Mitchell studied subjects with dyslexia who complained of visuo-perceptual distortions and who chose a colour  which reduced those distortions using the Intuitive Colorimeter. In addition the pretests scores on the Irlen Differential Test Schedule were high. It seems likely that the majority of participants did have 'visual stress'.
2) Ritchie et al 20011 used a limited range of colours. Not true - used the Irlen system which is nearly the same as the Wilkins system.
3)'With standard reading tests need to read for > 10 minutes before VS and colour benefits'  Tyrell et al 1995-  This paper used the same tinting system as Ritchie but now the results seem to perfectly acceptable. Actually Tyrell et al is an exploratory study which is at best hypothesis generating see blogpost for August 2015 'A trial that does not quite stack up' for a more detailed account.


Slide 5
 Slide 5 report on a review that has been conducted but is not yet published looking at intuitive overlays and precision tinted lenses ie the Wilkins system. As discussed in the March blog a systematic review looks at all published research in a given area rather than cherry picking that which suits an argument. However the next stage is crucial. That is to appraise all papers using the same template to assess the risk of bias.

CASP check list for assessing RCTs

This is bias in the statistical sense not the personal prejudice sense. If controlled trials are not set up properly they can easily 'veer-off' one way or the other like a car in which the tire pressures are unequal or with poorly balanced steering. The idea of a systematic review is that you only include studies at low risk of bias. They are not, as some proponents of visual stress treatment seem to think, a 'body-count' of all trials. You could do one hundred poorly controlled studies at high risk of bias and that would not trump one well controlled study at low risk of bias. The particular tool used here is the CASP tools for appraising randomised controlled trials. Which are summarised in table one above.  There are no trials at low risk of bias to support the use of the Wilkins tinting system using the CASP or any other widely used system for assessing bias.

Slide 12 - Research with the intuitive colorimeter
On slide 12 it is stated that there are three studies that support the use of the intuitive colorimeter.

Slide 12 from Bruce Evans presentation

The three studies cited by Evans are all at high risk of bias using the CASP tools . The studies have been reviewed in the blog-post of September 2015 -Intuitive Colorimeter: Technique for the 2st century?-
A study which is not mentioned was performed in an independent laboratory using a psychophysical endpoint which showed no improvement in contrast sensitivity function at the spatial frequencies that are said to be aversive in visual stress - see blogpost of September 3 2015.
Professor Evans makes much of the the RCT published by Arnold Wilkins in 1994. This study  has been reviewed in detail elsewhere see Holy Trinity number one. It was indeed a well masked study. However, according to question 6 of the CASP tools (see checklist above) it was at risk of bias because of a high rate of attrition and should have been excluded from a systematic review or at least excluded form any meta-analysis. Of the 68 participants who enrolled on the study, subjective comfort data was only available on 35 (51.5%) and for reading, data was available for 45 (66%) of subjects. The high rate of attrition could easily have introduced bias, For example there was also evidence of novelty effects. 31children preferred the first pair of glasses and 17 preferred the second pair. Although at the start of the trial equal numbers may have started with chosen tint and placebo tint as a result of attrition equipoise may well have been lost - that is the problems with attrition although your groups may well have well matched and randomised at the start of the study losses to follow up are seldom purely random. It is generally accepted that a study with more than 15% drop out rate is high risk of bias. It is also somewhat disingenuous to mention the subjective comfort data while not mentioning the reading data for which the rate of attrition was lower. This part of the study showed no improvement in reading rate, accuracy and comprehension using the chosen tint.
The final line of slide 12 is perhaps the oddest part of the whole presentation - Wilkins et al (1994) prioritise double masking over treatment effect. What does this mean? Is he trying to say that you are less likely to see a treatment effect in masked studies?

Anyway more to come in my next post .....