Crossbow Visual Stress Solutions

Hype springs eternal when it comes to the claims made for visual stress treatments. Exploratory studies are misrepresented as hypothesis testing and embellished with p-values that mean almost nothing. In turn, these over-hyped studies are used for marketing. However, even by the standards of visual stress promotions, the claims below made on the Crossbow Visual Stress Solutions website are pretty remarkable and are likely to contravene the Advertising Standards Authority (ASA) guidelines.

Questionable claims to increase productivity and reduce lost work days
through the treatment of visual stress

So on with the questionable stuff. In the figure above, it is claimed that-
1) Reduced headaches and migraines (through the use of overlays). In general evidence for this is weak. There is one underpowered study using Precision Tinted Lenses but to the best of my knowledge none using Crossbow overlays.
2) Fewer lost work days. I have done a few literature searches and can find nothing to back up this claim. It is of note that no source is provided on the website.
3) Increased productivity. Now that really is a claim and if it was true you would have industry beating on your door. Again, unfortunately,  I can find nothing to back this up.
4) 1in 5 people affected. There are few epidemiological studies of visual stress and most of those are underpowered and have methodological problems. The largest study by a group with no financial interest in visual stress treatment products suggested a figure of about 5.5% of participants with dyslexia and 8.5% of normal readers (difference not statistically significant). See blog post of December 18

I have made a fresh complaint to the Advertising Standards Authority so let's see what happens. In some respects, I think Crossbow provide a socially useful function. They respond to a demand and provide inexpensive overlays that are unlikely to be better or worse than more expensive products prescribed by 'visual stress specialists' - that is to say equally ineffective.  If it keeps people away from Irlen practitioners and Colorimetry enthusiasts that is a good thing. Nonetheless, Crossbow Visual Stress Solutions should only make claims that are proportionate to the available evidence. The claims made on their website go way beyond that.

Postscript 5/2/19

I received the following reply from the ASA

Dear Mr Griffiths,

Thank you for contacting the ASA with your complaint about Crossbow Education.

We’ve assessed your complaint, and consider the ad may have breached our Codes. We have taken steps to address this.

We have explained your concerns to the advertiser and provided guidance to them on the areas that require attention, together with advice on how to ensure that their advertising complies with the Codes in future.

Thank you once again for taking the time to raise your concerns with us. Comments such as yours help us to understand the issues that matter to consumers and we will keep a record of your complaint on file for use in future monitoring. If you would like more information about our complaint handling principles, please visit our website at  https://www.asa.org.uk/Consumers/How-to-complain.aspx.

Yours sincerely,

Adam Elmahdi

Complaints Executive

Direct line 020 7492 2165

The misleading information has now been removed from the visual stress solution website. A small success for visual-stress-sceptic.

Finding reason between the extremes - Optometry Today - May 2018

This curious article that reports an interview with Bruce Evans and Arnold Wilkins appeared in Optometry Today in May 2018. Although the piece contains arguments that are not really scientifically based, it is a step forward because it acknowledges that a multicentred randomised controlled trial is required. Not so long ago one of the interviewees was claiming that the Intuitive and Irlen systems were 'validated by RCTs'. Now it seems that is not the case. It is also a small step forward compared to a previous article in Optometry Today because the interviewees refrain from describing opponents who do not share their views as 'deviants' or 'deniers' (with the awful connotations that word has). See my blog post of June 2017.

I will focus on Arnold Wilkins' (AW) response to one question

Q Visual stress and the intuitive colorimeter have attracted some controversy. Why is that?

According to AW, 'the recent controversy seems to have been orchestrated by one two individuals'

No justification is given for this assertion. However, if look at where recent papers and reports, critical of this treatment, have come from - the statement is manifestly wrong.

1) Coloured overlays and precision‐tinted lenses: poor repeatability in a sample of adults and children diagnosed with visual stress. Paper from City University London published in Ophthalmic and physiological optics showing that people with the putative disorder visual stress are unlikely to find the same colour beneficial on different occasions.  You can access the paper here and it is reviewed in my blog post of August 2017.

2) The effect of coloured overlays and lenses on reading: a systematic review of the literature.
Review commisioned by the editor of Ophthalmic and Physiological Optics which is the house journal of the College of Optometrists. The review commented on the methodological problems with most of the trials and did not support the use of Intuitive Overlays and Precision tinted lenses. 

3) Effectiveness of Treatment Approaches for Children and Adolescents with Reading Disabilities: A Meta-Analysis of Randomized Controlled Trials  Review originating from the University of Munich and the University of Cologne in Germany, published in the Journal Plos One. As is the case with pretty much every independent review it concluded that the evidence did not support the use of coloured lenses and noted that studies with a placebo control condition tended not to find meaningful effects. 

4) Diagnosis and  Treatment of Reading and/or Spelling disorders in Children and Adolescents - a clinical practice guideline published in Germany that does recommend the use of (Irlen) coloured lens to ameliorate reading difficulties. Accessible here.
5) The use of Coloured Filters and Lenses in the Management of Children with Reading Difficulties. A report prepared by Christine Malins for the New Zealand Ministry of Health. Reviews many of the Intuitive overlay and Precision Tinted lenses and concludes that 'coloured overlays and filters are not an empirically supported treatment'
6) Coloured Filters and Dyslexia: A quick Gliding over myth and (possible reality).  review from an Italian neuro-ophthalmologist that takes a sceptical but open-minded view of the subject.
7) Learning Disabilities Dyslexia and Vision A statement from the  American Academy of Pediatrics, Council on Children with Disabilities, American Academy of Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus and the American Association of Certified Orthoptists.
States that Science does not support the use of tinted filters or lenses to support long-term educational performance.
8) The effectiveness and cost-effectiveness of coloured filters for reading disability: A systematic review. This study came from a respected group at Birmingham with a record of conducting systematic reviews. Like all the reviews listed above, it commented on the methodological shortcoming of the studies and concluded that the evidence was not strong enough to justify the use of coloured lenses and overlays for the remediation of reading difficulties.
9) No scientific evidence that Irlen Syndrome exists, say ophthalmologists. Statement from the Royal Australian and New Zealand College of Ophthalmologists
10) Report prepared by the Royal College of Ophthalmologists in the UK concluded that 'manipulation of the visual system using colour to facilitate reading lacks scientific support'
11)Neuroscience in Education: The Good Bad and the Ugly. Sergio Della Sella and Mike Andeson.
A Scholarly book by Edinburgh based neuroscientist that that takes a critical look at neuroscientific interventions in education and in particular the use of coloured lenses and overlays.

I could go on. My point is this; scepticism about the use of coloured lens and overlays (whether AW agrees with those criticisms or not) is not confined to or orchestrated by 'one or two individuals'.
It ranges across countries and continents; for example, from the UK, Germany, Italy, France, Switzerland, USA, Australia and New Zealand. Furthermore, it ranges across academic disciplines including optometrists, psychologists, neuroscientists, ophthalmologists, statisticians, health care economists and basic scientists. How does AW believe this level of scepticism has been orchestrated by one or two individuals? Optometry Today should explain.

On the other hand.........

If you look at the literature supporting the use of Intuitive Overlays and Precision Tinted Lenses and the organisations that promote their use you could make a much more convincing case for a campaign orchestrated by two individuals.
The research literature is dominated by two or three workers in a way I have not seen in any other area of research.  If you look at the pseudo-systematic review published in the Journal of Optometry by Bruce Evans and Peter Allen (reviewed Jan 2018) 8 of the 10 studies of Intuitive overlays were authored by at least one out of Arnold Wilkins, Bruce Evans and Peter Allen. Similarly, in the section on the Intuitive Colorimeter 2 of the 3 papers was authored by Arnold Wilkins. The failure of this research to generalise from a small clique of researchers over the last 25 years tells its own story.
Again, if you look at some of the bodies that promote the use of Intuitive Overlays and Lenses the same names keep recurring.
The Institute Of Optometry The IOO is a self-funding charity that relies to an unknown extent on sales of Visual Stress paraphernalia through its trading arm i.o.o sales. Bruce Evans is director of research at the IOO.
Society for Coloured Lens Prescribers.  Among the claims of this body is that it promotes an evidence-based approach to the use of coloured lenses and overlays. For this reason, I believe most of its members are in breach of the code of conduct. The committee includes Arnold Wilkins and Bruce Evans.
Arnold Wilkins' Web page at the University of Essex. AW answers his critics, at least to his own satisfaction and provides a partial list of references. Through the University of Essex, Intuitive Overlays and Precision Tinted lenses receive the kind of endorsement that money can not buy through advertising.

Reason between the extremes?

The article implies that Arnold Wilkins and Bruce Evans are in some way in the centre ground of a scientific debate. First, this is not a rational reason for believing an opinion to be well founded. The middle ground between 2+2=4 and 2+2=6 is not 2+2=5. Second, it is not even true. When your opinions are out of step with almost every independent body that has reviewed the evidence for the use of coloured lenses and overlays to facilitate the acquisition of reading in people with the putative disorder visual stress, you probably do not occupy the middle ground. That does not in itself mean that you or right or wrong but if you are going to use that line of argument at least make sure it is true.

Another problem for the visual stress/reading hypothesis

A high quality, adequately powered study from a respected research group with no financial ties to visual stress treatment shows no link between the putative disorder visual stress and dyslexia

Visual stress is not the same thing as dyslexia – we all know that. 

Dyslexia simply means an impaired ability to read and the prevalence depends on where you draw the line; the bottom 5% or 8% say.  It is wrong to think in terms of ‘garden variety’ poor readers and poor readers with dyslexia.

The best evidence is that dyslexia is a language-based disorder that is based on the ability to link certain sequences of letters to sounds. Without this ability, it is more difficult to decode text and link it to the spoken word. However, that does not preclude the possibility that visual deficits can play some role in dyslexia. That said, the best evidence to date, from population-based studies, is that visual factors do not play a large role(1).

You might expect visual stress to be overrepresented in the population with dyslexia.  Take the example of someone with borderline phonetic skills who also has visual stress. The additional handicap of visual problems while reading would be more like to tip them into the bottom 5% of readers. Furthermore, dyslexics with visual stress would be more resistant to existing teaching methods and would less likely to leave the category of impaired readers as a result of intense remedial tuition.

The claim that VS is over-represented among impaired readers has been made by VS enthusiasts such as Bruce Evans who has argued that VS is found 2-3 times more frequently in the population with dyslexia compared to normal readers(2). This seriously flawed piece of research is reviewed in my blog post of May 2015.  Other research (admittedly underpowered) using Irlen methodology found no link between VS and reading ability(3). Also reviewed May 2015.

New Research

Recently an adequately powered study has been published in the journal Developmental Psychology(4). You can access the paper here.

The study comes from an internationally recognised research group based in France and Switzerland who have no financial ties to any visual stress diagnostic or treatment products.

The authors looked at 164 participants with dyslexia and 118 controls with normal reading. The authors did not rely on a self-reported diagnosis of dyslexia. Instead, all participants were tested with a bank of tests to diagnose dyslexia and also a bank of tests to exclude other co-morbidities including uncorrected sight problems, neurological disease and deafness.

Both groups were investigated for 3 possible causes of dyslexia- a phonological deficit, reduced visual attention span and visual stress.

The diagnostic criterion for visual stress was a response to gratings of a spatial frequency of 3 cycles per degree (CPD) and a control grating outside the range said to be aversive in VS (0.5 CPD) to check for response bias. The authors criticise tests based on increased reading fluency with overlays and argue that it does not prove that participants had visual stress or that their reading deficit was caused by visual stress, to begin with. Such tests also exclude subjects with visual stress who do not respond to overlays. The authors go on to make an analogy with attention deficit hyperactivity disorder (ADHD) arguing that response to methylphenidate would be an irrational diagnostic test for that disorder. 

The results of this study came as no surprise to me. Most dyslexic children showed phonological deficits  (92.5%) – in terms of response accuracy, speed or both. There was a small difference in visual attention span that affected 28.1% of dyslexic children (all of whom also had phonological deficits). There was no association between visual stress and dyslexia - 5.5% of dyslexics and 8.5% of controls had visual stress.  The sample size of this study was big enough to look at the subgroup of patients in whom a phonological deficit could not be measured - the group in whom you might be most likely to detect a visual deficit - again no increased prevalence of VS could be found.

Conclusions. 

An odds ratio above 1 would favour VS being more common among the population
with dyslexia: below 1 in normal readers. Bars represent 95% confidence
intervals.

This study is a major problem for the visual stress/dyslexia hypothesis and needs to be put alongside previously published studies. Taken individually or together they do not make a convincing case for VS being more common in reading impaired individuals compared to normal readers. It is striking that the two studies at the lowest risk of bias, in the statistical sense of the word, offer the least support for the VS/dyslexia hypothesis.

How will enthusiasts and those with a financial interest in VS respond to these findings I wonder?

Probably by ignoring it or misrepresenting the findings altogether. That said there are signs that the claims being made are being toned down a bit. For example, by only suggesting that ‘visual-stress may co-occur with dyslexia’ a vacuous and meaningless statement if ever there was one.

1.         Creavin AL, Lingam R, Steer C, Williams C. Ophthalmic Abnormalities and Reading Impairment. PEDIATRICS. 2015 Jun 1;135(6):1057–65. 

2.         Kriss I, Evans BJW. The relationship between dyslexia and Meares-Irlen Syndrome. J Res Read. 2005 Aug;28(3):350–64. 

3.         Kruk R, Sumbler K, Willows D. Visual processing characteristics of children with Meares-Irlen syndrome: Visual processing in Meares-Irlen syndrome. Ophthalmic Physiol Opt. 2008 Jan 14;28(1):35–46. 

4.         Saksida A, Iannuzzi S, Bogliotti C, Chaix Y, Démonet J-F, Bricout L, et al. Phonological skills, visual attention span, and visual stress in developmental dyslexia. Dev Psychol. 2016 Oct;52(10):1503–16.