Coloured lenses and visual stress - a misleading 'web resource'

You can follow this link to find the above website which purports to provide a resource for parents and professionals 'in order to increase awareness of visual stress and its treatment'. It was even announced in a paper published in the British Journal of Occupational Therapy: Supporting Children with visual stress: the development of a web resource for parents and professionals. The authors claim to have produced a 'centralised information resource'. This all sounds very balanced and objective - but it isn't. If you want accurate and impartial information you should avoid Coloured Lenses and Visual Stress.

Section one: What is visual stress?
This section opens with the extraordinary claim that 'visual stress is thought to be the most common visual reason for reading difficulties' The only evidence cited for this is a letter from Bruce Evans.
The evidence that visual problems (outside simple refractive error) are an important cause of reading difficulties is very weak indeed. For example, a recent population-based study from Bristol looking at 5822 children aged 7-9 which compared reading impaired individuals with their peers found no significant differences between the two groups. (link)
There is no credible evidence that visual stress is an important cause of reading difficulties. The case control study at lowest risk of bias, with subjects drawn from a school setting and screeners blinded to their reading status, did not find any relationship between 'visual stress' and reading difficulties. See the review of Kruk 2008.
The general tone of this section of 'coloured lenses and visuals stress' is wrong. Despite 20+ years of research into visual stress neither the International Classification of Disease published by the World Health Organisation nor the Diagnostic and Statistical Manual of Mental Disorders lists visual stress or Meares-Irlen syndrome as a recognised disorder. Despite this, the authors transmute a hypothetical disorder with very questionable theoretical foundations and no agreed diagnostic criteria into an accepted diagnostic entity.

Do Coloured Lenses Work? - Research Papers
This section provides an incomplete list of research papers.
The papers are divided into to those that are said to support the use of colour to treat visual stress and those that do not. On the surface that might appear to be even handed.
Unfortunately, there is no critical commentary provided. The only exception was the study by Ritchie and colleagues published in 2011.  This was a 'negative' study for which the authors managed to find a critical commentary from that portal of impartial information Irlen-Alberta! The criticism was that the data of three subjects who were unmasked were analysed separately. That is, the data was not strictly evaluated on an intention to treat basis. This is a grey area. However, much worse faults in papers supporting the use of coloured lenses and overlays are overlooked. For example, the link to the first RCT to assess the effect of coloured lenses on reading makes no mention of the fact that data was analysed with data from 32 of 68 subjects missing - a much worse sin (see Holy Trinity One for a critical commentary of this paper).
Another problem is that some of the references said to support the use of coloured lenses do not. For example, the paper by Mitchell published in 2008 was a parallel groups study that looked at poor readers with visual perceptual difficulties. One group was treated with their chosen colour determined with the intuitive colorimeter and the other with a placebo filter. Both groups improved but there was no difference in reading speed, accuracy or comprehension at the end of the study period. While improvements were noted in comparison with the no treatment control group that is evidence for placebo effects not the use of colour.
The authors also fail to reference a number of important reviews.
First, the most rigorous systematic review to date from the West Midlands Health Technology Assessment Centre which critically evaluates the papers. The Effectiveness and Cost Effectiveness of Coloured Filters for Reading Difficulty: A Systematic Review.
Second, a quasi-systematic review written by Christine Malins an educational psychologist in New Zealand-The Use of Coloured Filters and Lenses in Children with Reading Difficulties - which again takes a critical view of the evidence.
Third a joint statement from the American Academy of Pediatrics, American Academy of Ophthalmology, the American Association for Pediatric Ophthalmology and Strabismus and the American Association of Certified Orthoptists. Learning Difficulties Dyslexia and Vision- which again takes a critical view of the evidence for coloured lenses and overlays to treat 'visual stress.

The Views of Experts
The opinion of a few hand picked 'experts' hardly constitutes evidence for the use of colour. It is no substitute for a critical appraisal of the evidence.
Even so, how were the experts selected? It would be just as easy or easier to produce a panel that might have reached the opposite conclusion. The panel could have included Professor Sergio Della Sala, Professor of Human Cognitive Neuroscience at Edinburgh or Margaret Snowling formerly Professor of psychology at York and currently the President of St Johns College Oxford or Professor Dorothy Bishop Professor of Experimental Neuropsychology at Oxford. While I can not claim to know their exact views I doubt if you have received any kind of endorsement for the concept of visual stress and use of colour to ameliorate reading difficulties.

Conclusions
This website does a disservice to professionals and the public alike by failing to give an objective account of the evidence for the treatment of visual stress with coloured filters. In truth, the authors also damage their own cause. The systematic misrepresentation of small scale studies at high risk of bias has brought this subject area into disrepute in the wider scientific community.
Returning to the paper in the British Journal of Occupational Therapy the authors state' Reflecting, the research team felt satisfied they had maintained a neutral perspective...' This claim is unjustified. The authors end their paper with some high minded rhetoric that the research was based on a 'democratic impulse'. I am not sure what that means but I am sure there is nothing democratic about providing a partial list of papers mostly hidden behind a paywall and the opinions of a few carefully selected experts.

Placebo effect 2: novelty effects

Some years ago after my mother died I had the bittersweet job of sorting through the possessions left in the family home. One especially poignant moment came when I found a box that contained some school exercise books of mine from when I must have been six or seven. It brought a lump to my throat that she had kept them for all these years. When I started to leaf through the pages I noticed something interesting, the first page of two of each book contained relatively neat and tidy handwriting but by page three or four it reverted to the 'deranged spider on acid' style where it has remained ever since. A good example of the novelty effect I would say.
The idea that an intervention that is new and exciting can improve motivation in the short term is not new(1) and it is believed to play an important part in testing interventions in an educational setting.
Novelty effects can also be seen as part of the Hawthorne effect.  Almost all interventions introduced at the Hawthorne works between 1922 and 1934 to improve productivity worked for while only for productivity to slump back to its previous level. Perhaps the most interesting of these interventions was changes to the lighting of the factory. The improvement in productivity has been attributed to both novelty effects and the fact that performance was being measured. In fact, the Hawthorne effect may not have been observed at the Hawthorne works because most changes were made on a Monday when productivity was, in any case, greater after the weekend break. However, there is evidence from trials that it can play an important role in determining outcomes(2).
This brings me back to my exercise books. A new book with a shiny cover and fresh clean paper improved motivation and for a while at least I was able to improve my writing. Proponents of coloured overlays to treat visual stress have presented handwriting samples as evidence that their treatment works. The example below comes from Bob Hext's webinar 'Visual Stress What Everyone Should Know' prepared for the BDA. It shows changes in handwriting produced by switching to a yellow tint. Although the small improvements may be attributable to the colour used, a more plausible explanation is that they were due to the novelty effect.

Evidence of novelty effects in visual stress treatment trials

There is plenty of evidence of novelty effects in treatment trials of visual stress in dyslexia.
For example, take the key trial by Arnold Wilkins and colleagues published in 1994- A double masked placebo controlled trial of precision spectral filters in children who use overlays (3) (follow the link to see my previous post on this paper). Subjects were randomised to receive their chosen tint first or a placebo tint first. The trial was hampered by such a high rate of drop out that the results are almost meaningless. However, by far the most significant result was that 31 subjects preferred the first lens they tried and 17 the second and four did not know. Assuming equipoise was maintained and equal numbers used the chosen lens and placebo lens first this is very suggestive of novelty effects. The novelty of tinted lenses had worn off by the time they came to try the second set of lenses. This is likely to be an important confounder in the interpretation of the symptom scores. If more subjects who used the chosen lens first then it would not be surprising if results seemed to be slightly better.

Another study this time of ChromaGen contact lenses also provides intriguing evidence 'Application of ChromaGen haploscopic lenses to patients with dyslexia: a double-masked, placebo-controlled trial' (4)in fact this trial probably wasn't double masked but that's for another post. What intrigued me was that subjects who received experimental lenses before placebo showed a bigger improvement in reading rate than those who received them after the placebo lenses. If the result was purely due to tint who not expect the order of testing to make any difference.

Another 'field study' of Irlen overlays showed that in those children who used overlays for 6 months improvements in reading were only noted for the first 3 month period(5). Although the authors attributed this 'plateauing off' as subjects reaching the grade level a more plausible explanation is the novelty effect. The intervention was no longer new and interesting and it lost its motivational effect.

The high rates of attrition observed in many trials suggests novelty effects. Once the newness and excitement of the intervention had worn off the use of overlays and lenses declines sharply. For example the study published by Jeanes et al showed that although 54% of secondary school children initially chose an overlay by 3 months only 35% of the sample were still using and overlay and wished to continue and by ten months this figure had fallen to 5.1%(6).

Conclusion 
Novelty effects are an important confounder in trials of educational interventions. The proponents of colour to treat visual stress in poor readers have not given sufficient thought to this problem.

1) Love A, Kavazis AN, Morse A, Mayer Jr. KC. Soccer-Specific Stadiums and Attendance in Major League Soccer: Investigating the Novelty Effect. J Appl Sport Manag Res Matters Vol 5 No 2 2013 [Internet]. 2013; Available from: http://js.sagamorepub.com/jasm/article/view/3294  

2) McCarney R, Warner J, Iliffe S, van Haselen R, Griffin M, Fisher P. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol. 2007;7(1):30.

3) Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt. 1994 Oct;14(4):365–70.  

4) Harris D, MacRow-Hill SJ. Application of ChromaGen haploscopic lenses to patients with dyslexia: a double-masked, placebo-controlled trial. J Am Optom Assoc. 1999 Oct;70(10):629–40.  

5) Noble J, Orton M, Irlen S, Robinson G. A controlled field study of the use of coloured overlays on reading achievement. Aust J Learn Disabil. 2004 Jun;9(2):14–22.  

6) Jeanes R, Busby A, Martin J, Lewis E, Stevenson N, Pointon D, et al. Prolonged use of coloured overlays for classroom reading. Br J Psychol. 1997 Nov;88(4):541–8.  

Placebo effects

To suggest that the reported beneficial effects of coloured lenses and overlays could be due to the placebo effect or more properly placebo effects causes offence. Replies of the ‘I have worn overlays for x years and I could not possibly be subject to the placebo effect’ sort usually come thick and fast. Most people acknowledge the existence of the placebo effect but believe themselves to be too sophisticated to be affected themselves. The truth is that we are all vulnerable and it’s no bad thing.

I have experienced the placebo effect myself when I obtained pain relief from a morphine infusion that wasn’t even working. At first, I felt humiliated but on reflection, it was great to get pain relief without the small but significant risks of intravenous morphine. It is not only medical interventions that provoke placebo effects.  I like playing tennis socially and badly. From time to time it's good to try something new to boost my position on the tennis ladder; a new racket perhaps with a novel stringing pattern. Recently I tried some fancy new strings -‘biphase multifilament’ which also happened to be red. I did play a little bit better for a while although more so in my own estimation than my opponents. However, in spite of my subjective opinion, my position near the bottom of the tennis ladder didn’t change and what improvements I have made have come about through practice and one to one coaching.

What I am trying to say is this. When I discuss the importance of placebo effects I do not do so from a position of superiority I am just as vulnerable  (or blessed) as the next person in this regard. However, in appraising clinical trials of coloured overlays you can’t get away from inadequate attempts to control for research participation effects.

The components of the placebo effect, which need to be considered in relation to trials of coloured overlays and lenses are
1) Novelty effects, that is, the use of an intervention that is new and exciting may enhance motivation and performance.
2) The effect of being observed and tested, which is sometimes called the Hawthorne effect.
3) The therapeutic ritual of the intervention itself.
4) The relationship with the carer.

The usual way around this problem is to have a placebo control group and ensure that both research participants and experimenters are blinded to which intervention the subject is receiving - a so called double blind trial. I acknowledge that this is not easy in studies of coloured overlays and lenses but it is not impossible.

In Arnold Wilkins’1994 study (1) this was achieved by assessing subjects with the intuitive colorimeter so that they did not see the actual lenses they are going to receive and by leaving an interval between the assessment and being given the experimental and control lenses. By this means 23 subjects did not know which lens matched the colorimeter setting 10 thought the experimental lens matched and 10 though the placebo lens matched. Unfortunately, this trial was severely compromised by attrition of participants which is a common problem in studies of coloured overlays and lenses. The study failed to show any difference in reading rate, accuracy or comprehension between optimum filter and control filter for the 45 of 68 subjects for whom data were available. This study also provides some evidence for novelty effects. 31 subjects preferred the first lens they were tested with 17 the second and 4 expressed no preference. This is by far the most statistically significant result of the whole study!

Some studies made no attempt to mask subjects or experimenters to the intervention being used but claimed that they were able to control for this by enhancing the placebo effect of the control intervention. For example, in the trial reported by Bouldoukian and colleagues (2) the placebo overlay consisted of a pale yellow filter which would have been distinct from the more saturated colours of Intuitive Overlays^TM.  The placebo filter was described to the subjects as ‘a new filter from the United States where it was thought to be a wonderful discovery’. It was labelled ‘Research Model A16 Anti UV IR Filter. Made in the USA’. The authors argued that, as a result of the enhancement of the placebo overlay, the modest increase in reading speed on the Wilkins Rate of Reading Test (WRRT) that was observed (4% on average) was unlikely to be accounted for by placebo effects. The unstated assumption, that it is possible to measure the placebo effect of the experimental intervention and balance it, by enhancing the placebo effect of the control intervention, is scientifically unfounded. The act of selecting the optimum tint from a set of overlays is likely to involve a more prolonged relationship with the experimenter and a richer therapeutic ritual, both of which are powerful drivers of the placebo effect (3).

Overall there is strong evidence in the literature that placebo effects play a big role in the reported benefits of coloured lenses and overlays to improve reading in subjects with visual stress.

Those trials that do have a proper placebo control report no improvements in reading. For example Wilkins 1994, Robinson Foreman 1999 (4) (for the three months it was a placebo controlled study), Mitchell 2008 (5)  and Ritchie 2012 (6). Those trials with enhanced placebo report small effects and those trials with no placebo control report significant effects. This gradient points strongly to a placebo effect.

Sometimes this gradient can even be seen within trials. The Robinson Foreman study of 1999 (4) and Mitchell 2008(5)  contain a placebo control and an untreated control group. There was a significant difference between the experimental lens group and the no treatment control group but again no significant difference between experimental lens and placebo lens groups.

Conclusion 
There are no studies that demonstrate improvements in reading with overlays that can not be more plausibly explained by placebo effects.

1.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70. 

2.         Bouldoukian J, Wilkins AJ, Evans BJW. Randomised controlled trial of the effect of coloured overlays on the rate of reading of people with specific learning difficulties. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 2002 Jan;22(1):55–60. 

3.         Kaptchuk TJ, Kelley JM, Conboy LA, Davis RB, Kerr CE, Jacobson EE, et al. Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome. BMJ. 2008 May 3;336(7651):999–1003. 

4.         Robinson GL, Foreman PJ. Scotopic sensitivity/Irlen syndrome and the use of coloured filters: a long-term placebo controlled and masked study of reading achievement and perception of ability. Percept Mot Skills. 1999 Aug;89(1):83–113. 

5.         Mitchell C, Mansfield D, Rautenbach S. Coloured filters and reading accuracy, comprehension and rate: a placebo-controlled study. Percept Mot Skills. 2008 Apr;106(2):517–32. 

6.         Ritchie SJ, Della Sala S, McIntosh RD. Irlen colored overlays do not alleviate reading difficulties. Pediatrics. 2011 Oct;128(4):e932–8.

'Prolonged use of coloured overlays for classroom reading'

Jeanes R, Busby A, Martin J, Lewis E, Stevenson N, Pointon D, et al. Prolonged use of coloured overlays for classroom reading. Br J Psychol. 1997 Nov;88(4):541–8. 

You can download this paper from Arnold Wilkins webpage here

The paper starts with some background information about Intuitive Overlays vs Irlen Overlays and the authors make their point by plotting the chromaticity coordinates of Irlen overlays and Intuitive overlays. However, they exaggerate the difference between the two systems by plotting both the x and y coordinates on different scales. See the figure below. In my opinion, this is poor practice in scientific reporting.

The authors exaggerate the difference between Irlen and Intuitive overlays by plotting the chromaticity coordinates on different scales and with different symbols. This is poor practice in representing data and it is noteworthy that one of the authors has a financial interest in Intuitive Overlays; the system that benefits from this distorted representation.

Prevalence of 'visual stress'
In the first part of the paper, the authors estimate the prevalence of visual stress in 93 children attending a primary school and 59 children attending the first year of secondary school. When asked to look at random letters (not naturalistic text) through overlays 51% of the former group and 54% of the latter chose an overlay to ameliorate perceptual distortions. When the group was reassessed at two months 35% of the primary school group said they were still wearing lenses and wished to continue (note this is based on self-report without independent verification). Similarly, 36% or secondary school children were still using and wished to continue with overlays. By ten months this had dropped even further so that 14% of the primary school group were still using overlays or glasses and only 5% of the secondary school group. By the criterion of voluntary sustained use, the prevalence of visual stress in a mixed ability group of children should be between 5-14%. What is remarkable about this and other papers looking at coloured overlays and lenses to treat visual stress is the astonishing rate of attrition over time.

Clinical trials
The paper also contains two trials of sorts. Both are at high risk of bias and use a non-standardised reading test of unknown relevance to real world reading. The Wilkins Rates of Reading Test (WRRT).
Those who continued using overlays for ten months were tested with and without overlays. There was no placebo group and the study was not masked. The authors found a small difference in words per minute on the WRRT.
However, another part of the paper describes a small study which did have a placebo control overlays of a complementary colour. This time there was no statistically significant difference between the two groups.

Conclusion
Overall some questionable reporting and what appears to be a posthoc data trawl. The data does not provide compelling evidence for the treatment of visual stress with colour.

Some problems with the Wilkins Rate of Reading Test

The figure above shows a visual fields test that displays the sensitivity of the visual field at locations around the fixation point. The black areas represent reduced sensitivity. You might think that there has been some improvement in vision as a result of some unspecified intervention. However, if you tried to publish a figure such as this,  a good peer reviewer would spot the problem straight away.

Before you can say whether the result of a psychophysical test has improved you need to know if there is been some change in the way the observer is performing the test. Are they performing the test in a more or less conservative manner for example? For this reason, machines for testing visual fields build in catch trials to determine if the subject is becoming a more or less conservative observer. For example, they make noise as if a stimulus has been presented when it has not, or they may measure the stimulus response time - anything less than 200ms is almost certainly a false positive. Going back the case above, the tests showed that the observer had become a less conservative observer or more trigger happy if you like and this was the most likely explanation for the improvement int he visual field see below.

So, what has this got to do with the Wilkins Rate of Reading Test (WRRT)? It is important to

remember that the WRRT is not a standardised reading test. It not used by anyone except a
aficionados of visual stress and its treatment. The WRRT does not consist of naturalistic text but closely spaced, commonly used words, ordered randomly without syntax, punctuation or paragraphs. Its proponents argue that it is well suited to detecting visual problems. That may be true, but it also has some key flaws. The principle one being that it can not measure 'response criterion' That is, has the observer become more or less conservative in his responses? It can not be said to isolate out visual factors if you can not measure other aspects of reading. For example, you might find that subjects were able to read faster but at the expense of comprehension, parsing or miscues.
I know this from my own experience of learning Spanish through skype. When asked to read text I can easily vary my rate of reading making it sound fast and 'spanishy'. However, my teacher notices the parsing goes a bit funny and she throws in a comprehension question. The rate of reading then comes crashing down. Although she wouldn't call it that, she has recognized that my response criterion has changed and I have become a less conservative reader - taking risks with pronunciation, sentence structure, and meaning.
So, returning to the WRRT unless you can show that other aspects of reading are conserved you can not claim to have isolated out visual aspects of reading. Also, you can not claim that any improvements in the WRRT have any relevance to real world reading.
Still, you have to admire somebody who has found a way to make money out of printing random words.

Intuitive Colorimeter: Technique for the 21st century?

The intuitive colorimeter is a device that is said by its proponents to provide the best means of prescribing colour to alleviate visual stress. Kriss and Evans for example state that ‘people almost invariably report more benefit from precision tinted lens than from coloured overlays because precision tinted lenses are easier to use (eg with white boards and when writing) and because colour can be prescribed with more precision’(1)

In a lecture hand-out, Bruce Evans has argued that the Intuitive Colorimeter Mark III is the 21st-century method for assessing visual stress. So I thought it would be worth assessing the evidence for this device.

Random letters (why?) illuminated in the Intuitive Colorimeter

The intuitive colorimeter is, in essence, an illuminated box with an aperture for viewing text. The hue and saturation of the light in the box can be changed. The light source is a halophosphate fluorescent tube. Rotating the dial alters the filter condition to change the illumination of the text which consists of random letters (why?) The colorimeter setting can then be assessed against a set of trial lenses which are then be prescribed in the form of glasses - so called precision tinted lenses.

So what is the evidence for lenses prescribed with the intuitive colorimeter?
I have been able to identify a hand-full of trials(2)(3)(4) (5)(6) which address this issue. Taken together or individually they do not amount to a compelling case for the use of the intuitive colorimeter.

The trial by Wilkins and Colleagues published in 1994 in the Ophthalmic and Physiological Optics in 1994 is often cited in support of precision tinted lenses even though it is a negative or at best inconclusive study. 68 poor readers with visual stress, diagnosed according to the criterion of voluntary sustained use of overlays, were enrolled. All subjects were tested with the intuitive colorimeter and their preferred tint identified. They were also prescribed a closely related tint which did not ameliorate their perceptual distortions.

The study had a crossover design and subjects were randomised to use experimental tint or placebo tint first.
Unfortunately, the dropout rate from the study was so high that it is not possible to draw meaningful conclusions. For the 45/68 for whom reading data was available, there was no difference in reading speed accuracy or comprehension comparing preferred lens with placebo lens. Interpretation of the symptom diaries is even more problematic because data was only available for 36/68 subjects. Using experimental lens 71% of days were symptom-free and with the placebo lens 66%. An unimpressive set of results when you remember that nearly 50% of the data is missing 

Perhaps the most striking result is that 31 preferred the first lens and 17 the second; a difference that is highly significant and suggestive of novelty or Hawthorne-effects.

The trial by Mitchell and colleagues published in 2008 was a parallel groups study with a control group of 17 who received no lens, a placebo lens group of 15 and an experimental lens group of 17 who received their chosen tint as determined with the Intuitive Colorimeter.

The subjects were individuals with reading difficulties who complained of visual distortions so the diagnosis of visual stress was purely symptom based.

The outcome measures were symptom scores from the Irlen Differential Perceptual Scale and the Neale analysis of reading Test. There was no difference between the experimental lens or placebo lens groups for any of the measures although there were differences with the no treatment control group which is suggestive of a strong placebo effect.

Singleton and Trotter 2005 Like many studies in this area, this study starts with a small group which is divided into even smaller groups. As a result, the statistical power of the study is limited.

There was a total of ten students with dyslexia of whom 5 had visual stress and there were ten normal readers of whom 5 had visual stress diagnosed according to the Visual Processing Problems Inventory VPPI. There was no placebo control group. The study was a crossover design that compared lenses prescribed with the intuitive colorimeter against no lens. The outcome measure was the Wilkins Rate of Reading Test.

The only group to show some improvement were the five patients with dyslexia and visual stress  However there are a number of serious problems with this study which include. Small sample size and no placebo control group.

The external validity of this study is poor because subjects were recruited via the disabilities service of the University and included subjects who may have been aware of the potential benefits of colour. Finally, because the WRRT is a non-standardised reading test and it is not clear if the results can be generalised to naturalistic text.

Machlachlan  Yale and Wilkins 1993 wasn’t a controlled trial at all. Fifty-five subjects were recruited. Twenty-three of the cases were volunteers responding to an article in 'Living magazine' and the remainder were referrals from educationalists, psychiatrists and neurologists. As a result, external validity is likely to be low.

The criteria for diagnosing visual stress was awareness of visual distortions, sustained use of overlays and having chosen a colour with the Intuitive Colorimeter.

The outcome measure was what percentage continued wearing glasses over 10 months. 82% reported that they were still using the glasses 10 months later. 

Unfortunately, this based on self-report and is unreliable. The external validity of this study is questionable because of the way the subjects were recruited and finally.. so what?

Simmers et al 2001. This study comes from a reputable psychophysical laboratory at Imperial College. It has been reviewed in a previous post. Instead of using text it looks at the kind of geometric patterns that are said to be most aversive in visual stress. The authors found no difference between subjects with visual stress and controls. In those cases with visual stress no difference in aversive symptoms when using precision tinted lenses.

So -Intuitive ColorimeterTM method for the 21st century? If you believe that you will believe anything.

1.         Kriss I, Evans BJW. The relationship between dyslexia and Meares-Irlen Syndrome. J Res Read. 2005 Aug;28(3):350–64.

2.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70.

3.         Mitchell C, Mansfield D, Rautenbach S. Coloured filters and reading accuracy, comprehension and rate: a placebo-controlled study. Percept Mot Skills. 2008 Apr;106(2):517–32.

4.         Singleton C, Trotter S. Visual stress in adults with and without dyslexia. J Res Read. 2005 Aug;28(3):365–78.

5.         MacLachlan A., Yale S., Wilkins A. Open trial of subjective precision tinting: A follow-up of 55 patients. Ophthalmic Physiol Opt. 1993;13(2):175–8.

6.         Simmers AJ, Bex PJ, Smith FK, Wilkins AJ. Spatiotemporal visual function in tinted lens wearers. Invest Ophthalmol Vis Sci. 2001 Mar;42(3):879–84.

A big problem for the visual stress hypothesis

Spatiotemporal function in Tinted Lens Wearers
Anital Simmers, Peter Bex, Fiona Smith, and Arnold Wilkins
Invest  Ophthalmol Vis Sci 2001;42:879-884

For once, a rather good study which comes from a reputable group at Imperial College London. You can download it here.
Proponents of visual stress as a factor in reading difficulties argue that the sort of discomfort and movement illusions that most of us experience in response to recurring geometric patterns, varies in the population from person to person. They go on to argue that in some susceptible individuals it can be brought on by print, resulting in distortions and movements of letters and words - so called visual stress.
The conditions that induce these symptoms, for geometric patterns, are said to be surprisingly specific and peak with high contrast square gratings between 2-8 cycles per degree. Proponents argue that text has spatial frequencies within this range see the figure below.

The text has been filtered to remove high spatial frequencies and the contrast exaggerated to make the stripes more apparent

Text obviously isn't a striped grating. However, like a piece of music which contains multiple frequencies of sound waves at the same time (which you don't notice as the music washes over you), text contains multiple spatial frequencies and it is those between 2-8 cycles per degree that are the said to be a problem in 'visual stress'.
If this is the case we would expect to see this effect in simplified settings using actual gratings rather than text. Such a study has been done on subjects with visual stress, in a reputable psychophysical laboratory. Even more interesting is that Arnold Wilkins was one of the authors although he does not emphasise the results of this study a great deal.
The subjects were twenty individuals with 'visual stress' who had successfully worn tinted lenses (prescribed with the intuitive colorimeter) for at least six months. Twenty control subjects were recruited from the staff and siblings of staff and students of the University of Exeter.
Subjects were tested with a range of psychophysical tests which are designed to assess visual processing in the range of frequencies that are said to be aversive in visual stress.  The participants with visual stress, they were tested with and without their lenses.
The tests included spatiotemporal contrast sensitivity, contrast increment thresholds, random dot motion coherence, and motion perception
Results
To put it simply, there was no difference between control subjects and visual stress subjects and just as important, in the group with visual stress there was no difference with and without their lenses.

Contrast sensitivity function for subjects with visual stress with without

 lenses

Despite this, some people (mostly with a vested interest) still claim that lenses prescribed with the intuitive colorimeter are the 'gold standard.
Will this study dent their confidence? I doubt it.