Some problems with the Wilkins Rate of Reading Test

The figure above shows a visual fields test that displays the sensitivity of the visual field at locations around the fixation point. The black areas represent reduced sensitivity. You might think that there has been some improvement in vision as a result of some unspecified intervention. However, if you tried to publish a figure such as this,  a good peer reviewer would spot the problem straight away.

Before you can say whether the result of a psychophysical test has improved you need to know if there is been some change in the way the observer is performing the test. Are they performing the test in a more or less conservative manner for example? For this reason, machines for testing visual fields build in catch trials to determine if the subject is becoming a more or less conservative observer. For example, they make noise as if a stimulus has been presented when it has not, or they may measure the stimulus response time - anything less than 200ms is almost certainly a false positive. Going back the case above, the tests showed that the observer had become a less conservative observer or more trigger happy if you like and this was the most likely explanation for the improvement int he visual field see below.

So, what has this got to do with the Wilkins Rate of Reading Test (WRRT)? It is important to

remember that the WRRT is not a standardised reading test. It not used by anyone except a
aficionados of visual stress and its treatment. The WRRT does not consist of naturalistic text but closely spaced, commonly used words, ordered randomly without syntax, punctuation or paragraphs. Its proponents argue that it is well suited to detecting visual problems. That may be true, but it also has some key flaws. The principle one being that it can not measure 'response criterion' That is, has the observer become more or less conservative in his responses? It can not be said to isolate out visual factors if you can not measure other aspects of reading. For example, you might find that subjects were able to read faster but at the expense of comprehension, parsing or miscues.
I know this from my own experience of learning Spanish through skype. When asked to read text I can easily vary my rate of reading making it sound fast and 'spanishy'. However, my teacher notices the parsing goes a bit funny and she throws in a comprehension question. The rate of reading then comes crashing down. Although she wouldn't call it that, she has recognized that my response criterion has changed and I have become a less conservative reader - taking risks with pronunciation, sentence structure, and meaning.
So, returning to the WRRT unless you can show that other aspects of reading are conserved you can not claim to have isolated out visual aspects of reading. Also, you can not claim that any improvements in the WRRT have any relevance to real world reading.
Still, you have to admire somebody who has found a way to make money out of printing random words.

Intuitive Colorimeter: Technique for the 21st century?

The intuitive colorimeter is a device that is said by its proponents to provide the best means of prescribing colour to alleviate visual stress. Kriss and Evans for example state that ‘people almost invariably report more benefit from precision tinted lens than from coloured overlays because precision tinted lenses are easier to use (eg with white boards and when writing) and because colour can be prescribed with more precision’(1)

In a lecture hand-out, Bruce Evans has argued that the Intuitive Colorimeter Mark III is the 21st-century method for assessing visual stress. So I thought it would be worth assessing the evidence for this device.

Random letters (why?) illuminated in the Intuitive Colorimeter

The intuitive colorimeter is, in essence, an illuminated box with an aperture for viewing text. The hue and saturation of the light in the box can be changed. The light source is a halophosphate fluorescent tube. Rotating the dial alters the filter condition to change the illumination of the text which consists of random letters (why?) The colorimeter setting can then be assessed against a set of trial lenses which are then be prescribed in the form of glasses - so called precision tinted lenses.

So what is the evidence for lenses prescribed with the intuitive colorimeter?
I have been able to identify a hand-full of trials(2)(3)(4) (5)(6) which address this issue. Taken together or individually they do not amount to a compelling case for the use of the intuitive colorimeter.

The trial by Wilkins and Colleagues published in 1994 in the Ophthalmic and Physiological Optics in 1994 is often cited in support of precision tinted lenses even though it is a negative or at best inconclusive study. 68 poor readers with visual stress, diagnosed according to the criterion of voluntary sustained use of overlays, were enrolled. All subjects were tested with the intuitive colorimeter and their preferred tint identified. They were also prescribed a closely related tint which did not ameliorate their perceptual distortions.

The study had a crossover design and subjects were randomised to use experimental tint or placebo tint first.
Unfortunately, the dropout rate from the study was so high that it is not possible to draw meaningful conclusions. For the 45/68 for whom reading data was available, there was no difference in reading speed accuracy or comprehension comparing preferred lens with placebo lens. Interpretation of the symptom diaries is even more problematic because data was only available for 36/68 subjects. Using experimental lens 71% of days were symptom-free and with the placebo lens 66%. An unimpressive set of results when you remember that nearly 50% of the data is missing 

Perhaps the most striking result is that 31 preferred the first lens and 17 the second; a difference that is highly significant and suggestive of novelty or Hawthorne-effects.

The trial by Mitchell and colleagues published in 2008 was a parallel groups study with a control group of 17 who received no lens, a placebo lens group of 15 and an experimental lens group of 17 who received their chosen tint as determined with the Intuitive Colorimeter.

The subjects were individuals with reading difficulties who complained of visual distortions so the diagnosis of visual stress was purely symptom based.

The outcome measures were symptom scores from the Irlen Differential Perceptual Scale and the Neale analysis of reading Test. There was no difference between the experimental lens or placebo lens groups for any of the measures although there were differences with the no treatment control group which is suggestive of a strong placebo effect.

Singleton and Trotter 2005 Like many studies in this area, this study starts with a small group which is divided into even smaller groups. As a result, the statistical power of the study is limited.

There was a total of ten students with dyslexia of whom 5 had visual stress and there were ten normal readers of whom 5 had visual stress diagnosed according to the Visual Processing Problems Inventory VPPI. There was no placebo control group. The study was a crossover design that compared lenses prescribed with the intuitive colorimeter against no lens. The outcome measure was the Wilkins Rate of Reading Test.

The only group to show some improvement were the five patients with dyslexia and visual stress  However there are a number of serious problems with this study which include. Small sample size and no placebo control group.

The external validity of this study is poor because subjects were recruited via the disabilities service of the University and included subjects who may have been aware of the potential benefits of colour. Finally, because the WRRT is a non-standardised reading test and it is not clear if the results can be generalised to naturalistic text.

Machlachlan  Yale and Wilkins 1993 wasn’t a controlled trial at all. Fifty-five subjects were recruited. Twenty-three of the cases were volunteers responding to an article in 'Living magazine' and the remainder were referrals from educationalists, psychiatrists and neurologists. As a result, external validity is likely to be low.

The criteria for diagnosing visual stress was awareness of visual distortions, sustained use of overlays and having chosen a colour with the Intuitive Colorimeter.

The outcome measure was what percentage continued wearing glasses over 10 months. 82% reported that they were still using the glasses 10 months later. 

Unfortunately, this based on self-report and is unreliable. The external validity of this study is questionable because of the way the subjects were recruited and finally.. so what?

Simmers et al 2001. This study comes from a reputable psychophysical laboratory at Imperial College. It has been reviewed in a previous post. Instead of using text it looks at the kind of geometric patterns that are said to be most aversive in visual stress. The authors found no difference between subjects with visual stress and controls. In those cases with visual stress no difference in aversive symptoms when using precision tinted lenses.

So -Intuitive ColorimeterTM method for the 21st century? If you believe that you will believe anything.

1.         Kriss I, Evans BJW. The relationship between dyslexia and Meares-Irlen Syndrome. J Res Read. 2005 Aug;28(3):350–64.

2.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70.

3.         Mitchell C, Mansfield D, Rautenbach S. Coloured filters and reading accuracy, comprehension and rate: a placebo-controlled study. Percept Mot Skills. 2008 Apr;106(2):517–32.

4.         Singleton C, Trotter S. Visual stress in adults with and without dyslexia. J Res Read. 2005 Aug;28(3):365–78.

5.         MacLachlan A., Yale S., Wilkins A. Open trial of subjective precision tinting: A follow-up of 55 patients. Ophthalmic Physiol Opt. 1993;13(2):175–8.

6.         Simmers AJ, Bex PJ, Smith FK, Wilkins AJ. Spatiotemporal visual function in tinted lens wearers. Invest Ophthalmol Vis Sci. 2001 Mar;42(3):879–84.

A big problem for the visual stress hypothesis

Spatiotemporal function in Tinted Lens Wearers
Anital Simmers, Peter Bex, Fiona Smith, and Arnold Wilkins
Invest  Ophthalmol Vis Sci 2001;42:879-884

For once, a rather good study which comes from a reputable group at Imperial College London. You can download it here.
Proponents of visual stress as a factor in reading difficulties argue that the sort of discomfort and movement illusions that most of us experience in response to recurring geometric patterns, varies in the population from person to person. They go on to argue that in some susceptible individuals it can be brought on by print, resulting in distortions and movements of letters and words - so called visual stress.
The conditions that induce these symptoms, for geometric patterns, are said to be surprisingly specific and peak with high contrast square gratings between 2-8 cycles per degree. Proponents argue that text has spatial frequencies within this range see the figure below.

The text has been filtered to remove high spatial frequencies and the contrast exaggerated to make the stripes more apparent

Text obviously isn't a striped grating. However, like a piece of music which contains multiple frequencies of sound waves at the same time (which you don't notice as the music washes over you), text contains multiple spatial frequencies and it is those between 2-8 cycles per degree that are the said to be a problem in 'visual stress'.
If this is the case we would expect to see this effect in simplified settings using actual gratings rather than text. Such a study has been done on subjects with visual stress, in a reputable psychophysical laboratory. Even more interesting is that Arnold Wilkins was one of the authors although he does not emphasise the results of this study a great deal.
The subjects were twenty individuals with 'visual stress' who had successfully worn tinted lenses (prescribed with the intuitive colorimeter) for at least six months. Twenty control subjects were recruited from the staff and siblings of staff and students of the University of Exeter.
Subjects were tested with a range of psychophysical tests which are designed to assess visual processing in the range of frequencies that are said to be aversive in visual stress.  The participants with visual stress, they were tested with and without their lenses.
The tests included spatiotemporal contrast sensitivity, contrast increment thresholds, random dot motion coherence, and motion perception
Results
To put it simply, there was no difference between control subjects and visual stress subjects and just as important, in the group with visual stress there was no difference with and without their lenses.

Contrast sensitivity function for subjects with visual stress with without

 lenses

Despite this, some people (mostly with a vested interest) still claim that lenses prescribed with the intuitive colorimeter are the 'gold standard.
Will this study dent their confidence? I doubt it.