Prevalence of visual stress in dyslexia and controls (2)

If the first study was reasonably rigorous this study is the opposite. First, the personal biases of the authors are made transparent in the citation distortions which litter the paper. Second, for reasons which I will outline, the study is at high risk of bias in the statistical sense of the word.
The paper?
Kriss I, Evans BJW. The relationship between dyslexia and Meares-Irlen Syndrome. J Res Read. 2005 Aug;28(3):350–64.

I will be listing the more blatant citation distortions in the appendix to the post. In the meantime on with the paper.

Participants

• Controls: 32 children selected from a classroom setting who had a reading age that was appropriate for their chronological age.

• Cases: 32 children with dyslexia recruited from classrooms in various state schools and dyslexia clubs. It can be seen straight away that there is ample room for bias to creep in. Cases and controls are not drawn from the same population. Children attending dyslexia clubs are probably not be representative of the generality of poor readers and are more likely to have been exposed to the belief that coloured overlays and lenses may be beneficial. The authors state that an additional criteria was to have been labeled as having dyslexia by an education psychologist. It is not clear what criteria the educational psychologist was using. Was the assessment part of the study or was it simply reported by the parents of the children? It is well recognised that it is possible to 'shop around' among educational psychologists to get a diagnosis of dyslexia. The selection of cases puts this study at high risk of bias and contrasts unfavourably with the study by Kruk and colleagues described in the previous post.

Diagnostic criteria

Visual stress was diagnosed using the Wilkins Rate of Reading Test (WRRT) which has been previously described in this blog. It does not consist of naturalistic text. Instead, commonly used words are presented in random order in a small font with closely spaced lines. Subjects had to read 5%, 8% or 10% faster using their chosen overlay to be diagnosed with visual stress. The problem was that the screeners were not blinded to the reading status of the subjects. As a result ascertainment bias is likely.

The potential sources of bias in this study were

• Cases and controls not selected form the same population
• Diagnostic criteria for dyslexia not outlined
• Screeners not blinded to reading status of subjects

Results

Although the odds ratio for 5%, 8% and 10% faster on the WRRT are all above one, the 95% confidence intervals were very wide and the results do not reach statistical significance. Given the positive spin people like to put on data, the results have been described as approaching statistical significance. How do you know? They could have been running in the opposite direction.

Conclusions

There are so many sources of bias that are not even acknowledged by the authors,that this study can not be used in a meta-analysis. Even if the data is taken at 'face value' it remains unproven that the visual stress, as measured with WRRT, is more prevalent in the population with dyslexia.

Appendix -citation distortions

Page 351 Paragraph1 ..to date there have been two rigorous double masked randomised controlled trials (Wilkins et al 1994; Robinson Foreman 1999) These trial support the existence of this syndrome and validate the treatment with individually prescribed coloured filters.
See Holy Trinity one for a review of Wilkins et al 1994 and Holy Trinity three for a review of Robinson and Forman 1999. These studies can not be described as rigorous and neither do they support the use of coloured filters.

Page 351 Paragraph 1 This accounts for a great deal of the controversy in the literature: studies using individually prescribed filters tend to be positive whilst those that test all participants with the same colour or a very limited range of colours tend to be negative (Evans 2001)
Again, not true. the reference is to narrative review by one of the authors. RCTs consistently show that individual tints are no better or worse than placebo(1-5)

Page 351 paragraph 2 The first double-masked randomised placebo-controlled trial found that individually prescribed coloured filters (precision tinted lenses) brought about a significant reduction in symptoms of eyestrain and headache compared with control lenses of a similar but different colour (Wilkins et al., 1994). 
Oh no it didn't - data was only available for 36 out of 68 participants. The loss to follow up was so high that no valid interpretation of the results is possible. They also neglect to mention that the slightly more robust conclusion with a follow up of 45/68 was that there was no improvement in reading rate, accuracy or comprehension with optimal tint compared to control tint.

Page 351 paragraph 3 The second double-masked randomised-controlled trial investigated the effects of coloured filters on reading speed, accuracy, comprehension and self-perception of academic ability, with the widely used Neale Analysis of Reading Test (Robinson & Foreman, 1999). A total of 113 participants were divided into three groups either using placebo filters, standard blue filters or optimal (individually prescribed) filters. Compared with the other groups, the group using optimal filters increased markedly in reading accuracy and comprehension, but not in speed (see below) 
Again not true. There was no difference between optimal tint, placebo tint and blue tint for the three months that was actually an RCT. The difference was to the untreated control group. 

Page 361 paragraph 4 The current ‘gold standard’ treatment is precision tinted lenses that have been individually prescribed after systematic testing with a wide and comprehensive range of colours, for example using the Intuitive Colorimeter
Irrelevant to the subject of this paper. Not supported by RCTS (1,3). Professor Evans give lectures paid for by Cerium who manufacture the 'Intuitive colorimeter'

There is plenty more like this. It is widely accepted that making up results is scientific fraud. In my opinion a change of culture is required to stop this sort mis-representation of the research literature. It is quite prevalent and viewed as a 'fair game' by some it is therefore unfair to single the authors out for criticism in this regard. See my post citation distortion.

1.         Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70.

2.         Robinson GL, Foreman PJ. Scotopic sensitivity/Irlen syndrome and the use of coloured filters: a long-term placebo-controlled study of reading strategies using analysis of miscue. Percept Mot Skills. 1999 Feb;88(1):35–52.

3.         Mitchell C, Mansfield D, Rautenbach S. Coloured filters and reading accuracy, comprehension and rate: a placebo-controlled study. Percept Mot Skills. 2008 Apr;106(2):517–32.

4.         Ritchie SJ, Della Sala S, McIntosh RD. Irlen colored overlays do not alleviate reading difficulties. Pediatrics. 2011 Oct;128(4):e932–938.

5.         Harries P, Hall R, Ray N, Stein J. Using coloured filters to reduce the symptoms of visual stress in children with reading delay. Scand J Occup Ther. 2015 Mar;22(2):153–60.