Imaging studies and visual stress (1)

OK Mr Phillip Jones. You are quite welcome to perform both the SPECT brain scan and the fMRI brain scan in a series of brain scans. Then we can both satisfy our curiosities. Of course I am an Irlen Syndrome sufferer and a severe one at that. Migraines are common place for me without my Irlen tints. But I believe you can't be bothered because you are ultimately arrogant and up yourself. However you are welcome to prove me wrong and then the egg is on my face. Look forward to hearing from you.

So says this individual who posted on a YouTube video about coloured glasses and dyslexia (not migraine). Although she was -looking forward to hearing from me- she disabled the reply function so I was not able to respond. Typical of these people. Although I have not yet got round to looking at visual stress and migraine I will be sure to to do so in future posts. Nonetheless, I thought it would be interesting to look at imaging studies that purport to show brain changes in visual stress.

Background
There is a good deal of concern about the misuse of fMRI and other types of imaging studies of the brain. So much so that there are websites which specifically aim debunk some of the wilder claims of fMRI studies. A good place to start is this section of the Brain Decoder Blog. 
The problem is that the colours on MRI scans are only statistical maps of blood flow, they are not direct measures of neuronal activity and as such they very prone to statistical artefacts.

fMRI study of a dead salmon (without visual stress)

For example, this fMRI scan of a dead salmon was made using the kind of protocols commonly used in fMRI studies. It clearly shows areas in the salmon's brain lighting up. To counter the problems of low statistical power and imperative to publish, a number of groups are proposing that researchers pre-register their proposed experimental methods and statistical analyses. This should prevent post hoc data trawls which are highly likely to produce false positive results. There is good evidence that this kind of bias is present in the published literature. John Ioannidis has looked at the statistical power of most published studies using fMRI scans and finds far more positive results are being found than would be expected (1) (download here). Clearly, there is a problem afoot. John Ioannidis goes on to suggest that small sample sizes and a flexible approach to data analysis account for the numerous false positive studies which can not be replicated. Unfortunately, the small-scale studies of fMRI scans in visual stress are at a high risk of bias and are at most hypothesis generating.
A second problem is that a positive finding on fMRI studies is not a substitute for finding a real clinical effect. For example, if you found the overlays improved reading but had no effect on fMRI parameters you would still implement treatment. Conversely, if you found changes on the fMRI that were not matched by clinically significant effects so what?
We are treating real people, not fMRI scans.
Of course, there are good reasons why proponents of coloured lenses and overlays promote weak evidence from neuroimaging studies. 

• Brain branding as it is called - sells. If you can link a product to some 'neuro-sciencey' sounding data particularly if it involves images from fMRI scans it is likely to sell (2).
• fMRI scans are persuasive. Explanations couched in neuroscientific terms are more likely to be believed and logical shortcoming overlooked(3). Although the findings of this paper have perhaps, in their turn, been overhyped by sceptics.
• It helps them overlook the weakness of the clinical evidence base.

fMRI studies
So what about fMRI studies in the overlap group visual stress and dyslexia(4,5). The two studies published so far, are at best pilot studies. They do not provide evidence that counterbalances the results from clinical trials that show colour does not alleviate reading difficulties in the overlap group with 'visual stress'. Neither do they provide support for Arnold Wilkins' theory of hyper-excitability of occipital cortical dominance columns. The effects, if real, seem to be confined to the left superior and middle temporal gyri.

In short; fMRI studies do not provide any support for the use of coloured overlays and lenses in individuals with dyslexia and 'visual stress'. In addition, they do not provide support for the speculative neuro-scientific theories used to explain 'visual stress'.

SPECT studies

So what about the SPECT scans mentioned at the beginning of the post? SPECT stands for single photon emission tomography. The spatial resolution is very poor and it is not used for clinical research anymore.

These scans originate from the lucrative Amen clinics run by

TV regular Daniel Amen. They appear where anyone is desperately

plugging Irlen filters.

The images on the left originate for the lucrative private clinic run by Daniel Amen; psychiatrist and TV pundit. According to the president of the American Psychological Association (APA) "in my opinion, what he [Amen] is doing is the modern equivalent of phrenology" and "the claims he makes are not supported by reliable science, and one has to be sceptical about his motivation."
In a consensus statement, the APA states "There are currently no brain imaging biomarkers that are currently clinically useful for any diagnostic category in psychiatry" However, this has not stopped the 'Irlen establishment' from plugging these images.


Conclusion: Neuroimaging studies do not support the use of colour to treat visual stress.

1.         David SP, Ware JJ, Chu IM, Loftus PD,Fusar-Poli P, Radua J, et al. Potential Reporting Bias in fMRI Studies of theBrain. Kilner J, editor. PLoS ONE. 2013 Jul 25;8(7):e70104.

2.         ChancellorB, Chatterjee A. Brain Branding: When Neuroscience and Commerce Collide. AJOBNeurosci. 2011 Oct;2(4):18–27.

3.         Weisberg DS, Keil FC, Goodstein J, Rawson E, Gray JR. The seductive allure of neuroscience explanations. J Cogn Neurosci. 2008 Mar;20(3):470–7.

4.         Chouinard BD, Zhou CI, Hrybouski S, Kim ES, Cummine J. A Functional Neuroimaging Case Study of Meares–Irlen Syndrome/Visual Stress (MISViS). Brain Topogr. 2012 Jul;25(3):293–307.

5.         KimJH, Seo H-J, Ha S-G, Kim S-H. Functional Magnetic Resonance Imaging Findings inMeares-Irlen Syndrome: A Pilot Sudy. Korean J Ophthalmol. 2015;29(2):121.

The 5% - 25% myth

Coloured overlays have been shown to reduce visual stress and increase reading speed on the Rate of Reading Test by more than 25% in 5% of children

This 5% read 25% faster myth crops up regularly wherever people are marketing coloured overlays and lenses. For example, it is repeated in the British Dyslexia Association Webinar Visual Stress: What Everyone Should Know. At 1minute 2 seconds it states - 1994 - Arnold Wilkins published peer reviewed research showing that 5% of children improved reading speed by 25% when reading through coloured overlays.  In fact, Arnold Wilkins' RCT published in 1994 showed nothing of the sort. Using the Neale Analysis of reading test, no difference in reading speed, accuracy or comprehension was found between children wearing experimental and control lenses(1).
In the same vein, in their abstract to the paper 'The relationship between dyslexia and Meares-Irlen Syndrome'  Kriss and Evans repeat this statistic. Again, in a way that can not be evaluated.
I think I have tracked down the paper that reports this finding and according to the abstract 5% of a sample of children read more than 25% faster with coloured overlays (2).

Wilkins Arnold, Lewis Elizabeth, Rowland Elizabeth, Tweedie Wendy.
Coloured overlays and their benefit for reading.
J Res Read. 2001;24(1):41–64.  

It's a long and involved paper that describes three observational studies and contains multiple statistical comparisons. My concerns can be summed as. 1) No placebo control group. 2) Reading measured with jumbled text (Wilkins Rate of Reading Test) 3) Questionable statistical reasoning. 4) 40% of schools involved dropped out of part III the study

First, however, a little more about the paper. The first part of the study looked at 47 boys and 42 girls attending middle school in Norwich. They were assessed for coloured overlays on two occasions by different examiners and 53% of cases subjects chose a different colour overlay on the second occasion they were tested. More importantly this part of the paper contains crucial data which aids in the interpretation of the third part of the paper. The Wilkins Rate of Reading test which consists of jumbled words was administered to subjects on two occasions without any overlays so we can get a measure of test/retest variability.

Wilkins Rate of Reading Test on two occasions
red line = >25% faster on second occasion

Looking at the scatterplot to the left it can be seen that some subjects read faster or slower the on the second occasion of testing. Drawing a line indicating 25% faster or more it can be seen that three subjects meet this criterion. This means that without the intervention of any overlays 3.4% read 25% faster or more. Without the use of any overlays, using the same kind of statistical reasoning employed by the authors,  a similar percentage of subjects read 25% faster or more.

Moving on to part III of the paper; the part that generated the '5% -25%' myth.

20 Schools started out participating in the study but only 12 ended up completing the project. So there are issues of selection bias. We need to know more about why and when 8 schools dropped out. Intuitive overlays were give to classroom teachers and pupils who were assessed with the Wilkins Rate of Reading Test and a conventional test of reading the Young's Group Reading Test.

Initially 60% of of 426 children reported an improvement in perception using overlays. Eight months later 48% of this group had dropped out so 31% of the original sample were still using overlays. And don't forget that 40% of schools had already dropped out for one reason or another so this is a pretty select group.

The 5% read 25% faster statistic is not mentioned outside the abstract. Was it 5% of the 31% children in 12 schools who remained in the study? In which case on an intention to treat basis it would be less than 2.5% who read 25% faster. You can't to tell from the data presented. Even if it is taken at face value it is not an impressive statistic. Part one of the study showed that without overlays 3.4% of subjects read 25% faster at a second test. 

An additional problem was the poor correlation between scores on the WRRT and measures of reading of naturalistic test. The figure on the left shows that many good readers scored badly on the WRRT and conversely some weak readers did well on the WRRT which only measures words per minute without any measure of comprehension. For the same reading quotient subjects varied from 30 to 120 words per minute on the WRRT. We can not assume that improvements on the WRRT translate into benefits for real world reading.

Conclusion:  The claim that 5% of children read 25% faster with coloured overlays is weak and not really supported by evidence.

1)Wilkins AJ, Evans BJ, Brown JA, Busby AE, Wingfield AE, Jeanes RJ, et al. Double-masked placebo-controlled trial of precision spectral filters in children who use coloured overlays. Ophthalmic Physiol Opt J Br Coll Ophthalmic Opt Optom. 1994 Oct;14(4):365–70.

2)Wilkins Arnold, Lewis Elizabeth, Rowland Elizabeth, Tweedie Wendy. Coloured overlays and their benefit for reading. J Res Read. 2001;24(1):41–64