Wednesday 29 July 2015

Postscript to -The trials they don't want you to know about (1)

As a follow up to their paper published in 2011 Ritchie and colleagues published a study looking at the subjects of their first trial, one year on.

Ritchie SJ, Sala S Della, McIntosh RD. Irlencolored filters in the classroom: A 1‐year follow‐up. Mind Brain Educ. 2012Jun;6(2):74–80.

Though not as rigorous as their previous study, outlined in my last post, it does contain some fascinating data that does nothing to support the use of of colour to treat the overlap group with poor reading and visual stress. I suspect it was a bit of an afterthought put together following criticisms of their first study - that benefits would not be seen without longer term follow up.
In the first study an Irlen screener diagnosed 47 out of 61 poor readers with 'visual stress'.
Those 47 children were then tested with prescribed overlay and placebo overlay and no difference in the rate of reading using the WRRT or reading naturalistic text measured with the Gray Oral Reading Test (GORT).
After 12 months 22 out of 47 of the original sample were still using colour in some form and 18 of these were still available to study. So this was a highly selected group who passed two criteria for visual stress; the Irlen screening tests and voluntary sustained us of overlays. According to their teachers this group had used overlays or lenses for a substantial part of the time. For this reason, if coloured overlays are effective, you would expect to see real world improvements in reading in this group.
Unfortunately the controls were a group of poor readers without visual stress from the previous study. The authors ackowledge that the ideal control group would have been individuals with visual stress who had been treated with a placebo overlay.
The visual stress group was tested with the WRRT using prescribed filter, placebo filter and clear filter. In short this part of the study was a crossover or within subject design.
The other test used was the Gray Oral Reading test which is a standardised reading test that measures accuracy and rate of reading and crucially, questions to test comprehension are included. The outcome is standardised for age to produce the Oral Reading Quotient (ORQ). This means that  if a subject progresses normally for age the ORQ should remain the same. If there is a catch up in reading, as predicted by proponents of the treatment of visual stress, the ORQ should improve and then perhaps stabilise.
Results
 The WRRT. No surprises here really. There was no significant difference in reading rate between presecribed overlay, placebo overlay and clear overlay.












GORT Oral Reading Quotient ORQ
Figure two on the right shows that for both groups, the Irlen wearers and the controls, the ORQ declined over the study period. This occured in spite of imrovements in WRRT for both groups bringing in to doubt the validity of WRRT as a reading test.



Conclusions
The authors are good scientists who are careful not to over-interpret their study and they also acknowledge its weaknesses. In particular the small sample size and the lack of a visual stress control group. Nonetheless it is one of the few studies to take a longer term look at outcomes in a group where a difference might be expected to be found. That is, those with visual stress diagnosed by an Irlen-screener and who pass the criterion of voluntary sustained use over 12 months. In spite of perceived subjective benefit no improvement in real world reading was seen.

Saturday 25 July 2015

The trials they don't want you to know about (1)

Not only do proponents of the treatment of visual stress with coloured overlays and lens mis-represent their own trials, they keep quiet about a number of negative studies. This study is arguably the most rigorous trial to date and was performed by reputable scientists with no financial interest in coloured lenses and overlays.
The paper?
RitchieSJ, Della Sala S, McIntosh RD. Irlen colored overlays do not alleviate readingdifficulties. Pediatrics. 2011 Oct;128(4):e932–8.

Participants
75 below average readers were selected from a school setting in Glasgow.
These children were assessed by a qualified Irlen screener. 13 were unable to co-operate with the test and one moved out of the area. Therefore 61 children were assessed and  47 (77%) were diagnosed with Irlen syndrome (visual stress). Although the study has been criticised for the high number diagnosed it is entirely consistent with the case control studies published by Kruk et al and Kriss and Evans. Similarly, the observational study published by Tyrell and colleagues in 1995 found that 100% of well below average readers  and 75% below average readers chose an overlay
The study had a cross over design and subjects received a prescribed overlay, an overlay of a complementary colour and clear sheet.
Testing was with the Mini Mental State Exam MMSE, Wilkins Rate of Reading Test, and Gray Oral Reading Test.

The trial was rigorously conducted. Even though it was a cross over study, sequence generation was adequately generated and allocation concealment was was good so testers could not have foreseen whether subjects would have received experimental or placebo overlay first.
Steps were also take to ensure that subjects did no know which overlay was being used.
Data collection was complete and the analysis was on an intention to treat basis. In short there were no obvious sources of bias.

The results
 Irlen overlays had no demonstrable effect on reading compared to placebo overlays.
It can been seen from the figure on the left that the test retest variability for the WRRT was high and some individuals read more words per minute and some read fewer with their chosen overlay some as much as 30% fewer. 
Interestingly the two outliers shown in triangles both knew what their chosen tint was. Strong evidence for placebo effect.




WRRT reading rates for the Irlen group, non Irlen group and
3 children for whom the treatment was not masked
The pooled data doesn't look any better. Neither the Irlen group or the 'normals' read any better with overlays. The only group to do better were those who were not masked. Strong evidence for the placebo effect.







What about reading naturalistic text? - The Gray Oral Reading Test -here the study design was slightly different 22 of the subjects with visual stress received their chosen colour and 22 a clear overlay. There was no difference between the two groups.

Conclusions
The authors were unable to demonstrate any improvement in reading the WRRT or naturalistic text using Irlen overlays prescribed by a qualified Irlen practitioner.


There was also a one year follow up which will be discussed in a future post

2.Ritchie SJ, Della Sala S, McIntosh RD. Irlen Colored Filters in the Classroom: A 1-Year Follow-Up. Mind Brain Educ. 2012 Jun;6(2):74–80