Some problems with the Wilkins Rate of Reading Test

The figure above shows a visual fields test that displays the sensitivity of the visual field at locations around the fixation point. The black areas represent reduced sensitivity. You might think that there has been some improvement in vision as a result of some unspecified intervention. However, if you tried to publish a figure such as this,  a good peer reviewer would spot the problem straight away.

Before you can say whether the result of a psychophysical test has improved you need to know if there is been some change in the way the observer is performing the test. Are they performing the test in a more or less conservative manner for example? For this reason, machines for testing visual fields build in catch trials to determine if the subject is becoming a more or less conservative observer. For example, they make noise as if a stimulus has been presented when it has not, or they may measure the stimulus response time - anything less than 200ms is almost certainly a false positive. Going back the case above, the tests showed that the observer had become a less conservative observer or more trigger happy if you like and this was the most likely explanation for the improvement int he visual field see below.

So, what has this got to do with the Wilkins Rate of Reading Test (WRRT)? It is important to

remember that the WRRT is not a standardised reading test. It not used by anyone except a
aficionados of visual stress and its treatment. The WRRT does not consist of naturalistic text but closely spaced, commonly used words, ordered randomly without syntax, punctuation or paragraphs. Its proponents argue that it is well suited to detecting visual problems. That may be true, but it also has some key flaws. The principle one being that it can not measure 'response criterion' That is, has the observer become more or less conservative in his responses? It can not be said to isolate out visual factors if you can not measure other aspects of reading. For example, you might find that subjects were able to read faster but at the expense of comprehension, parsing or miscues.
I know this from my own experience of learning Spanish through skype. When asked to read text I can easily vary my rate of reading making it sound fast and 'spanishy'. However, my teacher notices the parsing goes a bit funny and she throws in a comprehension question. The rate of reading then comes crashing down. Although she wouldn't call it that, she has recognized that my response criterion has changed and I have become a less conservative reader - taking risks with pronunciation, sentence structure, and meaning.
So, returning to the WRRT unless you can show that other aspects of reading are conserved you can not claim to have isolated out visual aspects of reading. Also, you can not claim that any improvements in the WRRT have any relevance to real world reading.
Still, you have to admire somebody who has found a way to make money out of printing random words.