Systematic reviews

It is essential to evaluate the claims and counterclaims using evidence from systematic peer reviewed research. This code of conduct is based upon the best available scientific evidence. In particular double masked randomised controlled trials...

Code of conduct of the Society for Coloured Lens Prescribers. Committee members Arnold Wilkins, Bruce Evans, Andrew Field, Sir Patrick Cable-Alexander and Catherine Porter.

A systematic review is not just a very thorough narrative review that takes care to include all of the literature, although that is an important component. Neither is it a relative score of papers that support or do not support a treatment. The reasons for this should be obvious. Small scale studies at high risk of bias usually outnumber the larger and better studies because they are easier to do. Furthermore, it would be easy for unscrupulous proponents of a treatment to 'flood the market' with low-quality trials that support an argument.
An exhaustive search of the literature is only the first step. However, even this is unlikely to uncover all relevant research. This is because of publication bias or the file drawer effect which means that negative studies tend not to get published and languish in filing cabinets. This is why there is a call for all trials to be pre-registered and published in some format even if they are 'negative' and do not support the proposed treatment.
Having identified all trials the next key step is to analyse all studies according to a template to assess the risk of bias. The most commonly used are those published by the Cochrane Collaboration.

The first stage is the analysis of all studies according to tools for analysing the risk of bias in a number of domains including random sequence generation, allocation concealment, the similarity of groups at baseline, blinding of personnel and participants, blinding of outcome assessment, attrition bias and reporting bias.

Then a composite picture can be built up of all the trials and their risk of bias which is usually tabulated in some form (see below).

The next stage is building up a table indicating the risk of bias in different domains. Red = high risk Pink = uncertain

The analysis of bias is used as a filter so that the studies at high risk of bias are rejected and only those at low risk of bias included. No studies are perfect and there may be some areas of uncertainty but studies at high risk of bias are excluded

Studies at high risk of bias are excluded

The final stage, if there is enough high quality data and if it is sufficiently homogeneous, is meta-analysis where the data from a number of studies are combined to see if there is a significant effect.
However, not all systematic reviews contain a meta-analysis. This may be because there may not be enough data at low risk of bias or that data may be too heterogeneous to allow summation and pooled analysis.

The final step may be meta-analysis of there is sufficient homogeneous high quality data

Often this evidence based approach is expressed in negative terms - 'abominable no men debunking treatments'. However, it does have a positive side. Not only are people spared ineffective and possibly dangerous treatments through this approach, it can confirm that there is a treatment effect without recourse to larger and more expensive trials. For example, if the trials of aspirin to reduce stroke risk had been subjected to meta-analysis at an earlier stage a positive impact of treatment could have identified and a large a number of preventable strokes avoided.

Systematic reviews of coloured overlays and lens for reading difficulties

There are two systematic reviews that cover this area both of which are reviewed in a previous blog post of March 2106 'A Review of Reviews of Visual Stress Treatment' Both reviews conclude that there is insufficient evidence to justify the use coloured lens and overlays to ameliorate reading difficulties in subjects with visual stress. So, returning to the Society for Coloured lenses Prescribers; according to their own code of conduct which calls for an evidence based,  systematic approach to double blind randomised controlled trials we should not endorsing the use of coloured lenses and overlays.