Cochrane (systematic) reviews regularly reject lots of studies they consider on the grounds of poor methods. For instance, here’s a quote from a paper in the October 2000 issue of the Journal of Clinical Oncology: “From the 80 retrieved articles, 48 were rejected. All of these studies were of small sample size and many were also of doubtful quality.” 
A different perspective is provided by Richard Smith (former editor of the British Medical Journal):
“With medical research you can get anything published, there are so many journals and have been for a long time. Nobody knows exactly how many there are but there’s probably something like 15 to 20,000 and you – to be honest you can get any rubbish published, you just go down and down and down and down the food chain, as we call it.” The Other Medicine Programme 2. BBC Radio 4 Tuesday 28/09/04
Another problem is that while a study may be of high quality, it might not be appropriate to your area of practice.
A research study may exclude certain types of patients, (such as those with complex or multiple pathologies) in order to make the study ‘clean’. However, the nature of many areas of practice is that patients present with more than one problem, so an intervention which works in the study may not be appropriate.
For example, imagine a (fictional) study that looked at the effectiveness of Cognitive Behavioural Therapy (CBT) in patients with schizophrenia. In order to make the study easier to carry out the researchers decide to exclude clients with co-existing problems like alcohol or drug dependence. The study shows that CBT works. However, we would need to ask the question, can it be applied to a client group who have both schizophrenia, and drug/alcohol problems?
|Netting the evidence||A large collection of resources for Evidence Based Practice, provided by the University of Sheffield School of Health and Related Research (ScHARR). The resources are sorted into categories – Appraising, Implementing, Journals, etc – and are carefully-annotated and selected.|
|Cochrane Library||The Cochrane Library, freely available to the public. You can carry out detailed searches for reviews and trials.|
|Evidence-Based Nursing||The online version of the quarterly professional journal Evidence-Based Nursing, which is “designed to alert practising nurses to important and clinically relevant advances in treatment, diagnosis, aetiology, and prognosis”. The site hosts a full searchable archive of published articles, with abstracts and full texts.|
|Why appraise the evidence? a case study of vitamin C and the healing of pressure sores||A case study of what happens when research evidence is not properly appraised.North, G. and Booth, A. Why appraise the evidence? a case study of vitamin C and the healing of pressure sores. Journal of Human Nutrition & Dietetics, Volume 12, Number 3, 1 May 1999, pp. 237-244(8)|