Wednesday, March 7, 2012

Mechanics of Reading a Scientific Paper

I have 250+ articles in my EndNote library and although I'd like to claim that I've read every single one, I'd be lying if I did.  Reading papers can be tiresome and tedious and, in some cases, overwhelming if your research topic isn't yet narrowly defined or it's objectives keep shifting.  Frustrations aside about defining the boundaries of the lit review, though, how does one read the papers once they're retrieved?  Does one skim the papers or carefully read them?  If the latter, is there a prescribed and efficient way to do so?  When I was a TA for a Research Design course several years ago I vaguely remember a lecture on the 'how' of reading scientific papers and although I don't remember the specifics from that lecture, I do remember thinking, "mentally archive this information -- you may eventually need it."  Fortunately, I had the foresight to recognize that knowing how to systematically read a paper would come in handy but, unfortunately, I couldn't recall the specifics from that lecture.  Since that lecture many years ago, evidence-based medicine has moved mainstream and with it, the need to critically appraise literature and scientific evidence in order to shape clinical practice.  I'm, obviously, not a clinician and won't be practicing evidenced-based medicine (EBM) anytime soon but the ability to critically read scientific papers is a useful skill in any scientific discipline.  And that is where Trisha Greenhalgh's concise and readable "How to Read a Paper:  The Basics of Evidence-Based Medicine" is a must-have for virtually all researchers.  The book isn't cheap given its length -- 238 pages -- although I think the clarity and density of the information makes it well worth it.  The range of topics discussed range from how to search the literature to how to assess economic analyses.  Depending on your discipline and research perspective, some chapters are going to be more relevant than others.  For me, the chapters on how to determine what the paper is about, how to assess it's methodological quality, and the summary of statistical techniques commonly used in scientific papers were the most relevant and informative.

When trying to determine what a paper is about, Greenhalgh emphasizes that a paper should be 'trashed' because of its methods, not its results.  Given the emphasis on the methods, then, three preliminary questions should initiate the appraisal:  
  1. What was the research question -- and why was the study needed?  This should be clearly stated somewhere in the first few paragraphs of the paper.
  2. What was the research design?  The type of design has implications for the statistical analyses used (if any), conclusions, and rigor of the paper.  
  3. Was the research design appropriate to the question?  Not all research questions require a randomized controlled trial (RCT).  
In this chapter, Greenhalgh also briefly discusses each of the research designs common to scientific papers then assigns them a place in the "hierarchy of evidence" with those at the top commanding the most weight and influence re: clinical interventions.  Aside from placing systematic reviews/meta-analyses at the top (particularly helpful in EBM), I think most other disciplines would report a similar hierarchy:
  1. Systematic reviews and meta-analyses.
  2. RCTs with definitive (i.e. statistically significant) results.
  3. RCTs with non-definitive (i.e. suggestive but not statistically significant) results.
  4. Cohort studies.
  5. Case-control studies.
  6. Cross-sectional surveys.
  7. Case reports. 
In the methodological quality chapter, assessment relies on five key questions:  
  1. Was the study original?  Does it duplicate previous research or add something new to the literature?
  2. Whom is the study about?  How were subjects recruited and what were the inclusion/exclusion criteria?  
  3. Was the design of the study sensible?  What and how were the outcomes measured?
  4. Was systematic bias avoided or minimized?  Study adequately controlled?  Was assessment 'blind'?   
  5. Were preliminary statistical questions addressed -- how many subjects enrolled, duration of follow-up, and completeness of follow-up?  
The chapter on statistics is intended for non-statisticians but I still found it helpful and amusing (especially the 'advice' on how to cheat on statistical tests when writing up results).  I've written about this in a previous post and won't repeat it here since what Ben Goldacre wrote relied largely on what Greenhalgh wrote.  Suffice to say, Greenhalgh breaks down the most common statistical analyses used and their interpretation such that even the most timid statistically-averse researcher can make sense of the results. 

Even after I've long since finished slogging through all (most?) of the articles collected for my lit review, I suspect this book will still sit prominently on my bookshelf.

No comments:

Post a Comment