Let’s take a short trip into the woods and discuss Forest Plots (They are called that because if you lay the graph on its side ups and downs look like tree line…or maybe because everyone gets lost in them…). Let’s look at the one below from a Cochrane review 1.
It looks daunting but really once we break it down its not bad:
- The main graph looks like an upside down T. The bottom horizontal line is a binary decision that tells you if one side or the other is favored. The far left favors control and and the far right favors the experiment. In this case spacer (left) or nebulizer (right). The vertical line is smack dab in the middle of the two and is the point of “no effect”. If anything touches that line then it means that whatever you are looking at has no effect (effect of control=effect of treatment).
- Starting from the top down on the graph we see a box with whiskers. The box is the average result of the study and the whiskers are the 95% confidence interval. Remember that when we ask a question there is no ONE answer (average). The answer lies somewhere within an interval that we can be 95% sure contains that answer.
- Another thing to note about the box and whisker is that a study with a large sample size will have short whiskers and a small sample size will have long whiskers (larger studies by their nature are more precise)
- Included data: The plot will also contain columns to the right of the graph. In this case we have: Study, Mean, Standard Error (SE), Weight, and the 95% confidence interval. I want to point out the weight column. Because some studies have more patients than others they contribute differently to the overall forest plot. The weight tells you how much that single study contributed. Larger studies have higher weights and will contribute more to the overall effect. This is important because a meta-analysis might have its entire results based on ONE study that overpowers all the others
- The Diamond: The middle of the diamond is the mean of the result of all the above trials. The ends of the diamond are the 95% confidence interval. So if the diamond touches the “line of no effect”, there is no difference between control and experiment.
- Heterogeneity: Lastly is the measure of how similar the studies are. Ideally a meta analysis combines a number of randomized control trials that are all done on the SAME type of patient with the same outcome. But remember studies aren’t usually done like that (which sounds crazy, I know). They are done the way the researcher has the ability to get them done. So, for example in this study although we compare spacers to nebs did the researchers really get only COPD patients? Did they look at the same primary outcome (e.g. FEV or Dyspnea score)? This could make the trials fundamentally different. Therefore, we need to have some way to say yes all these studies are very similar (homogenous) or no these studies are very different (heterogenous). So for once we have a stat that makes sense! The “heterogeneity score”! Also referred to as I2, a high heterogeneity score means the studies are very different (and maybe can’t be compared together) and a low heterogeneity score means. A heterogeneity score >50% is not great. The example study is 47% (not great).
REFERENCES
- van Geffen, W. H., Douma, W. R., Slebos, D. J. & Kerstjens, H. A. Bronchodilators delivered by nebuliser versus pMDI with spacer or DPI for exacerbations of COPD. Cochrane Database Syst Rev CD011826 (2016).
Reviews in EM (REM) 