Metaanalysis
From PsychWiki  A Collaborative Psychology Wiki
Line 25:  Line 25:  
<li>For more <u>indepth</u> discussion and explanations, we recommend...</li>  <li>For more <u>indepth</u> discussion and explanations, we recommend...</li>  
  *start first with  +  *start first with '''[[Johnson & Eagly, 2000]]''' (from the Handbook of Research Methods in Social and Personality Psychology) which provides an excellent detailed explanation of each step in the metaanalysis process, the advantages of metaanalysis compared to traditional literature reviews, statistical forumlas for analyzing effect sizes, categorical, and continuous variables. 
*  *  
*  *  
Line 54:  Line 54:  
==What is a metaanalysis?==  ==What is a metaanalysis?==  
===Definition===  ===Definition===  
  A metaanalysis  +  A metaanalysis statistically combines the results of several studies that address a shared research hypotheses. 
  +  Just as individual studies summarize data collected from many individual participants in order to answer a specific research question (e.g., each participant is a separate data point), a metaanalysis summarizes data from individual studies that concern a specific research question (e.g., each data point is each individual study).  
  +  
  Just as  +  
The results of each individual study are converted to a standardized effect size. A metaanalysis combines...  The results of each individual study are converted to a standardized effect size. A metaanalysis combines...  
Line 72:  Line 70:  
===Five Basic Steps===  ===Five Basic Steps===  
:There are generally five separate steps in conducting a metaanalysis:  :There are generally five separate steps in conducting a metaanalysis:  
  #'''Define your hypothesis'''  +  #'''Define your hypothesis'''  
  #'''Locate the Literature'''  +  #'''Locate the Literature'''  
  #'''Identify and input data'''  +  #'''Identify and input data'''  
  #'''Cacluate Effect Sizes'''  +  #'''Cacluate Effect Sizes'''  
  #'''Analyze Variables'''  +  #'''Analyze Variables'''  
Revision as of 04:07, 18 July 2006
► Have you ever wanted to learn about metaanalysis or conduct a metaanalysis but didn't know where to start? This webpage is devoted to providing you expert opinion on what you need to know to start your own metaanalysis.
► With the thousands of metaanalyses conducted in all areas of psychology over the past few decades, there has been an everincreasing number of articles, books, and software programs devoted to how to conduct metaanalyses. Below, experts on metaanalysis provide their suggesstions on which which of the many sources of information are the most useful and why  so that the user has an easytouse starting place for learning everything about metaanalyses.
Contents 
Where should I start?
If you want to learn what is a metaanalysis...

If you want to learn how to start conducting a meta...

What is a metaanalysis?
Definition
A metaanalysis statistically combines the results of several studies that address a shared research hypotheses.
Just as individual studies summarize data collected from many individual participants in order to answer a specific research question (e.g., each participant is a separate data point), a metaanalysis summarizes data from individual studies that concern a specific research question (e.g., each data point is each individual study).
The results of each individual study are converted to a standardized effect size. A metaanalysis combines...
Three Basic Questions
 A metaanalysis answers three general questions:
 Central tendency – The central purpose of a metaanalysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect?
 Variability – There is always going to be some degree of variation between the outcomes of the individual studies that compose the metaanalysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity.
 Prediction – If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables?
Five Basic Steps
 There are generally five separate steps in conducting a metaanalysis:
 Define your hypothesis 
 Locate the Literature 
 Identify and input data 
 Cacluate Effect Sizes 
 Analyze Variables 
How do I conduct a metaanalysis?
First, choose which statistical approach suits your needs
 There are generally three different statistical approaches to conduct a metaanalysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see (Johnson, Mullen, & Salas, 1995) (Comparison of Three Major MetaAnalytic Approaches. Journal of Applied Psychology, 80, 94106). Some basic information from that article is posted below to get you started:
 Hedges & Olkin Approach – see (Hedges, 1981); (Hedges, 1982); (Hedges & Olkin, 1985)
 Rosenthal & Rubin Approach – see (Rosenthal, 1991); (Rosenthal & Rubin, 1978); (Rosenthal & Rubin, 1988)
 Hunter, Schmidt, & Jackson  see (Hunter, Schmidt, & Jackson, 1982); (Hunter & Schmidt, 1990)
Second, choose which effect size to calculate?
 asdfadsfsdf
Third, choose your statistical software
 asdfasdfds
If you want more detailed information about...
...the Hedges & Olkin approach...
 See (Lipsey & Wilson, 2001) (Practical MetaAnlaysis)  which is relatively new book that provides a concise summary of all stages of the metaanalyses process, including providing ...
 See (Cooper & Hedges, 1994) (Handbook of Research Synthesis)  which is great indepth articuluation of every step involved in designing, analyzing, and writingup a metaanalysis.
 See (Hedges & Olkin, 1985) (Statistical Methods for MetaAnalysis)  which is the original source of information about the Hedges & Olkin approach.
...the Rosenthal & Rubin approach...
 See (Rosenthal, 1991) (Metaanalytic Procedures for Social Research)  which is the definitive source of information on the Rosenthal & Rubin approach.
 See (Rosenthal & DiMatteo, 2001) (MetaAnalysis: Recent Developments in Quantitative Methods for Literature Reviews)  which is an updated summary of the Rosenthal approach.
 See (Rosenthal, 1995) (Writing MetaAnalytic Reviews  which is an excellent Psychological Bulletin article on how to write a metaanalysis.
...the Hunter, Schmidt, & Jackson approach...
 What is the good number of studies to have bare minimum for a metaanalysis? A metaanalysis with 10 studies have been published before but is not recommended.
 In a metaanalysis, have judge rate each variable across studies, one moderator at a time, instead of rating all variables in a single study before moving on to next study.
 With metaanalysis coding with a high number of studies to code, such as 75+, can have some coders rate the entire set, but can also have some coders (undergrads) code only a subset as long there is overlap, so that more than 1 judge is rating each study.
► Back to Research Tools mainpage