# Meta-analysis

(Difference between revisions)
Jump to: navigation, search
 Revision as of 04:07, 18 July 2006 (view source)Stenstro (Talk | contribs)← Older edit Revision as of 04:21, 18 July 2006 (view source)Stenstro (Talk | contribs) Newer edit → Line 64: Line 64: ===Three Basic Questions=== ===Three Basic Questions=== :A meta-analysis answers three general questions: :A meta-analysis answers three general questions: - #'''Central tendency''' – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect? + #Central tendency – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect? - #'''Variability''' – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity. + #Variability – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity. - #'''Prediction''' – If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables? + #Prediction – If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables? ===Five Basic Steps=== ===Five Basic Steps=== :There are generally five separate steps in conducting a meta-analysis: :There are generally five separate steps in conducting a meta-analysis: - #'''Define your hypothesis''' - + #Define your hypothesis – From readhing literature, define, set boundaries, etc. - #'''Locate the Literature''' - + #Locate the Literature – The central - #'''Identify and input data''' - + #Identify and input data – Gather empiricial findings from primary studies (e.g., p-value, effect size, etc) and input into statistical database. - #'''Cacluate Effect Sizes''' - + #Cacluate Effect Sizes – Calculate the overall effect -- effect size, significance level or confidence intervals associated with the effect size, and variaiblity (homogeneity) of the effect. - #'''Analyze Variables''' - + #Analyze Variables – Code your study variables, input the data into the database, and analyze the results. There are two types of study variables: (a) objective variables -- such as type of IV or DV, ..., (b) subjective variables -- inferential judgements made by two or more judges.... Line 80: Line 80: ===First, choose which statistical approach suits your needs=== ===First, choose which statistical approach suits your needs=== :There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see '''[[Johnson, Mullen, & Salas, 1995]]''' (Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started: :There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see '''[[Johnson, Mullen, & Salas, 1995]]''' (Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started: - #'''Hedges & Olkin Approach''' – see [[Hedges, 1981]]; [[Hedges, 1982]]; [[Hedges & Olkin, 1985]] + #Hedges & Olkin Approach – see [[Hedges, 1981]]; [[Hedges, 1982]]; [[Hedges & Olkin, 1985]] - #'''Rosenthal & Rubin Approach''' – see [[Rosenthal, 1991]]; [[Rosenthal & Rubin, 1978]]; [[Rosenthal & Rubin, 1988]] + #Rosenthal & Rubin Approach – see [[Rosenthal, 1991]]; [[Rosenthal & Rubin, 1978]]; [[Rosenthal & Rubin, 1988]] - #'''Hunter, Schmidt, & Jackson''' -  see [[Hunter, Schmidt, & Jackson, 1982]]; [[Hunter & Schmidt, 1990]] + #Hunter, Schmidt, & Jackson -  see [[Hunter, Schmidt, & Jackson, 1982]]; [[Hunter & Schmidt, 1990]] ::[[Image:Table1JMS2.gif]] ::[[Image:Table1JMS2.gif]] Line 92: Line 92: ===Third, choose your statistical software=== ===Third, choose your statistical software=== :asdfasdfds :asdfasdfds + + DSTAT calculates all of this for you + Can also use SPSS and macros from “Practical Meta-Analysis” + Calculate Categorical variables – DSTAT using weighted ANOVA + Calculate Continuous variables – SPSS using weighted Regression + + # # # #

## Revision as of 04:21, 18 July 2006

► Have you ever wanted to learn about meta-analysis or conduct a meta-analysis but didn't know where to start? This webpage is devoted to providing you expert opinion on what you need to know to start your own meta-analysis.

► With the thousands of meta-analyses conducted in all areas of psychology over the past few decades, there has been an ever-increasing number of articles, books, and software programs devoted to how to conduct meta-analyses. Below, experts on meta-analysis provide their suggesstions on which which of the many sources of information are the most useful and why -- so that the user has an easy-to-use starting place for learning everything about meta-analyses.

## Where should I start?

 If you want to learn what is a meta-analysis... For the basics, see below where we lay out: the definition of a meta-anlaysis, the three basic questions answered by a meta, the the five steps involved in a meta-analysis, For more in-depth discussion and explanations, we recommend... start first with (Johnson & Eagly, 2000) (from the Handbook of Research Methods in Social and Personality Psychology) which provides an excellent detailed explanation of each step in the meta-analysis process, the advantages of meta-analysis compared to traditional literature reviews, statistical forumlas for analyzing effect sizes, categorical, and continuous variables. If you want to learn how to start conducting a meta... For the basics, see below were we lay out: For more in-depth discussion and explanations, we recommend... start first with (Johnson, Mullen, & Salas, 1995) which provides a statistical comparision of the three major meta-analytic approaches using actual datasets, as well as the staistical forumulas for each approach and the methodological differences between each approach.

## What is a meta-analysis?

### Definition

A meta-analysis statistically combines the results of several studies that address a shared research hypotheses.

Just as individual studies summarize data collected from many individual participants in order to answer a specific research question (e.g., each participant is a separate data point), a meta-analysis summarizes data from individual studies that concern a specific research question (e.g., each data point is each individual study).

The results of each individual study are converted to a standardized effect size. A meta-analysis combines...

### Three Basic Questions

A meta-analysis answers three general questions:
1. Central tendency – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect?
2. Variability – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity.
3. Prediction – If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables?

### Five Basic Steps

There are generally five separate steps in conducting a meta-analysis:
1. Define your hypothesis – From readhing literature, define, set boundaries, etc.
2. Locate the Literature – The central
3. Identify and input data – Gather empiricial findings from primary studies (e.g., p-value, effect size, etc) and input into statistical database.
4. Cacluate Effect Sizes – Calculate the overall effect -- effect size, significance level or confidence intervals associated with the effect size, and variaiblity (homogeneity) of the effect.
5. Analyze Variables – Code your study variables, input the data into the database, and analyze the results. There are two types of study variables: (a) objective variables -- such as type of IV or DV, ..., (b) subjective variables -- inferential judgements made by two or more judges....

## How do I conduct a meta-analysis?

### First, choose which statistical approach suits your needs

There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see (Johnson, Mullen, & Salas, 1995) (Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started:
1. Hedges & Olkin Approach – see (Hedges, 1981); (Hedges, 1982); (Hedges & Olkin, 1985)
2. Rosenthal & Rubin Approach – see (Rosenthal, 1991); (Rosenthal & Rubin, 1978); (Rosenthal & Rubin, 1988)
3. Hunter, Schmidt, & Jackson - see (Hunter, Schmidt, & Jackson, 1982); (Hunter & Schmidt, 1990)

asdfadsfsdf

### Third, choose your statistical software

asdfasdfds

DSTAT calculates all of this for you Can also use SPSS and macros from “Practical Meta-Analysis” Calculate Categorical variables – DSTAT using weighted ANOVA Calculate Continuous variables – SPSS using weighted Regression

## If you want more detailed information about...

### ...the Hedges & Olkin approach...

• See (Lipsey & Wilson, 2001) (Practical Meta-Anlaysis) - which is relatively new book that provides a concise summary of all stages of the meta-analyses process, including providing ...
• See (Cooper & Hedges, 1994) (Handbook of Research Synthesis) - which is great in-depth articuluation of every step involved in designing, analyzing, and writing-up a meta-analysis.
• See (Hedges & Olkin, 1985) (Statistical Methods for Meta-Analysis) - which is the original source of information about the Hedges & Olkin approach.

### ...the Rosenthal & Rubin approach...

• See (Rosenthal, 1991) (Meta-analytic Procedures for Social Research) - which is the definitive source of information on the Rosenthal & Rubin approach.
• See (Rosenthal & DiMatteo, 2001) (Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews) - which is an updated summary of the Rosenthal approach.
• See (Rosenthal, 1995) (Writing Meta-Analytic Reviews - which is an excellent Psychological Bulletin article on how to write a meta-analysis.

### ...the Hunter, Schmidt, & Jackson approach...

• What is the good number of studies to have bare minimum for a meta-analysis? A meta-analysis with 10 studies have been published before but is not recommended.
• In a meta-analysis, have judge rate each variable across studies, one moderator at a time, instead of rating all variables in a single study before moving on to next study.
• With meta-analysis coding with a high number of studies to code, such as 75+, can have some coders rate the entire set, but can also have some coders (undergrads) code only a subset as long there is overlap, so that more than 1 judge is rating each study.

► Back to Research Tools mainpage