# Meta-analysis

### From PsychWiki - A Collaborative Psychology Wiki

(Difference between revisions)

Line 1: | Line 1: | ||

Meta-analysis | Meta-analysis | ||

- | + | have you ever wanted to learn about meta-analysis or conduct a meta-analysis but didnt know where to start? This webpage is devoted to providing | |

==Where should I start?== | ==Where should I start?== | ||

*If you want to learn what is a meta-analysis... | *If you want to learn what is a meta-analysis... | ||

- | :#For the basics behind | + | :#For the basics behind meta-analyses see |

:#For more in-depth | :#For more in-depth | ||

:# | :# | ||

*If you want to start conducting a meta-analysis... | *If you want to start conducting a meta-analysis... | ||

- | :#For the basics | + | :#For the basics behind conducting a meta-analysis, see |

- | :# | + | :#For more in-depth |

:# | :# | ||

Line 18: | Line 18: | ||

===Definition=== | ===Definition=== | ||

===Three Basic Questions=== | ===Three Basic Questions=== | ||

- | A meta-analysis answers three general questions: | + | :A meta-analysis answers three general questions: |

- | # Central tendency – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect? | + | #'''Central tendency''' – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect? |

- | # Variability – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity. | + | #'''Variability''' – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity. |

- | # Prediction – If there is heterogeneity (variablitiy), then we look for moderating variables that explain the variablitty? In other words, does the effect of X on Y differ with moderator variables? | + | #'''Prediction''' – If there is heterogeneity (variablitiy), then we look for moderating variables that explain the variablitty? In other words, does the effect of X on Y differ with moderator variables? |

===Five Basic Steps=== | ===Five Basic Steps=== | ||

- | + | :There are generally five separate steps in conducting a meta-analysis: | |

- | Define your hypothesis | + | #'''Define your hypothesis''' |

- | Locate the Literature | + | #'''Locate the Literature''' |

- | Identify and input data | + | #'''Identify and input data''' |

- | Cacluate Effect Sizes | + | #'''Cacluate Effect Sizes''' |

- | Analyze Variables | + | #'''Analyze Variables''' |

==How do I conduct a meta-analysis?== | ==How do I conduct a meta-analysis?== | ||

===First, choose what statistical approach suits your needs=== | ===First, choose what statistical approach suits your needs=== | ||

+ | :There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see '''[[Johnson, Mullen, & Salas, 1995]]''' (Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started: | ||

+ | #'''Hedges & Olkin Approach''' – [[Hedges, 1981]]; [[Hedges, 1982]]; [[Hedges & Olkin, 1985]] | ||

+ | #'''Rosenthal & Rubin Approach''' – [[Rosenthal, 1991]]; [[Rosenthal & Rubin, 1978]]; [[Rosenthal & Rubin, 1988]] | ||

+ | #'''Hunter, Schmidt, & Jackson''' - [[Hunter, Schmidt, & Jackson, 1982]]; [[Hunter & Schmidt, 1990]] | ||

+ | |||

+ | ::[[Image:Table1.gif]] | ||

+ | |||

===Second, choose which effect size to calculate?=== | ===Second, choose which effect size to calculate?=== | ||

===Third, choose your variables=== | ===Third, choose your variables=== | ||

Line 45: | Line 52: | ||

- | |||

- | |||

- | |||

- | |||

- | |||

- | |||

- | |||

- | |||

Just as an individual study collects data from many individuals (data points) that is statistically summarized to answer the question of interest, a meta-anlay | Just as an individual study collects data from many individuals (data points) that is statistically summarized to answer the question of interest, a meta-anlay |

## Revision as of 00:49, 18 July 2006

Meta-analysis

have you ever wanted to learn about meta-analysis or conduct a meta-analysis but didnt know where to start? This webpage is devoted to providing

## Where should I start?

- If you want to learn what is a meta-analysis...

- For the basics behind meta-analyses see
- For more in-depth

- If you want to start conducting a meta-analysis...

- For the basics behind conducting a meta-analysis, see
- For more in-depth

## What is a meta-analysis?

### Definition

### Three Basic Questions

- A meta-analysis answers three general questions:

**Central tendency**– The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect?**Variability**– There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity.**Prediction**– If there is heterogeneity (variablitiy), then we look for moderating variables that explain the variablitty? In other words, does the effect of X on Y differ with moderator variables?

### Five Basic Steps

- There are generally five separate steps in conducting a meta-analysis:

**Define your hypothesis****Locate the Literature****Identify and input data****Cacluate Effect Sizes****Analyze Variables**

## How do I conduct a meta-analysis?

### First, choose what statistical approach suits your needs

- There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see
**(Johnson, Mullen, & Salas, 1995)**(Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started:

**Hedges & Olkin Approach**– (Hedges, 1981); (Hedges, 1982); (Hedges & Olkin, 1985)**Rosenthal & Rubin Approach**– (Rosenthal, 1991); (Rosenthal & Rubin, 1978); (Rosenthal & Rubin, 1988)**Hunter, Schmidt, & Jackson**- (Hunter, Schmidt, & Jackson, 1982); (Hunter & Schmidt, 1990)

### Second, choose which effect size to calculate?

### Third, choose your variables

### If you want to follow the Hedges & Olkin approach...

### If you want to follow the Rosenthal & Rubin approach...

### If you want to follow the Hunter, Schmidt, & Jackson approach...

Just as an individual study collects data from many individuals (data points) that is statistically summarized to answer the question of interest, a meta-anlay

- What is the good number of studies to have bare minimum for a meta-analysis? A meta-analysis with 10 studies have been published before but is not recommended.
- In a meta-analysis, don’t have raters code conditions for which no effect sizes can be calculated.
- In a meta-analysis, have judge rate each variable across studies, one moderator at a time, instead of rating all variables in a single study before moving on to next study.
- With meta-analysis coding with a high number of studies to code, such as 75+, can have some coders rate the entire set, but can also have some coders (undergrads) code only a subset as long there is overlap, so that more than 1 judge is rating each study.

► Back to Research Tools mainpage