Meta-analysis

From PsychWiki - A Collaborative Psychology Wiki

Revision as of 01:47, 18 July 2006 (view source) Stenstro (Talk | contribs) ← Older edit		Revision as of 02:12, 18 July 2006 (view source) Stenstro (Talk | contribs) Newer edit →
Line 3:		Line 3:
	==Where should I start?==		==Where should I start?==
-	If you want to learn <u>what is a meta-analysis</u>...	+
-	#For the '''basics''' behind meta-analyses see below where we lay out	+	<TABLE BORDER=1 CELLSPACING=0 CELLPADDING=5>
-	::*the [[#Definition | definition]] of a meta-anlaysis,	+	<TR>
-	::*the [[#Three Basic Questions | three basic questions]] answered by a meta-analysis,	+	<TD valign=top width=370>
-	::*the [[#Five Basic Steps | the five steps]] involved in a meta-analysis,	+	If you want to learn '''what is a meta-analysis'''...
-	#For more '''in-depth''' discussion and explanations, we recommend...	+	<ol>
-	::start first with	+	<li>For the <u>basics</u>, see below where we lay out:</li>
		+	*the [[#Definition | definition]] of a meta-anlaysis,
		+	*the [[#Three Basic Questions | three basic questions]] answered by a meta,
		+	*the [[#Five Basic Steps | the five steps]] involved in a meta-analysis,
		+
		+
		+	<li>For more <u>in-depth</u> discussion and explanations, we recommend...</li>
		+	*start first with
		+	*
		+	*
		+	</ol>
		+	</TD>
		+	<TD valign=top width=370>
		+	If you want to learn '''how to start conducting a meta-analysis'''...
		+	<ol>
		+	<li>For the <u>basics</u>, see below were we lay out:</li>
		+	*how to [[#First, choose what statistical approach suits your needs | choose a statistical approach]]
		+	*how to [[#Second, choose which effect size to calculate? | choose which effect size to calculate]]
		+	*how to [[#Third, choose your statistical software | choose your statistical software]]
		+
		+
		+	<li>For more <u>in-depth</u> discussion and explanations, we recommend...</li>
		+	*start first with '''[[Johnson, Mullen, & Salas, 1995]]''' which provides a statistical comparision of the three major meta-analytic approaches using actual datasets, as well as the staistical forumulas for each approach and the methodological differences between each approach.
		+	*
		+	*
		+	</ol>
		+	</TD>
		+	</TR>
		+	</TABLE>
		+
-	::
-	::
-	If you want to learn <u> how to start conducting a meta-analysis.</u>..
-	:*For the '''basics''' behind conducting a meta-analysis see below were we lay out:
-	::how to [[#First, choose what statistical approach suits your needs | choose a statistical approach]] to conducting a meta-analysis,
-	::the [[
-	::the [[
-	:*For more '''in-depth''' discussion and explanations, we recommend...
-	::start first with '''[[Johnson, Mullen, & Salas, 1995]]''' which provides a statistical comparision of the three major meta-analytic approaches using actual datasets, as well as the staistical forumulas for each approach and the methodological differences between each approach.
-	::
-	::
Line 51:		Line 69:
	===Second, choose which effect size to calculate?===		===Second, choose which effect size to calculate?===
-	===Third, choose your variables===	+	===Third, choose your statistical software===
		+
	===If you want to follow the Hedges & Olkin approach...===		===If you want to follow the Hedges & Olkin approach...===

---

Revision as of 02:12, 18 July 2006

Have you ever wanted to learn about meta-analysis or conduct a meta-analysis but didn't know where to start? This webpage is devoted to providing

Contents 1 Where should I start? 2 What is a meta-analysis? 2.1 Definition 2.2 Three Basic Questions 2.3 Five Basic Steps 3 How do I conduct a meta-analysis? 3.1 First, choose what statistical approach suits your needs 3.2 Second, choose which effect size to calculate? 3.3 Third, choose your statistical software 3.4 If you want to follow the Hedges & Olkin approach... 3.5 If you want to follow the Rosenthal & Rubin approach... 3.6 If you want to follow the Hunter, Schmidt, & Jackson approach...

Where should I start?

If you want to learn what is a meta-analysis... For the basics, see below where we lay out: the definition of a meta-anlaysis, the three basic questions answered by a meta, the the five steps involved in a meta-analysis, For more in-depth discussion and explanations, we recommend... start first with

If you want to learn how to start conducting a meta-analysis... For the basics, see below were we lay out: how to choose a statistical approach how to choose which effect size to calculate how to choose your statistical software For more in-depth discussion and explanations, we recommend... start first with (Johnson, Mullen, & Salas, 1995) which provides a statistical comparision of the three major meta-analytic approaches using actual datasets, as well as the staistical forumulas for each approach and the methodological differences between each approach.

What is a meta-analysis?

Definition

Three Basic Questions

A meta-analysis answers three general questions:

1. Central tendency – The central purpose of a meta-analysis is to test the relationship between two variables such that X causes Y. Central tendency refers to identifying whether X affects Y via statistically summarizing signficance levels, effect sizes, and/or confidence intervals. You are trying to answer whether X affects Y, is the effect significant, and how strong is that effect?
2. Variability – There is always going to be some degree of variation between the outcomes of the individual studies that compose the meta-analysis. The question is whether the degree of variablity is signficantly different than what we would expect by chance alone. If so, then its called heterogeneity.
3. Prediction – If there is heterogeneity (variability), then we look for moderating variables that explain the variability. In other words, does the effect of X on Y differ with moderator variables?

Five Basic Steps

There are generally five separate steps in conducting a meta-analysis:

1. Define your hypothesis
2. Locate the Literature
3. Identify and input data
4. Cacluate Effect Sizes
5. Analyze Variables

How do I conduct a meta-analysis?

First, choose what statistical approach suits your needs

There are generally three different statistical approaches to conduct a meta-analysis so first you need to choose which approach best fits your needs. For an excellent detailed comparison of these three approaches, see (Johnson, Mullen, & Salas, 1995) (Comparison of Three Major Meta-Analytic Approaches. Journal of Applied Psychology, 80, 94-106). Some basic information from that article is posted below to get you started:

1. Hedges & Olkin Approach – see (Hedges, 1981); (Hedges, 1982); (Hedges & Olkin, 1985)
2. Rosenthal & Rubin Approach – see (Rosenthal, 1991); (Rosenthal & Rubin, 1978); (Rosenthal & Rubin, 1988)
3. Hunter, Schmidt, & Jackson - see (Hunter, Schmidt, & Jackson, 1982); (Hunter & Schmidt, 1990)

Second, choose which effect size to calculate?

Third, choose your statistical software

If you want to follow the Hedges & Olkin approach...

If you want to follow the Rosenthal & Rubin approach...

If you want to follow the Hunter, Schmidt, & Jackson approach...

Just as an individual study collects data from many individuals (data points) that is statistically summarized to answer the question of interest, a meta-anlay

• What is the good number of studies to have bare minimum for a meta-analysis? A meta-analysis with 10 studies have been published before but is not recommended.
• In a meta-analysis, don’t have raters code conditions for which no effect sizes can be calculated.
• In a meta-analysis, have judge rate each variable across studies, one moderator at a time, instead of rating all variables in a single study before moving on to next study.
• With meta-analysis coding with a high number of studies to code, such as 75+, can have some coders rate the entire set, but can also have some coders (undergrads) code only a subset as long there is overlap, so that more than 1 judge is rating each study.

---

► Back to Research Tools mainpage