Effect Size Calculators (Lee Becker)

1. Overview -
Eta squared, h ^{2} -
Partial Eta squared, h _{p}^{2} -
Omega squared, w ^{2} -
Intraclass correlation, r _{I}
3. References |

Measures of effect size in ANOVA are measures of the degree of association between and effect (e.g., a main effect, an interaction, a linear contrast) and the dependent variable. They can be thought of as the correlation between an effect and the dependent variable. If the value of the measure of association is squared it can be interpreted as the proportion of variance in the dependent variable that is attributable to each effect. Four of the commonly used measures of effect size in AVOVA are: Eta squared (h^{2}), partial Eta squared (h_{p}^{2}), omega squared (w^{2}), and the Intraclass correlation (r_{I}). Eta squared and partial Eta squared are estimates of the degree of association for the sample. Omega squared and the intraclass correlation are estimates of the degree of association in the population. SPSS for Windows 9.0 (and 8.0) displays the partial Eta squared when you check the display effect size option. This set of notes describes the similarities and differences between these measures of association.

The measures of association will be calculated for the study of the effects of drive and reward on performance in an oddity task that was used as the example in the notes for a 2-way ANOVA (GLM: 2-way). The analysis of variance table with the corresponding Eta squared scores for each effect is shown in Table 1.

Source |
Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Eta Squared |

Corrected Model |
280.000 |
5 |
56.000 |
3.055 |
.036 |
.459 |

Intercept |
2400.000 |
1 |
2400.000 |
130.909 |
.000 |
.879 |

DRIVE |
24.000 |
1 |
24.000 |
1.309 |
.268 |
.068 |

REWARD |
112.000 |
2 |
56.000 |
3.055 |
.072 |
.253 |

DRIVE * REWARD |
144.000 |
2 |
72.000 |
3.927 |
.038 |
.304 |

Error |
330.000 |
18 |
18.333 |
|||

Total |
3010.000 |
24 |
||||

Corrected Total |
610.000 |
23 |

Eta squared is the proportion of the total variance that is attributed to an effect. It is calculated as the ratio of the effect variance (SS_{effect}) to the total variance (SS_{total}) --

h^{2} = SS_{effect} / SS_{total}

The values used in the calculations for each h^{2} along with the h_{p}^{2} from the ANOVA output are shown in Table 2.

Effect | SS_{effect} | SS_{total}_{(Corrected Total)} | h^{2} | h_{p}^{2} |
---|---|---|---|---|

Drive | 24.000 | 610.000 | .039 | .068 |

Reward | 112.000 | 610.000 | .184 | .253 |

Reward * Drive | 144.000 | 610.000 | .236 | .304 |

The reward by drive interaction was significant in this analysis, *F*(2,18) = 3.927, *p* = .038. Using h^{2} as the measure of effect size, the interaction between drive and reward accounted for 24% of the total variability in the performance score. Using h_{p}^{2} as the measure of association, the interaction between drive and reward accounted for 30% of the total variability in the performance score.

If you decide to calculate h^{2} rather than use the values of h_{p}^{2} displayed by SPSS then will you need to be careful about selecting the SS_{total }to be used in the calculation of h^{2}. The values reported in the "Total" row of the ANOVA table include the following SS:

(a) the SS for each of the effects,

(b) the SS for the error term, and

(c) the SS for the intercept.

The value for SS_{total} in the h^{2} formula includes the SS for each of the effects and the error term, but it does not include the SS for the intercept. The values reported in the "Corrected Total" row of the ANOVA table include the SS for each of the effects and for the error term but they do not include the SS for the intercept. Thus, the SS for the "Corrected Total" row should be used in the calculation of h^{2}.

A pie chart can be used to graphically display proportion of total variance that is attributable to each effect, see Figure 1. The entire circle represents the (corrected) total sums of squares. Each slice of the pie is an effect or the SS for error. The percent of the pie represented by each slice is the effect size, h In a balanced design with equal |
Figure 1. Relative effect sizes (Eta squared) for the drive, reward, and the drive by reward interaction. |

**Statistical Issues:**

One of the problems with h^{2} is that the values for an effect are dependent upon the number of other other effects and the magnitude of those other effects. For example, if a third independent variable had been included in the design, then the effect size for the drive by reward interaction probably would have been smaller, even though the SS for the interaction might be the same. Similarly, if the SS for reward had been larger and there was no change in the SS for the interaction effect, then the interaction Eta squared would have been smaller. For that reason many people prefer an alternative computational procedure called the partial Eta squared. SPSS reports the partial Eta squared rather than Eta squared. Partial Eta squared is described in the next section. Some authors (e.g., Tabachnick & Fidell, 1989) call partial Eta squared an "alternative" computation of Eta squared.

**Graphics Note: **

Here are the steps used to create the pie chart in Figure 1.

1. Create an SPSS data file with two variables, effect, and SSeffect. The values entered into the data file are shown in Table 3. The values for effect include the three effects and the interaction. The values for SSeffect are the sums of squares for each effect.

Effect | SSeffect | |
---|---|---|

1 | drive | 24.00 |

2 | reward | 112.00 |

3 | drive*reward | 144.00 |

4 | error | 330.00 |

The SSeffect variable was used as a "weight cases" variable.

2. Select **Graphs** ->** Interactive** -> **Pie **-> **Simple**

Move the variable *effect* to the **Slice By:** window. Move the variable *Count[$count]* to the **Slice Summary:** window. Press OK to create the pie chart.

Some editing was then done to delete some extra information for each slide.

The partial Eta squared is the proportion of the the effect + error variance that is attributable to the effect. The formula differs from the Eta squared formula in that the denominator includes the SS_{effect} plus the SS_{error} rather than the SS_{total }--

h_{p}^{2} = SS_{effect} / (SS_{effect} + SS_{error})

The values from the ANOVA output that were used in the calculations for each h_{p}^{2} are shown in Table 4.

Effect | SS_{effect} | SS_{error} | SS_{effect} + SS_{error} | h_{p}^{2} |
---|---|---|---|---|

Drive | 24.000 | 330.000 | 354.000 | .068 |

Reward | 112.000 | 330.000 | 442.000 | .253 |

Reward * Drive | 144.000 | 330.000 | 474.000 | .304 |

Graphic representation of partial Eta square measures requires set of pie charts, one for each effect.

**Figure 2. Partial Eta squared values for the drive, reward, and drive by reward effects.**

**Statistical Issues:**

The sums of the partial Eta squared values are not additive. They do not sum to the amount of dependent variable variance accounted for by the independent variables. It is possible for the sums of the partial Eta squared values to be greater than 1.00. In general, Eta squared values describe the amount of variance accounted for in the sample. An estimate of the amount of variance accounted for in the population is omega squared.

**Graphics Note: **

Here are the steps used to create the pie chart in Figure 2.

1. Create an SPSS data file with six variables, drive, SSdrive, reward, SSreward, int (interaction), and SSint. The values entered into the data file are shown in Table 5. There are two values for drive, reward, and int: 1 = the name of the effect (drive, reward, or reward*drive) and 2 = error. The sums of squares for the effect and its error term are entered in the variables SSdrive, SSreward, and SSint.

DRIVE |
SSdrive |
REWARD |
SSreward |
INT |
SSint | |
---|---|---|---|---|---|---|

1 |
SSdrive |
24.00 |
SSreward |
112.00 |
SSdrive*reward |
144.00 |

2 |
SSerror |
330.00 |
SSerror |
330.00 |
SSerror |
330.00 |

2. Create one pie chart for each effect. First, weight cases by the SS for the effect (e.g., SSdrive). Select **Graphs** ->** Interactive** -> **Pie **-> **Simple**. Using the effect name (e.g., drive) as the **Slice by:** variable, and* Count[$count]* as the **Slice Summary:** variable. And create the pie chart. Repeat these steps for the reward pie chart and the drive by reward pie chart.

Omega squared is an estimate of the dependent variance accounted for by the independent variable in the population for a fixed effects model. The between-subjects, fixed effects, form of the w^{2} formula is --

w^{2} = (SS_{effect} - (df_{effect})(MS_{error})) / MS_{error} + SS_{total}

(Note: Do not use this formula for repeated measures designs)

The values from the ANOVA table that are used in the calculation of w^{2} are shown in Table 5.

Effect | SS_{effect} | df_{effect} | MS_{error} | SS_{total}_{(Corrected)} | SS_{effect}-(df_{effect})(MS_{error})--------------------------------- MS _{error} + SS_{total} | w^{2} |
---|---|---|---|---|---|---|

Drive | 24.000 | 1 | 18.333 | 610.000 | 5.667 / 628.333 | .001 |

Reward | 112.000 | 2 | 18.333 | 610.000 | 75.334 / 628.333 | .120 |

Reward * Drive | 144.000 | 2 | 18.333 | 610.000 | 107.334 / 628.333 | .171 |

Graphic representation of w^{2} would be done with separate pie charts, similar to the way h_{p}^{2} was presented above.

Because h^{2} and h_{p}^{2} are sample estimates and w^{2} is a population estimate, w^{2} is always going to be smaller than either h^{2} or h_{p}^{2}. The three measures of association are shown in Table 6.

Effect | h^{2} | h_{p}^{2} | w^{2} |
---|---|---|---|

Drive | .039 | .068 | .001 |

Reward | .184 | .253 | .120 |

Reward * Drive | .236 | .304 | .171 |

Formulas for a random effects model are also available, see Kirk, 1982.

The intraclass correlation an estimate of the degree of association between the independent variable and the dependent variable in the population for a random effects model. Because it is for a random effects model it is not commonly used in psychology experiments. The formula for r* _{I} *is

r* _{I} *= (MS

The model in the example used in this set of notes is a fixed effect model so the intraclass correlation will not be computed.

The square of the intraclass correlation is an estimate of the amount of dependent variable variance accounted for by the independent variable.

Kirk, R. E. (1982). *Experimental design: Procedures for the behavioral sciences* (2nd ed.). Belmont, CA: Brooks/Cole.

Tabachnick, B. G., & Fidell, L. S. (1989). *Using multivariate statistics* (2nd ed.). New York: Harper & Row.

�Lee A. Becker, 1998-1999 -11/08/99

© University of Colorado Colorado Springs

1420 Austin Bluffs Pkwy, Colorado Springs, CO USA 80918

719-255-8227 (UCCS), 800-990-8227

1420 Austin Bluffs Pkwy, Colorado Springs, CO USA 80918

719-255-8227 (UCCS), 800-990-8227