library(pwr)
library(pwr2)
library(powertools)
library(Superpower)3 ANOVA
We will calculate power and sample size for a one-way ANOVA, a two-way ANOVA, and an ANCOVA. We will use base R, as well as the pwr, pwr2, powertools and Superpower packages.
3.1 One-way ANOVA
We use a one-way ANOVA to calculate power and sample size when we want to get the difference in means across groups that are from a single factor.
For power estimation based on sample size we will use:
- Number of groups and their means (or the group effects)
- Group sample sizes
- Population standard deviation
Imagine we have a book club with three groups that each use a different medium to interact with the book. Group A reads the book, group b listens to the audiobook, and group c watches the movie. We want to know how well each group understood the story line. Each group has 20 people, the expected population group means are 8, 5, and 12 and the populated standard deviation is expected to be 7.
We will calculate the power with \(\alpha\) = 0.05.
Using base R:
power.anova.test(groups = 3, # 3 groups
between.var = 13,# between group variance
within.var = 49, # within group variance
sig.level=0.05, # sig level
n = 20) # no. people in each group
Balanced one-way analysis of variance power calculation
groups = 3
n = 20
between.var = 13
within.var = 49
sig.level = 0.05
power = 0.8180084
NOTE: n is number in each groupUsing the package powertools, we call anova1way.F.bal() to calculate the overall sample size for balanced groups (each group has 20 people).
anova1way.F.bal(n = 20, mvec = c(8, 5, 12), sd = 7, power = NULL) |>
round(2)[1] 0.8We can also calculate sample size with the pwr package. Here, we need to provide the effect size which is specified by the argument f.
Cohen’s f is an effect size measure used in tests that evaluate significant differences among group means, such as one or two way ANOVA’s. Cohen’s f effect size defines defines effect sizes as: f = 0.1 as small, f = 0.25 as medium, and f = 0.4 as large, effect sizes (Cohen, 1988).
We will first calculate Cohen’s f by hand:
# Calculate Cohen's f
book.means <- c(8, 5, 12)
sd <- 7
k <- length(book.means)
mu <- mean(book.means)
# Calculate between group var
between.var.pop <- sum((book.means - mu)^2) / k
within.var <- sd^2 # within group var
f <- sqrt(between.var.pop / within.var)
f[1] 0.4096345Now, we use the function es.anova.f() from the package powertools to calculate f and provide the group means and standard deviation. We get the same result as above.
# Use powertools to get the effect size
f.pt <- es.anova.f(means = book.means, sd = sd, v = F)
f.pt[1] 0.4096345We now calculate power using the function pwr.anova.test() with our f and f.pt.
# Power analysis using f from manual calculation
pwr.anova.test(k = 3, f = f, n = 20, sig.level = 0.05)
Balanced one-way analysis of variance power calculation
k = 3
n = 20
f = 0.4096345
sig.level = 0.05
power = 0.796186
NOTE: n is number in each group# Power analysis using f calculated by powertools
pwr.anova.test(k = 3, f = f.pt, n = 20, sig.level = 0.05)
Balanced one-way analysis of variance power calculation
k = 3
n = 20
f = 0.4096345
sig.level = 0.05
power = 0.796186
NOTE: n is number in each groupFor base R, powertools, and pwr, we see that our power estimate is 0.8 telling us that for the given parameters we specified, we have an 80% chance of correctly rejecting the null hypothesis
Now, we will calculate sample size for a one-way ANOVA when we know the power we want to achieve, but we need to determine the sample size.
What you need:
- Number of groups
- Group means or effect size
- Significance level
- Power
Using base R, we leave n (sample size) empty and input the power we would like to achieve. We will include a power of 0.8, again indicating that if a difference exists between the mean and target, we have an 80% chance of detecting it.
power.anova.test(groups = 3,
between.var = 13,
within.var = 49,
sig.level = 0.05,
power = 0.8)
Balanced one-way analysis of variance power calculation
groups = 3
n = 19.19235
between.var = 13
within.var = 49
sig.level = 0.05
power = 0.8
NOTE: n is number in each groupUsing powertools:
anova1way.F.bal(n = NULL, mvec = c(8, 5, 12), sd = 7, power = 0.8)[1] 20.17208Our estimated sample size is 20 people per group.
Using the pwr package and inputting an effect size of 0.2 instead of the estimated mean values:
# Get f using powertools
f.2way <- es.anova.f(means = c(8, 5, 12), sd = 7, v = F)
pwr.anova.test(k = 3, # groups
f = f.2way, # effect size
sig.level = 0.05, power = 0.8)
Balanced one-way analysis of variance power calculation
k = 3
n = 20.17208
f = 0.4096345
sig.level = 0.05
power = 0.8
NOTE: n is number in each groupAgain, we see that our estimated sample size is 20.
3.2 Two-way ANOVA
We will run examples of power and sample size for a two-way ANOVA, which is used when you want to compare the difference in means across two factors.
What you need
- Number of factor levels
- Group means and standard deviation
- Significance level
- Target power
In our example, we want to evaluate how fertilizer and soil type influence plant growth. We have two types of fertilizer and two types of soil. Our dependent variable is plant growth.
Before setting up the full study, we wanted to make sure it is sufficiently powered, so we ran a pilot study and found the mean values. We want to calculate statistical power of 0.9 at a 95% confidence level.
# Generate matrix of mean plant values
plant.mat <- matrix(c(15.2, 19.1, 11.9, 13.3), nrow = 2, byrow = TRUE)
# Print
plant.mat [,1] [,2]
[1,] 15.2 19.1
[2,] 11.9 13.3Using the powertools package:
anova2way.F.bal(n = NULL, mmatrix = plant.mat, sd = 3, alpha = 0.05,
power = 0.9, v = F)[1] "NOTE: The 3rd value for f and power or n is for the interaction" nA nB nAB
5.1626 13.9835 61.0110 We see the sample sizes required to detect the main effect of Factor A, B, and their interaction.
Using the pwr2 package, we can calculate the power estimate. First, need to calculate Cohen’s f effect sizes. We can get these values for a balanced two-way ANOVA using the powertools package and inputting the plant matrix and the estimated standard deviation:
# Get Cohen's f effect sizes using powertools
cd.anova2way <- es.anova.f(means = plant.mat, sd = 3, v = FALSE)
print(cd.anova2way) fA fB fAB
0.7583333 0.4416667 0.2083333 The effect sizes for factors A, B, and their interaction(fAB) are returned. We will input fA and fB into our sample size calculation
pwr.2way(a = 2, b = 2, # n factors
alpha = 0.05, # significance level
size.A = 10, size.B = 10, # sample size
f.A = cd.anova2way["fA"], f.B = cd.anova2way["fB"]) # effect size
Balanced two-way analysis of variance power calculation
a = 2
b = 2
n.A = 10
n.B = 10
sig.level = 0.05
power.A = 0.9966198
power.B = 0.776444
power = 0.776444
NOTE: power is the minimum power among two factorsThe output shows that there is a high chance of detecting an effect for Factor A (power.A ~ 0.99), and a relatively moderate chance of detecting an effect for Factor B (power.B ~ .78). Overall, the design is well powered for detecting effects of Factor A but less so for Factor B. The power = 0.78 result is the lowest power returned among the factors tested.
Using the pwr package, we can calculate the estimated sample size for a two-way ANOVA
ss.2way(a = 2, b = 2, #2 groups in factors A and B
alpha = 0.05, #significance level
beta = 0.1, #Using 90% power
f.A = 0.2, f.B = 0.3, #Effect sizes for factors A and B
B = 100) #Number of iterations
Balanced two-way analysis of variance sample size adjustment
a = 2
b = 2
sig.level = 0.05
power = 0.9
n = 67
NOTE: n is number in each group, total sample = 268With the input parameters, our sample size per group is 67, or 268 total.
3.3 ANCOVA
With an ANCOVA, we extend our one-way ANOVA by including a variable that we want to account for in the model.
What we need:
- Group means and standard deviation
- Estimated \(R^2\)
- Significance level
- Power
When looking at story comprehension based on the medium (book, audiobook, or movie), we want to account for age. We will use the function anova1way.F.bal() from the powertools package and input our mean values, standard deviation, \(R^2\), the number of covariates, and the power.
anova1way.F.bal(n = NULL,
mvec = c(8, 5, 12),
sd = 7,
Rsq = 0.6^2, #estimated r^2
ncov = 1, #n covariates
power = 0.8)[1] 13.32866Using the Superpower package
power_oneway_ancova(n = NULL,
mu = c(8, 5, 12),
sd = 7,
n_cov = 1,
r2 = 0.6^2,
beta_level = 0.8,
alpha_level = 0.05)
Power Calculation for 1-way ANCOVA
dfs = 2, 8
N = 12
n = 4, 4, 4
n_cov = 1
mu = 8, 5, 12
sd = 7
r2 = 0.36
alpha_level = 0.05
beta_level = 0.7780806
power = 22.19194
type = exactSimilarly, we could compute the power by leaving the argument beta_level = NULL and inputting the estimated sample sizes per group:
power_oneway_ancova(n = c(15, 15, 15),
mu = c(25, 25, 32),
sd = 7,
n_cov = 1,
r2 = 0.4^2,
beta_level = NULL,
alpha_level = 0.05)
Power Calculation for 1-way ANCOVA
dfs = 2, 41
N = 45
n = 15, 15, 15
n_cov = 1
mu = 25, 25, 32
sd = 7
r2 = 0.16
alpha_level = 0.05
beta_level = 0.1548218
power = 84.51782
type = exactReferences
- Cohen J. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 2nd edition, 1988.