Section 6.1: Between Versus Within Group Analyses

As discussed before, examining and understanding how groups of individuals can differ is one of the key goals of psychology. There are many different groups of people that make up society, each with their own biology, environment, and values. It makes sense that many of these groups would vary on different psychological constructs and behaviour.  In psychology, examining these differences can be key to understanding differing mental and social processes. This is important when considering if treatments for mental health concerns actually work, or if there are long-term trends in behaviours across ages and groups.

There are two main types of difference tests: those that look at differences between groups, and those that look at differences within groups. Between groups differences examine how two groups can differ from each other across a variable.  Within-groups differences are similarly important. You’re not looking at the differences between two groups, but rather the differences between the same group taken at two time points.  Remember that when we talk about differences, we are talking about the difference between the mean of one group/time point and the mean of another.  To examine the differences between or within groups you also need to know the standard deviations of both means you are comparing, as well as the number of participants. With the mean, the measure of variance within the samples (standard deviation, and the number of data points or participants, you can use the t-test to calculate if the difference between the two means is statistically significant.

One-Factor ANOVA

You may have seen in journal articles, reference to an ANOVA and asked yourself what is an ANOVA test, and how does it differ from the t-tests discussed in previous chapters? Historically, ANOVA is an analysis for experimental design (like t-tests), that aims to determine the influence of the effect of one Independent Variable (IV) on the Dependent Variable (DV). The main aim of an ANOVA is to evaluate if the means of the samples are sufficiently different from each other to suggest they are representative of different populations. The main difference between an ANOVA and a t-test, is that you can use a one-way ANOVA to test if there are differences in more than two groups. An example would be if you wanted to check to see if there were differences in class attendance rates for first, second, and third-year university students in a particular course.

The ANOVA is a two-stage test.

Stage 1:

The first stage is an overall or omnibus test. This stage tests the overall hypothesis. In this example, you would be looking to test whether attendance rates differ across first, second, and third-year university students. If a significant result is found, that would indicate that rates do differ, and a second stage is required.

Stage 2:

Stage two introduces specific procedures for testing group differences. In this example, if a main effect is found, how do you know which year level differs from which year level? A main effect can be described as the effect of an independent variable on a dependent variable averaged across the levels of any other independent variables. For example, do first-year university students attend class at a higher rate than second or third years university students, and do second and third-year university students also differ?

Ts and Fs

The t-test is an examination of the mean difference and the standard error of the mean difference, and only examines the difference between two groups relative to the spread of their scores.

The F-test examines the variability of sample means (explained variance) to an estimate of error variance (unexplained variance). This can tell you if there is a difference between multiple groups, though the test can only tell if there is a difference, not tell which groups are different when there are more than 2 groups.

Planned contrasts or Posthoc Tests

Planned contrasts/comparisons are performed by a researcher who has a specific hypothesis in mind. Going back to the class attendance example, if you hypothesised that the first-year students would have a higher class attendance rate than second or third students, with no hypotheses relating to comparing second and third-year students, you would plan to compare or contrast first-year class attendance with second and then third-year class attendance. It would involve breaking down the variance accounted for by the model into component parts.

Posthoc tests are used when a researcher is expecting to find a difference, but isn’t sure in which group that difference will be found. Posthoc tests aim to compare every group (similar to carrying out several t-tests) but familywise error is controlled (depending on the choice of posthoc procedures). Familywise error rate (FWER) refers to the probability of making at least one Type I error within the family of tests under consideration. Another alternative is to use t-tests, but you need to be aware that can inflate the chances of a type 1 error.

Data Screening

The usual data screening methods are applied including having no out-of-range responses; correct coding; and missing data has been dealt with. Being aware of outliers is important for ANOVAs but, assuming there are no major outliers, the F model is still interpretable with data showing minor violations of normality  (Tabachnick & Fidell).