Control Group

Definition

A control group is a group of users in a test that do not experience the tested variable, and therefore help to create a comparison data set to measure results. If you want to start working with a new channel, or approaching users in a new way, your control group would continue to receive normal marketing, not the new channel or approach. If you see an increase in conversions that’s statistically significant for the tested group, compared to the control group, you can assume that your new approach was effective. However, if there is no significant difference between conversions for the two groups, you may want to try a new approach, because you can’t conclude that the new tactic made a difference.

It’s important that control groups are made up of similar demographics as the rest of your users (so that you don’t accidentally have mostly women in your control group, for example, which adds another variable to the mix that you’re not trying to test). You should also set a hypothesis before you start and focus on a single variable to test, to make the results as easy to understand as possible.

Usage

Control groups are pretty well understood by those with some experience with experiments and the social sciences. Just like in other experiments, control groups help marketers understand the effect of a variable. While most people understand it, not all apply it in their testing, so it’s an important component to discuss when setting up a test.