As someone who loves data, I often use stat speak in everyday situations. I also forget that most people have no idea what I’m talking about. It happens from time to time, even when explaining campaign results to clients. So I step back, take a deep breath, and explain exactly what it is I mean when I refer to medians, standard deviations and positive correlation.
This is my guide for some of the most common misunderstandings related to stat speak. I plan to mostly use this as a reference the next time one of them comes up in conversation. And now you can, too.
Trend Lines and the Scientific Method
Before we dive into the stat speak, here’s a primer on how we approach advertising in general. When problem solving, we try to be more scientific than some other marketing firms. Every client wants to know how a campaign will turn out before it begins. We use past experience to inform our decisions and theorize on how everything will play out.
The campaign itself will then serve as a test for our hypothesis, much like a science experiment. We can be entirely right. We can be completely wrong. We can fall in between. What matters is we’re open to change when presented with evidence that we need to shift course.
When a campaign is running, we look at daily running trends to give us some indication on what’s going on. Then, we run regressions to figure out where the actual trend line lies. If it’s positive, hooray we keep doing what we’re doing. If we need to make changes, we do. We aren’t beholden to the trend line, though, because there might be factors behind the scenes and beyond our control that are producing those results. And you shouldn’t live and die by its daily fluctuations, either.
Correlation vs Causation
Trend lines do more than inform our decisions on what to do next. They also help us figure out what’s happening, although they can’t always tell us why it’s happening. That’s the problem with correlation and causation.
To be clear, a correlation is when two variables appear to correspond to each other. A change in one produces a noticeable change in the other. Even so, it doesn’t follow that you can determine a cause and effect relationship between the two.
Correlation does not always mean causation. Just because two things have the same trend line doesn’t mean one causes the other, or even vice versa. In fact, there’s an entire website dedicated to correlations that mean absolutely nothing. Keep that in mind the next time you see two graphs superimposed on each other.
It’s so tempting to look at the numbers and attribute a cause to one thing, which is why you try and isolate variables to figure out their relationship while keeping everything else the same. But maybe the relationship works in the opposite direction. Or maybe it’s a coincidence. Or maybe there’s no relationship at all and you’re just inferring from the data what you want to infer.
Averages vs Median Numbers
Many people and news outlets use these stats interchangeably. That’s understandable since they do attempt to measure the same thing. However, there’s a fundamental difference between how they’re calculated, which impacts the accuracy of their assessment in any given situation.
An average is a sum total of figures divided by the number of figures being added. Add up six numbers and divide by six, for example. To take an average of 1 2 3 4 and 5, add up the numbers (15) and divide by 5, since there are five numbers you’re adding together. You get 3 as an average, which is good since it’s right in the middle of the number set. Simple, right?
What if those numbers were 1 2 3 4 and 50? Now your average is 12. Most of those numbers are way under 12. How can 12 be a representative average if most numbers are lower? That outliner really messes up the data.
This is when you might want to use a median number as a replacement for an average. As it is when you’re driving on a road, the median is the thing in the middle. In this case, the median is 3 since it’s the middle number when you line everything up numerically. It’s also closer to the spiritual average when you remove that really high data point.
Whenever you’re quoted a crazy average that sounds like it doesn’t make sense intuitively, remember this difference. The median might be a more accurate representation of where the actual midpoint is, and might be much closer to the center of the standard deviation.
Rate Stats vs Counting Stats
There’s a subtle difference between these two things. One counts physical numbers of things happening, one is a percentage or ratio in relation to another number, derived from a simple formula. While you want to optimize for the best rates, if they drop while you experience some massive growth, then how do you evaluate your success?
Here’s an example. A client of ours was recently concerned with his dropping email list open rate. What was once 20% had now fallen to 10%. However, in the same time, that email list quadrupled in size. That means that the number of people opening each email doubled, which is good. Even though it looks bad. Make sense?
Rate stats are always going to be dependent on your counting stats. Fluctuations are expected. Sometimes, one influences the other in unanticipated ways. This is why we look at both types of numbers to draw conclusions about what’s actually happening in a digital campaign. Knowing the difference with this stat speak can save you a lot of headaches when it comes to your digital strategy.
Embrace Stat Speak to Understand Your Business
Much like sabermetrics is a new way to talk about baseball, stat speak like this is just a different way to conceptualize what happens in a business. Don’t be afraid of the numbers. Invite them in so they can paint a vivid picture.
Many people still believe that statistics are damn lies. We don’t. There’s significant truth to be found when you know how to read numbers. Now that I’ve had a chance to explain some stat speak, remember what the numbers mean the next time you’re presented with a misleading figure or erroneous correlation. It’s not as hard to decipher the signal from the noise as you think.