1. Determine your top key performance indicators (KPIs)

In order to understand whether your automations are doing what you need them to do, you first need to know what you’re looking for.

We review the data, and the data speaks to us.

Sean Donahue

Director of email marketing, Power Digital

“We review the data—the results and the benchmarks we have available within Klaviyo—and the data speaks to us,” Donahue says. “We’re then able to use that data to figure out where to pivot.”

What constitutes the “right KPIs” “depends on the business, the flow, and the goal of the flow,” says Chris Gordon, head of client success at leading ecommerce and marketing agency Noticed.

Overall, though, experts recommend marketers pay attention to the following important metrics when evaluating whether their marketing automations are working:

• Open rate
• Click-through rate
• Conversion rate
• ROI
• Deliverability
• Bounce rate
• Unsubscribes
• Customer lifetime value (CLV)

While “conversions are always top of mind,” Gordon says, he actually pays more attention to clicks.

“In the school of thought I come from, we tend to think that the job of the email is to get the potential customer onto the site, ready to buy, and then it’s the site’s job to convert them,” he explains. “So the email should be trying to drive just as much website traffic as humanly possible, and then hopefully the site converts them from there.”

It’s Time To Rethink Email Marketing Metrics. Here’s How To Measure What Matters Now

Learn about the key email marketing metrics you should track along with actionable steps on how to improve each one.

Read full story

If your automations are earning big-time clicks and a high number of visitors but your subscribers are not converting, for example, “maybe you need to look at your homepage or the landing page you’re driving traffic to,” Gordon suggests. “Look at core KPIs across your overarching data, even outside of email.”

Visit our help centre to learn how to dig deeper into your marketing automation analytics in Klaviyo.

2. A/B test your automations

By using your core KPIs as a roadmap, you’ll be able to “figure out what’s working, what’s not, and then do some testing around that,” Donahue says. “There are all these different ways we can pivot the messages.”

In Klaviyo, you can set up both automatic and manual A/B testing to compare variables like:

• Message frequency
• Send time
• Time delays between messages
• Subject lines
• Copy
• Images
• …and more

“We always want to increase, or at least maintain, our engaged customers, so A/B testing is a non-intrusive way to see if something is working or if we need to make adjustments,” explains Heather Browne, director of performance marketing at leading digital customer experience company Blue Acorn iCi.

“We look to gain statistical significance in these tests in order to determine if we change things up now, or keep testing,” Browne adds.

“After we get everything launched, it’s an iterative process where we’re continuing to review how things are performing and then taking an educated guess and saying, ‘OK, we think if we tweak this specific thing, that it’s going to move this specific needle,’” Donahue agrees. “And then we make that change, see what happens, and just continue that process over and over again.”

How often should marketers modify their existing automations? “It depends on volume and cadence,” Donahue explains. “We’re going to get much better and quicker response data with flows that are going out on a more consistent and regular basis.”

We make that change, see what happens, and just continue that process over and over again.

Sean Donahue

Director of email marketing, Power Digital

It’s “much easier for us to make data decisions within a month” on an automation that sends multiple times a week, for example, than on a welcome series where sends are 14 days apart, Donahue points out. “The data coming in is going to be much slower, in that case, so it’s also going to be much slower for us to figure out what changes we need to make.”

Visit our help centre to learn how to A/B test a flow in Klaviyo.

3. Analyse individual steps in an automation, not just the automation as a whole

Overall flow performance is, of course, important to track. “You want to understand what’s working for you,” Gordon says. “If your welcome series is a big part of what drives revenue for you, you’ll want to prioritise that. If your abandoned cart is a big revenue driver for you, prioritise that.”

Look at your data, figure out what it’s telling you from a very high level, and then, from there, build your roadmap.

Chris Gordon

Head of client success, Noticed

On the other hand, if your welcome series is underperforming, for example, “maybe you’re not explaining your brand enough, or maybe you’re not providing enough social proof to show the customers who have tried this product really love it,” Gordon suggests. “Look at your data, figure out what it’s telling you from a very high level, and then, from there, build your roadmap.”

But getting a bit more granular with your data can reveal equally important insights, points out Lindsey Murray, VP of performance marketing at Blue Acorn iCi—and also help you make sure you’re respecting the inboxes of both paying customers and new customers.

“Because people are typically more engaged early on in a flow, we have checks built into our lengthier flows. If someone has not opened or clicked within the last 3 messages, for example, we might drop them out of the flow,” Murray explains. “We don’t want to continue sending someone emails if they’re clearly uninterested.”

We like to look at the overall flow and just see how it’s performing, but it doesn’t really tell us the whole story.

Sean Donahue

Director of email marketing, Power Digital

“We like to look at the overall flow and just see how it’s performing, but it doesn’t really tell us the whole story,” Donahue agrees. “You could have a first message that’s just crushing it, driving all of the revenue, but then everything else just drifts off. It’s important to look at every single one of those steps and figure out what’s going on.”

4. Make sure you don’t “creep anybody out”

The biggest challenge with marketing automation, experts agree, is finding the right balance—in other words, “boundaries,” Murray says.

“We know how beneficial all these automations are, but you don’t want to overwhelm your customer, either,” Browne points out. “This comes back to the fact that you need to know your customer—whether they’re email or SMS people, how often they like to be talked to, what’s too much, etc.”

We know how beneficial all these automations are, but you don’t want to overwhelm your customer, either.

Heather Browne

Director of performance marketing, Blue Acorn iCi

What’s too much—and also what’s alarming. As Donahue points out, we’ve all had that moment after having a casual conversation when we open our phones, start scrolling, and immediately see an advertisement for the exact thing we were just talking about.

“We don’t want to reflect that in email marketing,” Donahue emphasises. “We don’t want to creep anybody out. So how do we do all of this, and make it a really personalised experience, without coming across as Big Brother?”

As with so many things in marketing automation, the answer lies in the data, both real-time and historical. “If we see an increase in unsubscribe rates, or a decrease in engagement overall, it would be time to reassess the overall customer journey strategy and the quantity of marketing campaigns and flows,” Browne explains.

If we’re getting too close to the sun with personalisation, that is absolutely going to reflect in the data and we’ll know to back off.

Sean Donahue

Director of email marketing, Power Digital

“Engagement tells us everything we need to know,” Donahue agrees. “If we’re getting too close to the sun with personalisation, that is absolutely going to reflect in the data and we’ll know to back off.”

If you’re struggling to strike the right balance, remember that the ultimate goal of marketing automation, Donahue says, is to “give the customer value.”

“Be helpful,” Donahue says. “Be there for the customer when they need you, and not when they don’t. Marketing automation is about finding those opportunities—making sure you’re in the right place at the right time, and not where you shouldn’t be.”

Be helpful. Marketing automation is about finding those opportunities—making sure you’re in the right place at the right time, and not where you shouldn’t be.

Sean Donahue

Director of email marketing, Power Digital

Want to go beyond marketing automation KPIs? Check out these 10 email marketing automation examples to help you make money while you sleep.

Marketing automation KPIs FAQs

What are the most important marketing automation metrics?

The most important marketing metrics for automation depend on the business, but experts recommend marketers pay attention to engagement metrics like open rate, click rate, conversion rate, ROI, deliverability, and unsubscribes when evaluating whether their marketing automations are working.

How to test the success of the marketing automation process?

In a platform like Klaviyo, you can set up both automatic and manual A/B testing to compare automation variables like message frequency, send time, time delays between messages, subject lines, copy, images, and more. Look to gain statistical significance in these tests in order to determine if you should change things up now, or keep testing and optimising.

What are the best tools for measuring the success of marketing automation?

The best tool for measuring the success of marketing automation is one that provides automated A/B testing as well as marketer-friendly custom reporting and benchmarks that allow you to compare your performance to others in your industry or vertical. A solution like Klaviyo also empowers you to analyse individual steps in an automation, not just the automation as a whole, so that you can get more granular with your optimisations.