[Joanne Cotterell:] Testing is a really important tool for us. It tells us what works for our audience, what they like and what they don't, and also all small gains add up and helps us meet our tough targets.
[Danielle Woolley:] That's a really good point, because testing is something that you really need to stick at in order to see results. Let's have a look at what you've been testing.
[Joanne Cotterell:] We always have a testing plan, which is basically an Excel spreadsheet, and this helps us plan up front what we want to test on a weekly basis, and also gives us some rules to stick to. It also allows us to track what's worked and what hasn't, so we have a record to go back on. When we test, we only test one thing at a time.
We've completed a bit of template testing, so now we're focusing on subject line testing. You can see some of the versions we've been trying: we've been looking at putting numbers at the beginning of subject lines to make it seem more list-based, going for shorter call-to-actions such as "open me for cake," and also the use of emojis, whether it's a cake image or a knife and fork. Also, with our split tests, we initially send the two versions to 60% of our list, and then 2 hours later we send the most successful variant to the rest of the database.
These two examples will show you how we test numbers versus the clearer action - "open me" - in both of these instances, we've seen that the numbers at the beginning of the subject line result in 2% more opens than the ones without. And we've also been having fun testing with emojis. So, in this instance, we've created a chocolate cake recipe page which we've highlighted with the use of cake emojis. While we love using them, we only use them when they make sense - we don't want to force them or overuse them. In this instance, the emoji subject line won with around 2% more opens.
[Danielle Woolley:] So emojis can work then! So as well as subject line testing, have you done any other kind of testing?
[Joanne Cotterell:] Yeah, let me walk you through how we tested our templates. So, our original template was built with a row of three quite small thumbnails across the bottom. We didn't see much interaction with these, so we A/B tested removing this row and including the videos that were in this row in the main body of the newsletter template. To begin with, the small thumbnails were getting between 30-70 clicks, but we found that by moving them into the body and creating a new row of content, we found that they were getting over a 100 clicks per video, which was over double the engagement.
[Danielle Woolley:] I love to work with a client who really uses testing and tests consistently, and gets great results because of that. So, to wrap up, I've just pulled out some top testing tips. Obviously ,start with a hypothesis - if you have a clear objective at the start, you can better understand the impact of your tests. Also, be clear what metric you want your test to impact, so Jo was looking at increase open rates, so she tested subject lines. You might also want to change pre-header or from name in order to influence open rates, whereas if you want to improve your click-through rate, then you need to look at the content of your email.
Test one thing at a time - be sure to send both versions simultaneously (unless you're testing send times), as changing this could add another variable. Make sure that your sample size is statistically significant, and as you do have a testing plan, keep track of your results. And finally, don't stop testing, because audiences change, and you need to repeat your testing after time has passed to see if anything has changed. A year may be a good interval to wait, but this depends on your audience and your frequency of messaging.
