Introduction

In 2010, The Economist noted that the amount of digital information created increases tenfold every five years. Consider how much digital content your organization or brand creates every month. Now think about how much digital information is created about your brand or organization by your customers, audiences, staff, partners, fans, etc. Has this increased as new ‘social’ channels develop, younger consumers migrate to new platforms and older consumers adapt and adopt more ‘social’ habits?

Suddenly, it doesn’t seem like a bold prediction by The Economist. In fact, they may have even underestimated the sheer amount of digital data that we’ll see in five years time. With so many brand and keyword mentions, so many hashtags, posts, tweets, @replies, comments, likes, favorites, photos, videos and audio, it’s certainly getting harder to evaluate whether a marketing, communications and digital activity is genuinely ‘moving the dial’ and increasing sales, decreasing costs and improving brand reputation.

It’s also difficult to assess which element of paid, owned and earned campaign activity is the most effective. But, with a mix of clear objectives and the right tools, the sheer volume and complexity of data becomes less of an issue. Divided into two sections, this guide is written to overcome the potential ‘Option Paralysis*’ when it comes to measurement, to show how it is possible to separate the ‘signal’ from the ‘noise’ and to provide a number of potential templates for effective evaluation of campaign activity.

Section 1 distinguishes between three distinct measurement types to help you identify the most appropriate metrics for your campaign:

• Outputs
• Outtakes
• Outcomes

Section 2 provides a more in-depth view on campaign measurement frameworks and dashboards based on the most common campaign objectives.

You’ll see the requirement to tie campaign objectives closely to organizational objectives and the importance of assessing campaign ‘outcomes’ that affect the business. The days of generating online coverage and social media likes for likes’ sake are well behind us. The clearer the objectives for any activity, the more effective and useful the evaluation will be.

Section 1: Measurement - The How And Why

Why Measure?

On the face of it, asking ‘why measure’ seems an incredibly naïve question. The obvious answer is ‘to prove success’. But all too often, ‘success’ is not clearly defined from the outset - making it very hard to prove.

If different people within the same organization, even within the same team, define ‘success’ in different ways, then someone is always going to be disappointed with the results. Adopting and sharing a clear definition of success can help structure what should be measured, even if that definition is quite complicated. In the very early years of social media marketing, some definitions of success focused explicitly on boosting the number of followers/likes for a brand, mainly because organizations wanted to compare themselves favorably to their competitors and it was a simple-to-prove campaign metric.

One of the methods to boost followers and fans was to run competitions and sweepstakes, where a condition of entry was that people had to ‘like’ or ‘follow’ the brand to be in with a chance of winning.

This activity was often extremely ‘successful’ in increasing numbers of ‘likes’ or followers. However, few of the new followers/likers converted to customers or wanted an ongoing relationship with the brand. The long-term benefits of this type of campaign were hard to prove.

It became apparent that there were much broader ways of defining success, which matched up to the organization’s wider business objectives. With so many metrics available to measure, what are the right things to measure and report on?

What Should You Measure?

If you’re trying to work out what to measure and report on, without first asking ‘what do we want to achieve?’, then you should probably slow down. The starting point for any measurement framework is to establish clear objectives and then to work out what data will help demonstrate when/whether you’ve achieved them.

Defining clear objectives is at the heart of a good measurement and evaluation program, partly because it helps focus the whole organization on what is, and what isn’t, useful and relevant. Then agreed and transparent results can be used to justify the associated budgets. Because different organizations now use digital channels for very different things, including customer service, sales generation, stakeholder management, relationship management, recruitment, communications and marketing, the criteria to measure will be different as well.

It’s impossible to create a ‘one size fits all’ evaluation dashboard that will suit everyone. But, it is possible to create a framework around which anyone can measure and then evaluate pretty much any campaign. There are many management theories about how best to create clear objectives, but one of the most useful is to insist upon SMART objectives, where the desired outcomes are:

• Specific
• Measurable
• Achievable
• Realistic
• Time-related

Using this SMART Framework makes it much easier to translate general campaign activities, which clearly contribute to the organization’s objectives. For example:

• Improving customer service satisfaction
• Promoting specific deals and driving traffic to a specific website landing page
• Managing relationships with journalists, politicians and trade bodies online
• Maintaining strong brand awareness against a competitor
• Researching customers’ views on a potential new product launch
• Building and growing a community who will talk about us among our target audience

We’ll look at a few common objectives for campaigns and the appropriate SMART objectives for them in Section 2 of this paper.

The Three Campaign Measurement Categories – Outputs, Outtakes And Outcomes

There is such a thing as too much information. As we mentioned in the introduction, the sheer volume of data that can be tracked can seem daunting. The best measurement frameworks get to the heart of what are the best sets of data to collect to indicate whether a particular objective has been met or not. That data tends to fall within three clear categories:

• Outputs: Measures of what the organization has done or created
• Outtakes: How people have reacted to an organization’s activity
• Outcomes: How people behave differently as a result of that activity

For most organizations, Outcomes will be the most important campaign measurements. They are generally the most scrutinized Management Information. Example Outcome metrics include:

• How many people purchased something
• How many people and how much was donated to a good cause
• How many people pledged to change their behavior (e.g., to quit smoking or to microchip their pet)
• How many people attended an event

Before we look at a framework for measuring Outcomes, we should first consider how to measure Outputs (which tend to be an indication of how a campaign is performing) and Outtakes (which tend to indicate how campaigns are performing over time).

Measuring Outputs

A campaign’s Outputs are defined as everything that the campaign team is 100% responsible for. These measures further subdivide into operational, content, and response. They are all typically benchmarked and reported on daily/weekly/monthly/quarterly.

Different organizations will obviously set different benchmarks for these outputs, depending on what is most relevant to their business. Here are some examples:

Operational

• Hours of operation/response
• Number of social media posts per day
• Time of day/frequency of posting

Content

• % of social media posts with video or audio attached
• % of posts that are created vs curated
• % of posts that contain one or more key messages

Response

• % of social media posts responded to
• % of questions responded to
• Speed of response (fastest/slowest/average) to social media comments
• % of responses that solve an issue the first time

Some of these measures can be presented from within the template dashboards of the major campaign monitoring tools. But others will need to be downloaded into a regularly updated spreadsheet to enable a read-across over time.

Measuring Outtakes

Outtakes refer to how the world outside the organization reacts to the campaign activity—in short, how the organization’s campaign activity is being perceived. Most of the leading measurement tools have specific features to help present these measures. Outtake measurements might include:

• Sentiment associated with the organization or keywords that the organization is tracking
• Share of the conversation in which the organization or tracked keywords feature
• Increases in an ‘influencer’ scoring metric
• Reach of particular posts, including number of shares
• Numbers of followers/likers/subscribers

Of course each of these might be further tracked – to look at changes over time, or to look at their impact in a particular country, among a particular demographic or key audience. Again, many of these measures can be presented from within the template dashboards of the major social media monitoring tools. Others will need to be downloaded into a regularly updated spreadsheet.

Measuring Outcomes

Outcome measurements illustrate what people think, say or do as a result of campaign activity. Organizations focus on the outcomes of campaign activity as they can illustrate the impact of campaign activity on the organization’s core purpose….and its bottom line. When measured appropriately, they can demonstrate:

• Campaign activity improving branding/reputation
• Campaign activity generating sales or behavior change
• Campaign activity reducing costs

Outcomes are typically divided into two further categories for measurement:

• ‘Communications/Branding Outcomes’ - to illustrate how campaigns perform over time and the impact on the brand
• ‘Business Outcomes’ – to illustrate the financial impacts of campaigns for the whole business

In practice, the two categories are clearly very tightly interwoven. We’ll now look at both aspects in detail.

Communications/branding outcomes

Organizations routinely look for measures that demonstrate ‘engagement’ with their campaigns, yet there is no universally agreed definition of what ‘engagement’ is. But there is a broad recognition that all forms of media offer a sliding scale of ‘engagement’, which can be measured and weighted as appropriate if required.

Different organizations might capture different aspects of engagement – e.g., number of words written as a comment, amount of time spent watching a video, power of sentiment expressed.

Organizations can choose to define broad distinctions between strong and minimal engagement in all campaign channels and will apply different weighting to each.

For example, organizations that have been at the center of activist campaigning, such as Nestlé, Coca-Cola, or Royal Dutch Shell, may be more sensitive about the number of negative comments than an organization that thrives by polarizing opinion, such as Paddy Power or Marmite

A media organization wishing to show advertisers the time engaged with their media brand would upweight the ‘total time spent’. Example Social Media campaign Outcomes demonstrating strong engagement:

• Number of positive comments/conversations or replies per post (or per post per follower) across any channel
• Number of negative comments/conversations or replies per post (or per post per follower) across any/channel
• Number of purchase intent statements per post or replies per post (or per post per follower) across any channel
• Number of followers (lost) gained across any channel from a particular post
• Number of people who follow a ‘call to action’ within social content
• Bounce rate/length of time spent engaging with that call to action (e.g., on a branded microsite)
• Average/total number of shares of any owned content
• Total time spent engaging with owned content

Example Social Media campaign Outcomes demonstrating limited engagement

• Number of social shares of any content/post
• Number of retweets, likes, +1s across any social content/posts

Business outcomes

Understanding the business outcomes of a particular campaign requires access to a broad range of other management information depending on the task at hand. Three of the most commonly measured ‘Business Outcomes’ are:

1. Number of website visits driven by the campaign
2. Number/value of sales over a set time period, driven by or influenced by the campaign
3. Cost savings driven by the campaign

We’ll look at each of these below

1. Number of site visits driven by the campaign

If the business objective is to drive traffic to a website or page, then this metric will be tracked and reported on within a regularly updated spreadsheet or dashboard. When set up correctly, a combination of tracking and evaluation tools typically identifies the amount of traffic generated to a specific web page or site from a particular campaign element, for example:

• Google AdWords
• Programmatic advertising
• Native digital advertising
• Email
• TV or radio advertising
• Online PR
• Social media content/engagement

2. Number/value of sales (over a set time period) driven by the campaign

Increasingly, marketing teams want to attribute specific on-site actions (e.g., sales) to specific user-journeys. User-journeys can be incredibly complex.

Consider the number of online and offline touchpoints and campaign elements from different retailers and the brand itself. A lot went into generating that sale—and only one retailer benefited.

To evaluate complex customer journeys effectively, you need access to a suite of analytical tools. They need to be set up correctly to record long-term and short-term data to help provide genuine long-term insight. Most web analytics tools provide detailed ‘Goals’ and ‘Sales Funnels’ to help ensure that all campaign activities that lead to an outcome, such as sale or behavior, are recognized. This means they can receive the appropriate attribution when comparing ROI across different channels.

Google’s guide to the customer journey illustrates the different roles that different digital channels and touchpoints can have in different countries and industries. The best campaign measurement dashboards for analyzing sales will typically include the numbers of visits to an organization’s site, customer touchpoints and typical customer journeys. They will also distinguish between and attribute the impact of different elements of paid, owned and earned campaign activity.

Outcomes That Reduce Costs Per Serve/cost

Many organizations are using campaigns and engagement channels such as social media to reduce the costs associated with customer service. International businesses, including BestBuy, Ford, IBM, Dell and British Telecom (BT), credit their communications and social media activity for contributing to a reduction in costs and an increase in customer satisfaction.

Creating campaigns and content to help customers navigate potential issues and to ‘self-serve’ as well as handling customer service issues on social channels is now the norm, rather than the exception. Establishing a framework to understand the effectiveness and the costs associated with social media customer service is increasingly important to many businesses. It is relatively easy to track and then report on information in real-time:

• The themes people are complaining about
• Speed of acknowledgement of comments or complaints
• Speed of resolution of individual complaints
• Frequency of follow-up praise/criticism from customers (including Net Promoter Score)
• % of incidents cleared up within 1, 2, or 3 interactions
• Cost per resolved incident

These can then be tracked against other management information, such as the existing cost of resolving customer service issues by telephone, live chat or email.

Section 2: Looking at Frameworks

In this section, we’ll look at how campaign measurement frameworks could be established for some of the most common objectives. We’ll consider the following briefs:

• Increased brand awareness
• Increased share of voice
• Improved brand sentiment
• Increased engagement by key audiences/influencers/stakeholders
• Call to action
• Increased traffic, purchase and intent to purchase
• Conversion/sales
• Improved customer service
• Measuring Return on Investment (ROI)

Increased Brand Awareness

The tighter that ‘brand awareness’ can be defined, the easier it is to track effectiveness. A generic ‘increase brand awareness’ brief is relatively hard to create a measurement framework for. Tools like Brandwatch enable brands to track exactly who is talking about them and what they’re saying and then do some basic comparisons.

Using the social channels’ own metrics, or any of the better social media monitoring tools available, this is an easy SMART objective to measure.

Case Study 1 – Argos digital store trials

UK retailer Argos launched their digital-first shops in December 2013. They replaced the bulky catalogs, which customers had been used to flicking through for decades, with iPad screens and wanted to track the impact on their brand. Argos developed a series of SMART campaign objectives related to the statement:

What does our core target audience think of the changes in our stores, and specifically what else could we do to improve them?

Argos used the Brandwatch Demographic feature to generate brand-related insights based on the different experiences of men and women in their shops. They also analyzed and measured the different experiences and expectations based on the location of the shops and used this information to inform the strategy around the digital-first stores.

As a result of the campaign measurement, among other things, Argos decided to keep some of the paper-based catalogues behind the counters.

The best brand-run campaigns are those that enable the brand to be responsive. Another specific example of this during the Argos campaign was when a customer Tweeted to the @Argos_Online Twitter account informing them that the sign above the store in Clapham Junction, SW11 had been named incorrectly. Because Argos had implemented the Brandwatch Rules feature, the Tweet was automatically filtered into the Clapham Junction Category, picked up instantly by the appropriate member of staff and fed back to the store directly, where the sign was amended, much to the satisfaction of the customer.

Increased Share Of Voice

When several brands/organizations compete for the same audience, aiming for an increase in ‘share of voice’ is a useful measure for which brand’s campaign activity resonates most. Measuring solely an ‘increase in share of voice’ is not specific enough, though.

So this objective needs to be more tightly defined. For example:

• Among all of the neutral/positive conversations about [home insurance] in the next week/month/year, we want brand X to feature in y%
• Among all the neutral/positive conversations about [home insurance claims] over the next week/month/year, we want brand X to be discussed more than brand Y
• Among all the advertisements about [home insurance] within Q3 and Q4, we want brand X shared or discussed with positive/neutral sentiment more than our competitors

These ‘share of voice’ metrics are often tracked because they allow Marketing Directors who are used to comparing recall for different TV/radio/outdoor executions in brand tracking research to consider the role and impact of each channel and paid, owned and earned media. They are regularly cited during peak periods, such as Black Friday, Christmas advertising campaigns, or around the Super Bowl. In both 2014 and 2015, Budweiser’s Super Bowl ads were the most shared across all social media channels according to a broad range of criteria, including research from Shift Communications and Unruly Media and a report in Time Magazine.

Improved Brand Sentiment

Sentiment—whether a comment about a brand/organization/product/service/person is neutral, positive or negative—is notoriously difficult to evaluate. Sentiment is also not an ‘Outcome’ per se. The tighter the definition (ideally related to a business outcome), the easier it becomes to measure sentiment. Based on the proprietary techniques for measuring sentiment, it’s possible to set up dashboards and regular reports to track all sorts of sentiment-related and purchase intent data based on SMART objectives.

Use social listening to measure the amount of purchase intent mentions your brand receives before you release a new product. These can help you forecast the amount of sales you’ll receive once your product is released, or at least give you an idea of:

• Ensure the sentiment associated with Brand X and Keyword Y is A% above the sentiment score of Brand Z and Keyword Y over B time period
• Ensure that the sentiment associated with Brand X and Keyword Y among our existing social followers and our target audience remains broadly positive during a specific time period.

Most sentiment measurement and automated analysis is still an ‘imperfect science’. In fact, when two individuals are asked to evaluate whether a comment is positive, negative or neutral, there will often be a discrepancy. The best social media monitoring tools have invested significant resources in ensuring that sentiment can be monitored much more effectively than ever before.

This objective is a fairly regular request for PR, communications and social media teams. All organizations want to reach and engage with the ‘right type’ of person—to impact brand metrics; to get them to write positive brand commentary; to change other people’s hearts, minds and behaviors; to force political/regulatory change; to win their business or for many other reasons. The definition of ‘engagement’, the number and the type of people—whether they are categorized as ‘audiences’ ‘influencers’ or ‘stakeholders’, should be clearly articulated. Who are your key stakeholders? Are they:

• Analysts
• Celebrities or ‘Tastemakers’
• Top tier customers
• Potential high-value customers
• Media
• Politicians
• Business leaders
• A certain category or blogger/influencer/ journalist, etc.

Starting with a keyword search, or selection of different keyword searches, it is possible to search for people by influencer score, number of followers, retweets, shares or any other relevant metric, and to rank influencers accordingly—in a way that’s relevant for your brand and your campaigns.

Once that list has been created, whether it contains 10 or 1000 people, SMART objectives can be written and then reported against. They could include:

• 40% of our ‘Members of Congress’ or ‘Member of Parliament’ list will follow us on at least one social media channel within three months
• 10% of our ‘Financial Analysts’ list will engage with us on social media within three months
• 5% of our ‘Potential High Value customers’ will accept or ask us for a meeting with six months

Once these objectives are clear, then it becomes relatively easy to track performance (and manage activity) to achieve them, within an agreed measurement framework.

Calls To Action

Many campaign briefs include maximizing a call to action. Such calls to action can range from successfully following a link, sharing a social media post/hashtag, calling a specific number, claiming a voucher/coupon or any number of other specifics.

When set up effectively, campaign monitoring tools can track a broad range of calls to action. Campaigns therefore need a framework that takes data from multiple sources from within the business, e.g., social media analytics, inbound-call tracking, physical store visits, etc.

This type of information can help track broader business-based objectives including:

• Increase the percentage of successful ‘self-serve’ enquiries by X% in Q3
• Reach X people with our new product launch discount offer code within 1 month, with a X% coupon redemption rate
• Increase the number of loyalty card applications in our Seattle store by X% in 2015
• Increase traffic to a specific page on our website by XX% within a year.

Traffic, Purchase, And Intent To Purchase

Given the significant changes to our media consumption habits, and the impact on marketing, communications and digital strategies, it has become increasingly important to understand the user-journey of people who visit an organization’s website. Analytics tools can track the number of visits to a link shared via paid, earned or owned media.

Factors like where visitors come from, how long they stay for, whether they’re existing customers/fans/followers, whether they buy or recommend are all useful aspects to understand as part of a measurement framework. For example, it may be that for every 100 people who come from a link shared on an organization’s Facebook page, five people make a purchase within a month. But for every 100 people who come from a link shared via Twitter, 10 people make a purchase within a month. Recording this data, using a combination of data sources, web ‘goal’ or ‘funnel’ analytics, and reporting on it on a regular basis, can inform marketing strategies and spend.

Case Study 2: Peruvian super pulses and coconut sugar

Supermarkets regularly search for what’s trending (or what’s about to trend) across social media to help plan stock levels and see what ingredients people are going to start shopping for. In October 2014, UK supermarket Waitrose identified that mentions of quinoa were significantly increasing. They tracked 54,000 posts about the pulse on Instagram alone.

Like other large food retailers, Waitrose would have created very specific SMART measurements to track food trends discussed and shared on social media over time. Given their opportunity to collect sales and loyalty data both on and offline, they could also track whether increases in conversation translate into purchases. A more sophisticated way of tracking would be not just to track the word ‘quinoa’, but to track the emotions and intent behind the mentions.

Brandwatch have previously published a number of posts and reports illustrating how to analyze the longer-term impacts of social media activity on marketing campaigns. Analysis of the travel and hospitality sector in 2014 demonstrates that simply tracking initial ‘buzz’ doesn’t automatically give you the greatest picture. Tracking the online conversation around three campaigns, here labeled as A, B and C, we can see that although Campaign A made the biggest instant impact with a huge surge in volume of mentions, Campaign B and C’s buzz had a more sustained effect.

So, while Campaign A’s impact upon conversation was short-lived, the other two managed to inspire online discussion for longer. This means that both generated more buzz than Campaign A over the studied time period of seven months.

Similarly, the Consumer Tech report published in 2014 also illustrates a way of clearly modeling ‘intent to purchase’ based on levels of online conversation. When it comes to retrospective analytics, Twitter’s own analytics published in January 2015 claimed that Twitter conversations (paid for and organic) have contributed to 18% of cinema ticket sales over the previous three years.

Customer Service

Many organizations are saving significant sums of money as a result of aligning their contact and customer service centers with digital and social channels. In 2013, an advertising agency trade body, the Institute of Practitioners in Advertising, quoted British Telecoms as saving £2m a year by adopting a more social-media based approach to customer service. Frameworks around which to judge the success of customer service will vary from business to business, but some of the more common objectives include:

• Respond to XX% of all mentions on social media within XX hours/minutes
• Close XX% of enquiry tickets raised via social channels within XX hours/minutes/days
• Increase % of customers who ‘self-serve’ based on previously shared information
• Compare % change in sentiment (or net promoter score) of people who have used social channels, compared with ‘traditional’ channels
• Reduce the average ‘cost to serve’ customer service enquiries by $XX or X% over a defined time frame
• Reduce the total costs associated with customer service by $XX or X% over a defined time frame

Air France’s KLM (Royal Dutch Airlines) are one of the most innovative companies with regard to customer service. They run a 24/7 operation and refresh their Twitter and Facebook cover images every five minutes to give customers an expected query response time.

In early 2015, Brandwatch embarked on a piece of research to understand more about how mature social media operations are in retail brands in the US and the UK. After selecting almost 100 brands and tweeting over 300 times from the accounts of our community management team, supported by data gathered using Brandwatch Analytics, we put the world’s leading retail brands to the test.

The results reiterated that there is still a lot of work to be done—you can read the full research here. Customer service through digital channels can also help signpost potential problems earlier than might previously have been picked up—as long as appropriate recording of issues and escalation processes are in place.

As well as creating frameworks to track the ‘explicit’ cost savings associated with social media customer service, many organizations are also now actively joining up customers’ social footprints and public information, with their own internal information about them to help build a bigger picture of their audience, to gain broader insights into groups of customers’ interests, or to identify VIPs who might require fast-tracking.

Return On Investment (ROI)

When a campaign’s SMART objectives are very clearly marketing and sales focused, it is possible to track the specific contribution that paid, owned and earned media campaign activity made in generating sales.

However, some ROI measurements undervalue the contribution that content, communications, social media and customer service make in the middle to long term, because campaign ROI tends to focus on immediate results. Where possible, accurate ROI calculations for campaigns should also factor in the longerterm value and correlation analysis of social media, customer service and PR, which includes many imputed benefits; for example, a value afforded to:

• Increased brand reputation
• Increased propensity to recommend the brand/product
• Increased propensity to brand loyalty
• Increased propensity to repeat or higher value purchases

Your ROI framework should also recognize the limitations of last-click attribution with regard to valuing sales/outcomes. Establishing such accurate frameworks veer towards econometric modeling—entirely appropriate for large-budget campaigns, but perhaps out of reach for many organizations.