The big data opportunity? It’s not about volume and velocity, it’s about integration of transactional and behavioural data. For all the excitement about data at scale, 56% of marketers are still either collecting data ad-hoc with no centralisation, or are still working towards a single view of the customer.

A surprise finding in a new survey conducted by marketingfinder.co.uk in association with IBM SPSS Predictive Analytics Software, is that digital marketing data sources dominate over actual customer behaviour: Twice as many marketers record website visits as e-commerce transactions. While this gives them a view of what visitors are doing on a site, once they decide to purchase, half no longer manage that data. This divide between prospects and customers is historical and has clearly not been resolved in the new data-driven digital era.

What is not surprising, given these findings, is that the type of analytics most commonly used in marketing departments are purely functional: web analytics, email open and click through rates. While four in ten marketers have built customer segments, only one quarter understand customer lifetime value and just one fifth can model customer churn.

This highlights the data gap which a limited data strategy and data capture programme is leading to. The value that can be achieved by sales and marketing teams (who are shown to be very data-intensive in the survey) – is not being maximised, as they are not able to understand all of the information they are being provided with.

Many of these gaps are planned to be filled in the next 12 months as marketers seek to respond to the growing demand for data and insight within their business. Advanced analytics driven requirements, such as customer churn reduction or customer experience management, are highest on the wish list for the coming year.

If those plans are brought to fruition, expect marketing to become newly super-charged through its understanding of customers and prospects. Marketers who are not planning for analytics - or fail to deliver on their plans - risk being left behind.

1. The Big Data Picture

Big data is commonly defined as having three dimensions - volume, velocity and variety. To qualify as a big data user, an organisation needs to be managing and analysing information streams that display these three qualities.

Based on the findings of this survey, barely a handful of companies have genuine big data operations when measured across those three factors - only 9.3% have more than 50 data feeds, 32.6% have formal APIs with social networks and 27.9% refresh their data either in real time or hourly. As an aggregate (9.3 x 32.6 x 27.9), this means just 0.84% of data management and analysis programmes can truly be described as big data in nature.

Volume of data sources in use

For most companies, a small set of data provides most of the value to be derived from analytics. In four out of ten organisations, this actually means fewer than five data feeds are in use, reflecting a stable base but also a degree of lag in developing additional sources that could be of value.

As the number of product lines, operating divisions or delivery channels grows, so does the volume of potential data feeds. Three out of ten companies draw on 6-15 data sources - developing, but hardly complex to manage. 13.9% report using between 16 and 50 data sources - the basis of a more sophisticated level of data management and analysis, but one in its early stages.

True volume of data is found in less than one in ten organisations. Of these, 7% have between 51 and 100 data feeds in place - a maturing footprint. The real leaders in big data are the 2.3% who report having over 100 sources on tap. This represents both a very complex challenge for data management, but also the richest possible source on which to base analyses and models.

Variety of data sources in use

Data sources have proliferated over the last decade, both inhouse (on-premise e-commerce, CRM, customer service) and third party (digital marketing and social networks). Other third party sources add a further dimension to the information sources that can be drawn on, such as market research and purchased data. Use of this wide range of possible sources divides into roughly four levels. Three data feeds dominate, with three quarters of organisations tracking website visits, six out of ten monitoring the performance of their email and half drawing data out of their CRM systems. All three of these sources are mature and data-rich, providing a valuable asset to their users.

In the second tier can be found core data feeds that are in use at a third of organisations (e-commerce and in-store transactions) as well as those that are critical to digital marketing (paid and organic search). It is notable that the “voice of the customer” is being listened to by a third of companies through the use of three distinct, but complementary data sources - market research, social network APIs and call centres.

The third rank of companies is made up by the one quarter who are enriching their data through buying information from third parties, or who track the performance of their online display ads and affiliate network references.

The final layer is made up by around one in seven companies who are adding relatively indirect data into their analytics environment. This is being done by the inclusion of postal purchases and those made through agents or distributors, as well as by scraping public domain data from social networks. This last practice is often highlighted as a key characteristic of big data, but also presents some risks around compliance.

Velocity of data in use

Digital channels which operate on a 24/7 basis are producing data at an unprecedented rate. For some organisations it has become critical to their business to harness this data and to be as up-to-date as possible. That is reflected in the fact that one in five say they refresh their data in real-time, while another 7% do so hourly. For sectors with dynamic stock or distribution, such as retail or travel, this pace of refresh is essential.

Daily or overnight updating of data has become relatively standard practice, as can be seen in the fact that three out of ten maintain their information at this pace. This provides them with the ability to begin each day with a very recent snapshot of customers and markets, without compromising the performance of operating systems while data is added. Those taking a weekly feed (nearly one in five) are likely to be in sectors where the rate of purchase or competitive conditions can tolerate a degree of latency.

Updating on a slower cycle - monthly or even less often - now looks like a relic of a previous era when database refreshes were cumbersome and costly. While it is true that some data sets are only revised at this pace, such as third party mover and deceased files, the nearly 17% of companies refreshing their core data at this slow pace are yielding significant competitive advantage.

One in five organisations say they refresh their data in realtime, while another 7% do so hourly

2: The Big Strategy For Data

For most organisations, the goal of a single customer view and centralised analytics teams is a work in progress. In the majority of businesses, insight and analytics is being carried out at a departmental level, often by individuals who may find themselves isolated from the rest of the enterprise. The small group of companies which have already developed a fully-distributed model of analytics are gaining a significant competitive advantage.

It is also the case that most analytics deployments remain tactical, supporting both individual functions and constrained activities that show immediate returns. These “low hanging fruit” are important in building the skills base, business case and confidence in what data can do for the business, but there is also a risk that they do not provide a sustainable platform for the type of breakthrough performance which can ultimately be achieved.

Strategic view of data in the organisation

To take advantage of the big data opportunity, organisations have to “get” data at the highest level. This will be reflected in the presence of a centralised data strategy, which identifies, manages and exploits all data assets in a co-ordinated and consistent way. At present, just one in seven companies have established this strategy across their enterprises. There could be a tipping point within the next two to three years - half of companies are working towards a formal data strategy.

Once these strategies are implemented, it will transform the way those organisations deal with their data assets and usher in a new era of data-enabled business. Despite this predominant trend, one fifth of businesses lag behind without even having a formal data strategy at departmental level. Failing to ensure that data is captured and used in a defined way means value is being neglected (and potentially destroyed).

Organisational data strategy

The full picture of customers and prospects that can be built by combining direct transactional and behavioural data with external social and big data sources has only been created at around one in thirteen companies. This is no real surprise given the relatively recent availability of these additional data sources and the complexity of the data management environment required.

What is remarkable is how few companies have managed to build a single customer view. Despite a decade-long drive towards this data resource, fewer than one in twenty companies have arrived at the destination of SCV. It may be that overcoming the challenges of siloed data may be greater than anticipated, or that disruptive channels and data sources continually prevent this asset from being created.

For the majority, SCV remains a journey with the centralised, integrated view of the customer being a goal for more than half of businesses. At least this large group has a view and plan it is working on - for one in six companies there is no attempt at data integration. This severely constrains the understanding of customers which can be gained within those organisations.

Data programmes in place (organisation-wide v departmental)

Key resources need to be in place in order for a business to maximise the benefit it derives from data management activities. These include a single view of the customer (SCV), data governance, data quality, insight and analytics. Only one in four companies have established these organisation-wide.

Single view of the customer (SCV) across the organisation is claimed by 20.9%, compared to half of businesses which still operate with department-level customer databases. These standalone databases are an important building block of SCV and are often a stepping stone in the journey towards a fully-integrated view. Leveraging that data requires an insight and analytics team. Two-thirds of companies do have these practitioners in place, split nearly equally between those working in a centralised, organisation-wide team (30.2%) and those confined to a specific department (34.9%).

As the value of data gets recognised through its co-ordinated use across the organisation, so programmes to protect and develop it tend to emerge. Both data governance and data quality programmes are far more prevalent at an organisation-wide level, not least because the cost base of these is more affordable when data is being exploited as a core business asset, rather than being funded by individual functions.

Functions making use of customer and prospect data

Every business function has become a data user, but not every business has given access to customer and prospect data to every function. Only one in six organisations can boast of having data-enabled all the major departments across the organisation.

For the vast majority of companies, it remains the customerfacing operations which are primarily making use of data and primarily the long-established users - marketing and sales. The value they derive from data is very apparent, which may explain the widespread adoption within these units.

More surprisingly, two other major customer-facing functions are less well equipped with data - only four out of ten customer service operations have access, while inside e-commerce units, just one in six can use customer and prospect data. That seems like a significant gap given their role in the customer journey.

More strategic adoption of data is evident in between one fifth and one third of companies. Board-level usage at three in ten companies is very encouraging, while aligning product development with customers is likely to pay real dividends. Out of these top-level functions, only IT lags behind, although it is clearly more the enabler of data access than a consumer of that information.

3. The Big Role For Analytics

The coming year could be a big one in terms of data, insight and analytics. If all of the plans which are claimed to have been laid for the next 12 months do come to fruition, then it will transform the landscape of this practice. Evidence-based decision making and data-led forecasting will become the norm, rather than the exception.

This shift will be achieved through the migration of individual analysts out of their current departments and into a centralised insight team. If companies provide genuine strategic support - through funding at board level rather than from local budgets - then a new raft of business and marketing strategies will be realised.

Types of analytics being used or planned

Analytics is where data delivers its value and the data management practice becomes operational. By understanding internal dynamics (website and email performance) as well as customer dimensions, the organisation becomes able to make better decisions and forecast the outcomes of those decisions.

Digital insights dominate, with web analytics and email metrics the most widely used. Both of these tend to be built into systems which support websites and email distribution and, as such, do not represent sophisticated analytical work. By contrast, only half as many are analysing their marketing performance and ROI by channel. However, the intention to deploy marketing ROI by channel is the highest for all types of analytics which companies intend to adopt in the next 12 months.

There is a fine balance between the well-established and the newest analytical practices. Customer segmentation has a long track record and is being used by six out of ten companies, whereas social media analytics has only emerged in the last few years, but is already being deployed by half of companies. It is notable that a further three in ten plan to adopt this in the next 12 months.

One-third of companies are deploying the suite of activities which can be said to typify sophisticated analytics - these include propensity modelling, customer lifetime value (CLTV), data mining, data accuracy and customer churn. While one third intend to adopt CLTV in the next year and one quarter say they will adopt data mining, it is surprising that only a further one in ten expect to be measuring customer churn within 12 months.

Current and planned support for analytics

Analysts are currently most likely to find themselves working in isolation within one department. But that picture may be about to change dramatically - nearly half of all organisations intend to create a centralised insight and analytics team in the next 12 months. For the analysts currently working on their own in four out of ten businesses at the moment, the good news is that they maybe about to move into the heart of the business. At the same time, a further one in five companies will be catching up on the analytics trend by introducing a skilled practitioner into some of their business teams.

Although three out of ten companies do already have centralised insight and analytics, building this function will be quite a leap for those planning to introduce such a team. Despite the presence of individual analysts in some functions, just one in twenty organisations have an analyst in every department. That will make resourcing the centralised role a significant challenge, especially finding the individuals with appropriate skills.

Despite this existing analytics skills gap, very few companies are using outsourced partners to support their analytical objectives. This seems like a missed opportunity to leverage both the human resources and the knowledge which such partners can offer. Equally, very few companies have reached the mature stage where insight and analytics operates in a distributed model, accessible to business users everywhere in the organisation.

Functions providing funds for insight and analytics

Analytics is a cost centre which has to be funded - this investment is usually offset against the considerable returns which it delivers through improved performance or cost-savings. Not surprisingly, therefore, marketing and sales are the two major sponsors of insight and analytics teams. Both of these departments will see immediate rewards from the tactical application of the outputs from their analysts’ work.

More strategic support is less common, although nearly one in seven companies does have funding from the board for its analytics team. This is likely to be the case where the insight role has been centralised and its delivery has become part of the organisation-wide strategy and decision-making process.

Support from other functions is limited, even where these seem likely to enjoy significant benefits (such as e-commerce, customer service, product development and finance/risk). This may simply reflect the maturity curve on which adoption tends to happen within individual departments, before becoming centralised and subsequently distributed.

Business and marketing strategies currently or planned to be supported by analytics

Analytics has the potential to contribute to a wide range of different goals within a business, from the strategic to the tactical. For the majority of organisations however, insight has not yet broken out of the local and tactical domain. For the majority of companies, it is marketing planning and customer targeting which benefit from analytical input, closely followed by sales planning, prospect targeting.

Those five strategies will have been beneficiaries of analytics for the last few decades, previously as part of direct marketing programmes and now within the digital marketing sphere. That between one fifth and one-third of companies are only just now planning to adopt analytics for these purposes indicates a significant degree of lag behind standard practice.

Best practice with analytics sees it deployed into far more strategic areas, especially those focused on the customer (winback, churn reduction, listening). The next 12 months is likely to see at least a doubling in the number of organisations where this is happening, with “voice of the customer” analytics showing the second-highest planned rate of adoption. Despite this, reputation management has so far been slow to adopt analytics, although this shows the same level of planned uptake in the coming year.

Analytics to drive the business more effectively remains more limited - one quarter of companies are already using it for revenue optimisation, while four in ten are planning to do so. This is the most-planned area in which analytics is to be introduced. Compliance and reporting could undergo a significant increase in its use of analytics to become an exercise for the majority, but there is still a low level of understanding of how analytics could drive risk and fraud mitigation.

Marketing and sales are the two major sponsors of insight and analytics teams

4: The Big Benefits And Barriers To Analytics

Introducing analytics and deriving the full benefits requires commitment. Once the concept has been embraced, an organisation needs to provide executive support, proper funding, personnel and a specific IT infrastructure. All of these are potential challenges and a reason why many companies have struggled thus far.

Yet once they are in place, the organisation will migrate up the maturity curve of analytical deployment, seeing impact more extensively across the business. Getting the measure not only of how marketing and sales are performing, but also of where investment is delivering the greatest return is the indicator of a well-established and resourced analytics team.

How insight and analytics is impacting on organisations

On the maturity curve for the use of analytics, nearly half of organisations are at the developing stage. They have moved on from a condition in which analytics are hardly being used to drive business processes - only 7.9% of companies are still at this stage. Instead, some processes are now being optimised through the insights which can be derived from data. This has already been noted in the deployment of analytics for marketing planning and customer targeting.

The next step towards sustainable analytical deployments has been taken by nearly one fifth of businesses. Here, many business processes are being transformed through insight. A further one in six companies are at the point where they use analytics as the basis for decision-making, leading to optimised processes.

Reaching this optimised state is a significant challenge which requires budget, human resource and top-down backing. It is therefore not too surprising that relatively few organisations have been able to get to this level so far. But there does appear to be a tipping point on the horizon which will see the majority of businesses having widespread analytics in place which are delivering considerable impact.

Visible benefits of analytics and insight in the organisation

Given the level of maturity of most analytics deployments, it is to be expected that improved sales and marketing performance are in the top three areas where companies are seeing a visible benefit. The uplift of response and sales from targeting and planning tend to be immediate. Less expected is that optimised customer experience is rated as the number one benefit - in its fullest sense, customer experience is a complex process involving multiple touchpoints and a multi-stage journey from prospect to advocate. It may be that the use of web analytics is driving the perception that optimising the website ticks the box of customer experience.

Improving the way channels are managed, how budget is deployed and what it delivers back to the organisation are core elements of the business case for analytics, so it is encouraging that these are rated fourth, fifth and sixth for their impact. This benefit is fully delivered when conversion rates improve - something that 37.8% of companies are experiencing as a result.

Although well over a third of companies see improved brand reputation as a result of deploying analytics, only one quarter are enjoying improved levels of recommendation - driving net promoter score as part of the analytical programme is only likely to be undertaken by the more advanced practitioners.

It is encouraging that one third of companies have recognised the link between better targeting and relevancy in marketing and a reduction in their opt-out rate or improvement in the volume of customers giving permission to be marketed to. Sustaining marketable data levels will be critical if the data protection environment becomes more restrictive.

Areas which have yet to benefit from insight and analytics

As analytics spreads across an organisation, more areas of the business start to realise benefits. For most companies, this progression is taken a step at a time - deploying insight, measuring its impact and return before rolling out to the next area.

Given the level of development of analytics at nearly half of companies, it is logical that eight key areas are reported not to have experienced any benefits yet by three in ten. Those companies where customer experience, marketing and sales performance have yet to experience any benefit are seriously behind the curve, given the dominance of these three areas within companies that have already adopted insight.

More complex activities, such as fraud/risk management, channel management, customer management ROI and optimisation, tend to be addressed in the second wave of analytics deployments, which is why many companies have yet to see their impact. On a positive note, these gaps in the picture of analytical uplift imply substantial growth potential and grounds on which to build a business case.

Organisational barriers to the adoption of insight and analytics

For a company to adopt analytics, a number of elements need to be in place - executive support, funding and resources. Even assuming that the business has accepted the potential value of insight - either at board level or within an individual department - there may still be reasons why it is not able to move towards its implementation.

Data challenges are two of the top three obstacles named by companies, both of them related to integrating data from multiple systems. Even though seven out of ten companies have fewer than 15 data feeds to work with, the lack of a single customer view and having siloed data is named by four out of ten as a barrier to adoption. In addition to this, one quarter have data quality problems, while for nearly one in five there is a simple lack of data.

Human resources are the most commonly named barrier. With the increased interest in analytics across business and a decline in the number of skilled practitioners, finding suitable analysts to work in a new insight team is a genuine problem.

Two other key issues also confront the business when thinking about adopting analytics - the first is a data environment optimised for this activity, which one third identify as an issue, while the second is a lack of funding. These are closely related and can only be solved by creating a robust and credible business case that convinces the organisation to make the necessary investment.

Summary

For the vast majority of organisations, there is much that can be achieved with the implementation of analytics. Whilst it is true that the benefits of analytics increase exponentially as the number of data sources increase and as companies make data accessible to more departments and personnel, even simple departmental deployments can help shape marketing efforts and improve marketing ROI immediately.

It is clear that organisations plan to invest more time, money and resources over the mid-to-short term in an attempt to seek out revenue by growing their data and understanding the data that they already have but, perhaps without clear direction, they are struggling to know where to start.

The good news is that, for organisations of any size, analytics entry points are very low, both in terms of IT skills required and initial investment.

Even if your organisation doesn’t have a data strategy or you find it difficult to convince stakeholders of the opportunities, you can very easily use analytics to understand the data that you already have and begin to make improvements to your marketing right now. Happily, by its very nature, analytics will make it easy for you to demonstrate ROI and grow from there.