Big Data offers business enterprises a never-before opportunity to improve productivity and their revenue. However, enterprises have been struggling with the task of getting the best out of the Big Data they collect. A survey conducted in 2012 on 300 top executives and managers quite clearly summarized the struggles organizations have been facing managing Big data. Here are the salient points of the survey:
- 66% of the respondents want to widen the reach of the analytics tools to more people in their organizations.
- 59% of the respondents believe that the existing analytics framework is too slow in processing the Big Data.
- 57% of the respondents believe that their analytics framework is unable to match the speed at which Big Data is flowing into their network.
- 55% of the respondents believe that their analytics tools are too hard to use and do not provide information in an easy-to-understand format.
The above findings quite summarize the main challenges organizations face with the Big Data. However, since 2012, the problems have become more complex, especially because the Big Data volume has been increasing at a great speed and because of the attitude of many enterprises towards Big Data management strategies.
While there are challenges such as technical, hardware and software issues, it quite seems that organizations are not handling Big Data with the seriousness it deserves. That seems to be the root problem. The paragraphs below describe the main pain points with Big Data.
Poor attitude towards data
Big Data Management strategy actually reflects how marketers view the Big Data and its potential. And so far, the strategy has lacked seriousness and consistency. According to a Harvard Business Review article, marketers depend on Big Data for just 11% of their customer-related decisions. And that, after all the publicity and hype Big Data has been receiving. Clearly, the words and actions do not match. The Harvard Business Review survey found that most marketers relied more on their intuition about customers and their previous experiences in order to arrive at decisions. Reliance on data usually would be the last item on their list. This is a faulty and an outdated mindset. For example, if you are selling a product targeted at people over 60, you may not consider having a Facebook page because according to your experience, older people do not tend to use Facebook. But in a volatile business environment, such assumptions may no longer hold good.
There is another group of marketers that are excessively obsessed with data. They represent another dimension of poor data management. This group constantly monitors and tracks data received, focuses excessively on granular details and gets distracted from the main goals if even a small percentage of the data shows adverse results. This group of people can derail data management projects because they insist on constant resetting of goals and targets which can be unsettling.
Big Data Myths
Figure 1: Marketers and Big Data Myths
Faulty Data Management Strategy
In an interview, Michael Nevski, Consultant, Consumer and Shopper Marketing – IRI observed that organizations are not paying substantial focus on marketing efforts. In many organizations, the marketing department is understaffed and the available marketers are doing other assignments not related to data. So, there is no bandwidth available to focus on data analysis and decision making based on data. A number of marketers are focusing too much on product management activities such as packaging and labeling. To take care of deep data analysis and customer insights, organizations are outsourcing that activity to third party vendors. This strategy may not give significant Returns on Investment (RoI) if the entire arrangement is not planned well. First, many organizations also outsource data analysis on third-party providers. This may prevent the marketers within the organization to grow expertise on Big Data. If the organization is completely reliant on the third-party sources for data analysis and insights, then it needs to make sure that the third party completely understands its goals and targets and aligns its actions accordingly. This could, for example, be done by the vendor attending weekly and monthly meetings with the marketing teams.
The CMO needs to make sure that all marketing decision makers are able to access data and run the analysis and insights regularly.
Need to analyze more data quickly
Every day, your data capture framework is collecting enormous volumes of data from several sources — social media, sensors, cell phones, transaction records, and many more. The data keeps multiplying but it can prove useful only when you can process and analyze the data quickly to gain meaningful, relevant insights. Needless to say, you need advanced data capture and analytics framework to do this.
Interpreting the data
Data visualization is going to play a key part in interpreting the data. Data visualization, if you did not know already, is the presentation of information in a pictorial and graphical format. This obviously makes it easier to understand information. However, to perform data visualization, the data must first be understood in a context. For example, if the data is sourced from social media, then it is necessary to understand or decipher first the customer needs. Only then can you present the data in a more understandable format.
Ensuring data quality
Data quality is the main challenge you face when you are going to perform data visualization. Your data capture framework is going to collect data from multiple sources and depending on the quality of the filtering your tools apply, there might be minimal filtering on the data quality. In such a case, you waste a lot of time in processing and weeding out redundant data. So, you are losing out on time. Even if you are able to perform quality data visualization later, you might already have lost valuable time and your competitors have edged you out.
Making available meaningful data
It is a big challenge to make available meaningful data especially because of the huge volumes of data collected. For example, if you need to display 20 billion rows of retail data that you need to display, the user will find it extremely difficult to make any sense out of it. You can cluster the information into smaller and logical groups and present them to the audience. In such a way, the audience will be able to view the data they want.
Security issues
With the cloud infrastructure becoming more accessible to everybody, organizations are increasingly storing their Big Data into the cloud. Since the cloud storage is accessible from anywhere with an Internet connection, this is opening up new, sophisticated security challenges. Since Big Data is collected from multiple sources, it is a challenge to make sure that the data coming in is secured. Big Data can be manipulated at the time of processing mainly because the Big Data processing tools like Hadoop and NoSQL were not originally designed with security in mind. So, organizations are faced with the task of balancing between collecting Big Data and ensuring security and confidentiality.
Summary
Big Data is a huge opportunity for business enterprises but they need to pay more attention to how they view Big Data and its uses, especially the marketing department. They need to get rid of gut and experience-driven marketing decisions and rely more on objective information. When they do this, the other challenges can be more easily overcome.
