Opinion Piece: Benefit from global Big Data lessons as technology matures

Stay Connected! Follow @AfricaBizReview and @MrNLon on Twitter. Like our Facebook Page.

By Gary Allemann, Managing Director, Master Data Management

With the core big data trends of 2014 maturing around the globe, and new trends emerging, we South Africans find ourselves in a fortunate position of being able to learn from the trials of international organisations. By examining evolving big data trends around the world, local businesses are able to deploy it more quickly and for the right reasons.

One of the main trends is a shift in strategy with regards to big data as projects are increasingly business rather than IT driven.

Our US-based partner Datameer is seeing an increasing interest from business users with many new big data projects driven by the marketing or sales departments. This shift is driven by the reality that earlier adopters can gain a time advantage over competitors for example around customer insight, which will be hard to erode

The conversation has shifted from “What is big data?” to “Where can we gain quick benefit?”

Leading self-service big data platforms have emerged to put the entire big data lifecycle (preparation, integration, analytics and visualisation) into the hands of business users, while providing controls such as security, governance and even data quality.

Datameer also sees a shift in the application of big data in the US and Europe.

Where early adopters focussed more on web analytics, more mature implementations are focussing on more sophisticated uses including the optimisation of the enterprise data warehouse, customer analytics and customer experience management, fraud prevention and compliance.

The lesson for local companies is simple. Big data is not pie in the sky; financial services, telecommunications, healthcare, retail, education and government is gaining new insights into business that allow them to compete more effectively. We can look at these case studies to understand how big data can be applied in our market.

For big data to deliver value, as with any technology, it must shift from being IT driven (yet another platform) to being business driven. This reality is driving another international trend.

Big data is increasingly being deployed at a departmental level to answer specific, specialised questions. Since each department has its own set of unique business problems, an overarching enterprise solution often takes too long to deploy. Individual departments are finding that they can get the answers they need, now, by deploying their own solutions.

The introduction of Hadoop as a Service (HaaS) is contributing towards this reality, as business users can deploy quickly without relying in scarce technical resources.

In South Africa, cloud solutions for big data are at an early stage of adoption with cost, bandwidth and privacy all raised as legitimate concerns. However as data governance policies mature they must cater for cloud, with appropriate due diligence and controls to manage these risks.

IT can enable departmental initiatives, while retaining control, through the provision of private (or mixed public/private) cloud Hadoop deployments, coupled with tools such as Datameer that give visibility into the full Hadoop ecosystem. This makes it easier to track and control how the data lake is growing, who has access to which data, and what it is being used for.

Big data should be a focus in any companies’ data governance strategy as companies grapple with both the opportunities and the very real challenges that big data brings.

With some big data implementations running into years, and tens of millions of dollars, the opportunity cost for local business is high. As relatively late entrants, local business can learn from international mistakes and get to value quickly.

Read the September Issue of African Business Review.