Opinion Piece: Benefit from global Big Data lessons as technology matures
By Gary Allemann, Managing Director, Master Data Management
With the core big data trends of 2014 maturing around the globe, and new trends emerging, we South Africans find ourselves in a fortunate position of being able to learn from the trials of international organisations. By examining evolving big data trends around the world, local businesses are able to deploy it more quickly and for the right reasons.
One of the main trends is a shift in strategy with regards to big data as projects are increasingly business rather than IT driven.
Our US-based partner Datameer is seeing an increasing interest from business users with many new big data projects driven by the marketing or sales departments. This shift is driven by the reality that earlier adopters can gain a time advantage over competitors for example around customer insight, which will be hard to erode
The conversation has shifted from “What is big data?” to “Where can we gain quick benefit?”
Leading self-service big data platforms have emerged to put the entire big data lifecycle (preparation, integration, analytics and visualisation) into the hands of business users, while providing controls such as security, governance and even data quality.
Datameer also sees a shift in the application of big data in the US and Europe.
Where early adopters focussed more on web analytics, more mature implementations are focussing on more sophisticated uses including the optimisation of the enterprise data warehouse, customer analytics and customer experience management, fraud prevention and compliance.
The lesson for local companies is simple. Big data is not pie in the sky; financial services, telecommunications, healthcare, retail, education and government is gaining new insights into business that allow them to compete more effectively. We can look at these case studies to understand how big data can be applied in our market.
For big data to deliver value, as with any technology, it must shift from being IT driven (yet another platform) to being business driven. This reality is driving another international trend.
Big data is increasingly being deployed at a departmental level to answer specific, specialised questions. Since each department has its own set of unique business problems, an overarching enterprise solution often takes too long to deploy. Individual departments are finding that they can get the answers they need, now, by deploying their own solutions.
The introduction of Hadoop as a Service (HaaS) is contributing towards this reality, as business users can deploy quickly without relying in scarce technical resources.
In South Africa, cloud solutions for big data are at an early stage of adoption with cost, bandwidth and privacy all raised as legitimate concerns. However as data governance policies mature they must cater for cloud, with appropriate due diligence and controls to manage these risks.
IT can enable departmental initiatives, while retaining control, through the provision of private (or mixed public/private) cloud Hadoop deployments, coupled with tools such as Datameer that give visibility into the full Hadoop ecosystem. This makes it easier to track and control how the data lake is growing, who has access to which data, and what it is being used for.
Big data should be a focus in any companies’ data governance strategy as companies grapple with both the opportunities and the very real challenges that big data brings.
With some big data implementations running into years, and tens of millions of dollars, the opportunity cost for local business is high. As relatively late entrants, local business can learn from international mistakes and get to value quickly.
GfK and VMware: Innovating together on hybrid cloud
GfK has been the global leader in data and analytics for more than 85 years, supplying its clients with optimised decision inputs.
In its capacity as a strategic and technical partner, VMware has been walking GfK along its digital transformation path for over a decade.
“We are a demanding and singularly dynamic customer, which is why a close partnership with VMware is integral to the success of everyone involved,” said Joerg Hesselink, Global Head of Infrastructure, GfK IT Services.
Four years ago, the Nuremberg-based researcher expanded its on-premises infrastructure by introducing VMware vRealize Automation. In doing so, it laid a solid foundation, resulting in a self-service hybrid-cloud environment.
By expanding on the basis of VMware Cloud on AWS and VMware Cloud Foundation with vRealize Cloud Management, GfK has given itself a secure infrastructure and reliable operations by efficiently operating processes, policies, people and tools in both private and public cloud environments.
One important step for GfK involved migrating from multiple cloud providers to just a single one. The team chose VMware.
“VMware is the market leader for on-premises virtualisation and hybrid-cloud solutions, so it was only logical to tackle the next project for the future together,” says Hesselink.
Migration to the VMware-based environment was integrated into existing hardware simply and smoothly in April 2020. Going forward, GfK’s new hybrid cloud model will establish a harmonised core system complete with VMware Cloud on AWS, VMware Cloud Foundation with vRealize Cloud Management and a volume rising from an initial 500 VMs to a total of 4,000 VMs.
“We are modernising, protecting and scaling our applications with the world’s leading hybrid cloud solution: VMware Cloud on AWS, following VMware on Google Cloud Platform,” adds Hesselink.
The hybrid cloud-based infrastructure also empowers GfK to respond to new and future projects with astonishing agility: Resources can now be shifted quickly and easily from the private to the public cloud – without modifying the nature of interaction with the environment.
The gfknewron project is a good example – the company’s latest AI-powered product is based exclusively on public cloud technology. The consistency guaranteed by VMware Cloud on AWS eases the burden on both regular staff and the IT team. Better still, since the teams are already familiar with the VMware environment, the learning curve for upskilling is short.
One very important factor for the GfK was that VMware Cloud on AWS constituted an investment in future-proof technology that will stay relevant.
“The new cloud-based infrastructure comprising VMware Cloud on AWS and VMware Cloud Foundation forges a successful link between on-premises and cloud-based solutions,” says Hesselink. “That in turn enables GfK to efficiently develop its own modern applications and solutions.
“In market research, everything is data-driven. So, we need the best technological basis to efficiently process large volumes of data and consistently distill them into logical insights that genuinely benefit the client.
“We transform data and information into actionable knowledge that serves as a sustainable driver of business growth. VMware Cloud on AWS is an investment in a platform that helps us be well prepared for whatever the future may hold.”