Insight: The good, the bad and the ugly of the data industry

By Darren Watkins, managing director for VIRTUS Data Centres

As Facebook responds to a public relations nightmare — the fallout from news that a political consulting firm violated its rules for third party apps — it’s likely that any organisation which uses consumer data will be anxiously waiting to see what the long-term impact of the scandal will be on the industry.

However, despite the damning headlines, big data is a long way from being all bad. On the flipside of the Cambridge Analytica scandal, Facebook has recognised the positive impact of the data it collects. Its Data for Good program is focused on getting information in the hands of academics, NGOs, and the broader development community so they can make a positive impact on world issues. A great example of this is its disaster maps initiative, launched in 2018 with UNICEF, where aggregated and anonymised location data has been able to significantly improve situational awareness when responding to natural disasters.

See also:

• Business Chief, Europe edition - May issue OUT NOW!
• Interxion opens second data centre to increase Marseille's 'digital hub' status
• European Investment Bank approves €4.3bn for 20 new projects

Closer to home, data has become an intrinsic part of all of our lives. From shopping, to dating, to the connected devices in our pockets, data is fuelling nearly every aspect of our lives. It even helps us get to work on time. Numerous real-time data metrics go into ensuring traffic runs smoothly, including GPS systems, social media use and magnetic sensors - all of which help traffic agencies manage the roads.

So, whilst recent scandals have understandably caused concern, it’s vital that we don’t throw the baby out with the bathwater. Big data makes consumers’ lives easier and helps businesses function better: the ability to access and interpret it as meaningful actionable information, very quickly, will give a huge competitive advantage to those organisations that do it well. Whilst innovation must be encouraged, it should not come at the cost of privacy and security. If we don’t get the balance right between data protection and data driven innovation, the UK economy will suffer and personal data may be misused.

A balancing act

To be successful, firms need to strike a precarious balance between innovation and data protection. At one end of the spectrum, there is distrust of the use of data beyond limited, specifically identified purposes. At the other end, the recognition that data is a valuable asset suggests that its more widespread use could empower innovation and economic opportunity. This is a complex argument, which isn’t likely to be categorically solved anytime soon. but whichever side of the fence you fall on, there is a duty for companies who work with data to manage it appropriately. It is imperative that companies have an infrastructure that anticipates growth in data volumes and expansion of data types as well as developing an institutional culture that fundamentally understands the importance of big data. Personnel at every level must work with a mindset that ensures the entry of complete, accurate and uniform data - essentially good data “hygiene”.

Good data hygiene extends to the organisation of data too, encompassing well-defined schemas, an organised data architecture and more. This approach allows companies to perform analytics faster, ensuring the data they’re working with is accurate, current and used appropriately. But of course, data storage, processing and maintenance can be expensive.

Prioritising security

Good data hygiene is a crucial building block in good data use. But the most visible thing to get right, in the eyes of the consumer at least, is security.

Simple hacking is where big data shows weakness, thanks to the millions of people whose personal details can be put at risk with any single security breach. And the scope of the problem has grown considerably in a short time. For instance, it wasn’t too long ago that a few thousand data sets being put at risk by a hack was a major problem, which made major headlines. But in September last year, Yahoo announced that it had not secured the real names, date of birth, and telephone numbers of 500 million people. It’s data loss on an unimaginable scale, and of course, for the public, that’s scary stuff.

This, together with the computing power needed for big data applications, puts increasing pressure on organisations’ IT and data centre strategies. These challenges need to be overcome if big data isn’t to always spell danger. Indeed, it's no exaggeration to say that data centre strategy could be crucial to big data’s ultimate success or failure.

For even the biggest organisations, the cost of having (and maintaining) a wholly owned data centre can be prohibitively high. But, for many, security concerns mean that a wholesale move to standard, cloud platforms in a hybrid model isn’t an option. The savviest firms are turning to colocation for their data storage needs, recognising that moving into a shared environment means that IT can more easily expand and grow, without compromising security or performance.

By choosing colocation, companies are effectively achieving the best of both worlds; renting a small slice of the best uninterruptible power and grid supply, with backup generators, super-efficient cooling, 24/7 security and resilient path multi-fibre connectivity that money can buy that has direct access to public cloud platforms to provide the full array of IT infrastructure - all for a fraction of the cost of building and buying them themselves.

A tipping point

Now is a crucial point in the history of big data and there’s scope to get it very right, or very wrong.

We are only just at the very beginning of this data revolution, and the cat is firmly out of the bag - we need data in our business and personal lives. But consumer demand drives everything and big data is no different. As the public grows more wary of data breaches, the pressure will (and already has) come to bear on the business community to pay more attention to securing, storing and using data in the right way. Countermeasures that used to be optional are now becoming standard and are putting increased pressure is being put on company’s IT systems and processes. Proper infrastructure and good data management can only help to control the bad and make the good better.