The Data Divide: what is it and why closing it matters

With the Great Fragmentation greater than ever, we talk to Splunk’s Technical Chief Mark Woods about harnessing the power of data for positive impact

As leaders from business, government and society convened at this year’s WEF Forum in Davos, the largest in its history, the message was clear – the world is at a critical turning point, and we must all cooperate to find solutions.

Themed ‘Cooperation in a Fragmented World’, the Forum urged public private cooperation in dealing with what has become known as the ‘Great Fragmentation’.

The ‘Great Fragmentation’ is driven by multiple factors – from the environmental collapse that exacerbates social tensions to geopolitical splits that drive a wedge between and within countries. The COVID-19 pandemic and Russia’s invasion of Ukraine have further tested international relations and now the looming global recession tied in with rising energy costs is exacerbating the challenge further.

A severe fragmentation of the global economy after decades of increasing economic integration could reduce global economic output by up to 7%, according to the IMF, but the losses could reach 8-12% in some countries, if technology is also decoupled.

How can we inject greater economic dynamism to start and continue the road to recovery?

Mark Woods, Chief Technical Advisor, EMEA, at Splunk, a data platform that helps organisations turn data into doing, believes the key is investing in people and accelerating technology-led growth – and that rewiring the world for resilience will mean bridging the Data Divide to build more resilient economies.

In this new Data Age, data is produced, consumed and stored at a dizzying pace. The world has witnessed ten-fold growth in total zettabytes of data since 2013, and it’s set to double by 2025.

But there is a disparity between the expanding use of data to create commercial value and the comparatively weak use of data to solve social and environmental challenges such as climate change and global inequality.

We caught up with Mark to discuss the Data Divide, and the challenges and opportunities faced in bridging it.

Everyone talks about the Digital Divide, but what is the Data Divide, and why does it matter?

While the Digital Divide is the gap between those who have access to the internet, computers, skills and services in the digital age and those who do not, the Data Divide describes the disparity between the expanding use of data to create commercial value and the comparatively weak use of data to solve social and environmental challenges.

In the last few years, multiple overlapping crises, including the COVID 19 pandemic, urgent climate challenges and the cost of energy crisis, have driven home to governments how essential data is for policymaking as well as the significant costs of not investing.

The lack of long-term and consistent focus on building the data foundations to solve these broader challenges, has ultimately been one of the key causes of the divide.

Now, staring down the barrel of a challenging period of costly energy, high inflation, fluctuating work patterns and seismic environmental changes, governments are once again turning to data to solve the major societal challenge of the day.

This will take significant effort and a change in policy ethos. But there is a tangible opportunity for change with a clear mandate from citizens and most of the technological building blocks in place.

What can economies gain by bridging the data divide?

By closing the data divide, governments stand to gain economically, both directly and due to efficiency savings. The value of these economic benefits is hard to quantify, but according to Splunk’s recent report  it is estimated at up to 7.19% GDP improvement from open data alone.

Governments can transform their use of data to drive efficiency and improve public services. One example of this in practice includes when New Zealand used AI to transform the child protection system. A predictive model of children’s future lives gave the Ministry an indicative view of a child’s potential situation, enabling more robust cases for early intervention by highlighting impacts as well as avoidable fiscal costs.

More widely, just as the pandemic drove adoption of data solutions, the cost-of-living crisis in Europe is driving innovative data uses to monitor the crisis and derive new policies to address it, particularly within the energy sector.

These policies will only be effective if they address and link the need for data to understand both ongoing efficacy needs and the potential of longer-term transformational solutions.

What are the challenges in bridging the data divide, in using data to solve social and environmental challenges?

It’s not that the data does not exist. There’s just little that’s done outside of specific projects, focused teams (such as national statistic, audit bodies or public observatory teams) or regulated reporting (such as ESMA, national infrastructure directives).

There are several reasons, not least a fundamental lack of broad and flexible access to data, context and insights. Data itself might be captured but remains inaccessible, useless through lack of relevant context or difficult to reuse - which poses immediate problems. But even if data is accessible, we then must consider the ability to utilise and reuse to address these important challenges.  Policies and regulations have not yet created the permissive environment for data sharing and reuse which drives an ethos where it’s easier to ‘say no 50 times, than be punished for one mistake’.

This is further compounded by the lack of consistent systems and the resources, talent and knowledge required to use them, which isn’t always easy to find. When everyone, not just your technology team, is able to read, understand and communicate data in context, they're empowered to make actionable changes (for the better). This data and the context needed to understand it has often already been created by expert operational teams to address the imperative to provide efficient, effective, and secure products and services.

What’s more, too often, organisations prioritise resources towards short-term gain rather than using data to help solve long-term societal problems. Businesses must prioritise investment in data, not just to drive revenue, but to close the data divide – an essential step to boost the overall health of our society and economy and to have clear rules and regulations in place to protect privacy and security.

And what sorts of challenges could we potentially solve or improve if the data divide was bridged?

The COVID-19 pandemic brought into focus the importance of health data, and many governments rapidly developed new tools to harness existing data. Lessons have been learned, and the health arena is likely to see a permanently stronger data focus from governments.

In Singapore, the Housing and Development Board has implemented better home-based monitoring of the elderly with wireless sensors tracking activity and health. It alerts caregivers to any anomalies and provides a safe and cheaper alternative to retirement homes and community-based care.

In addition, data is going to play a key role in managing and mitigating the climate emergency, and governments are already looking at new ways data can be used to monitor and mitigate impacts. From IoT sensor technology, to forecasting and mitigating the effects of climate change, there are plenty of areas where improved government use of data will provide enormous benefit.

In India, for example, the Google Flood Forecasting Initiative is used to provide accurate real-time flood forecasting information and alerts through AI and physics-based modelling for scalable inundation models in real world settings.

What are the main solutions to bridging the data divide?

Public-private partnerships can help close the data divide by bringing together the resources and expertise of both sectors.

Private organisations have seized the wealth of opportunities offered by the explosion of data, leveraging accessible data-mining and analytics technologies to gain vast commercial benefits. The same cannot be said of government and third sector organisations, who have been much slower to grasp the same opportunities. This is not due to a lack of access to the right technologies, as these are now widely available – it is driven by the challenges of legacy infrastructure, data governance, data culture, data literacy and data skills, all of which need to be overcome in order to achieve the available data dividends.

With increasing available data, and technologies to exploit that data, many organisations have developed guidance and best practices for closing the data divide. Knowledge of the barriers and potential solutions they will face has helped governments along this journey. Working together will be the key to bridging the data divide even further.

Improve knowledge and skills around data in a targeted way is also essential. This includes providing education and training programmes that complement existing education pathways, as well as developing resources and materials that are accessible to people with different levels of data literacy. 

Everybody can and should interpret and utilise data to help them make an impact, that doesn’t mean that everyone needs a PhD. Governments should also encourage private sector involvement, which can play an important role by investing in infrastructure and services, as well as through the development of new technologies and business models.

At Splunk we've committed to donating more than US$100 million in software licenses, training and support to non-profit organisations and educational institutions around the world to foster data literacy and action. It's our goal to put the right tools and resources into the hands of the people who are solving the world's biggest problems.


Featured Articles

SAP creates new EMEA region and announces new President

SAP has announced it has appointed a new President for a newly-created EMEA region, aiming to make the most of the opportunities of cloud and AI technology

How SAP is facilitating continuous business transformation

Technology giant SAP has expanded its portfolio with the acquisition of LeanIX, a leader in enterprise architecture management (EAM) software

Siemens and Microsoft: Driving cross-industry AI adoption

To help businesses achieve increased productivity, Siemens and Microsoft are deepening their partnership by showcasing the benefits of generative AI

Sustainability must become central to corporate strategy


The endless benefits of putting your people first

Leadership & Strategy

Working from anywhere: SAP uncovers secret life of employees

Human Capital