Unlocking ‘dark data’ to transform ESG outcomes using AI

Author: Dr. John Bates, CEO of SER

Dr. John Bates, CEO of SEREven today, the vast majority of the knowledge in an organization is locked within static documents, emails, or some other unstructured state. Worse, over half of all enterprise information qualifies as ‘dark data’ in that it is currently undiscoverable by any digital search.

Yet therein often lies rich, untapped environmental, social, and corporate governance (ESG) performance data from indications that valued employees are on the verge of leaving to insights into carbon inefficiency in the supply chain. Dr. John Bates, SER’s AI technologist and CEO, explores AI’s latest potential in distilling, converting, and harnessing this hidden wealth of ESG intelligence.

The modern enterprise is creaking under the weight of its data. Just this year, 120bn terabytes of content will be created, captured, and consumed by organizations – growing by half as much again by 2025. Locked somewhere within that data is a vast opportunity to understand how well those enterprises are performing as responsible employers in terms of their social and corporate governance (ESG) deliverables to which they are increasingly being called to account.

Yet, as things stand, most of that knowledge is impenetrable. A good 80% of the content in an organization currently exists in static documents, emails, or some other unstructured form, making it very difficult to find and combine as a source of decision-supporting insight. Even more galling, as much as 54% of all enterprise data is thought to be ‘dark data’2 – in other words, completely undiscoverable (because it exists only in people’s heads or is filed away in paper form in some dusty basement or off-site storage facility).

But a plethora of next-generation AI tools now promises to unlock and make sense of these previously untapped resources, and this journey could take businesses closer to their ESG goals as they discover a range of subtle insights they were previously unaware of.

What’s possible?

To improve at ESG, employers need to be able to identify and pre-empt critical scenarios, such as underrepresented members of the workforce or high-potential people becoming stressed, demotivated, and leaving the company. Clues to employee burnout, neglect, or unfair treatment may reside within appraisal notes, calendar schedules, or sick leave records.

Another ESG goal might be to discover and address environmentally-inefficient use of resources along the supply chain – from new insights into order duplications, logistics mileage, and excessive energy consumption, which ordinarily would be dispersed across purchase orders, invoices, and delivery notes.

The challenge is not only to capture and structure all of this intelligence digitally and assign to it rich metadata (to aid its discovery), but also to link it in a meaningful way with associated data, and to then harness the latest AI techniques and tools to monitor, cross-analyze and distill meaningful insights from all of those inter-related knowledge assets – to support or trigger targeted actions.

The growing range of discovery tools in today’s AI suite

There are three stages through which companies can apply different forms of AI to move closer to their ESG and wider business transformation (for instance, through improving stakeholder experiences, a honed product or service vision, and improved cost efficiency).

First, AI technology is very effective in pattern matching, and never more so than today, thanks to a wide range of deep learning capabilities – from visual analysis/image recognition to natural language processing (NLP). These can help precisely identify and capture the content through continuous scanning and metadata generation (detailed tagging/indexing of content).

Then comes the application of ‘contextual AI’ to understand the content and how it adds to the company’s intelligence about a given topic. This is about joining the dots between content and related metadata to capture the content context and compare/contrast related information over time.

This builds the ability to understand correlations, trends, and outliers/red flags or untapped opportunities on demand. Through this application of AI, a company might determine the link between a particular manager and colleagues feeling held back or under-developed, for example.

A further opportunity for AI surrounds intelligent content assistants, which is about AI’s role in search and discovery. Think of this as a ChatGPT equivalent for the workplace – a bot that can query an enterprise’s metadata-enabled content to distill insights such as “Show me high-potential individuals in our employment who are not satisfied/showing signs of restlessness.”

Also Read: Risks of Dark Data

Connecting capabilities without closing off future potential

In an ESG context, the opportunity might be to reverse staff attrition and enhance employee wellbeing by boosting fair treatment and targeting new development opportunities; or identify new opportunities to limit carbon emissions across the supply chain. More broadly, it could present the chance to enhance the customer experience or to hone product/service development as new insights are discovered from across helpdesk exchanges, sales/indirect channel feedback, and review forums.

Even just transforming the everyday lives of knowledge workers, who still, on average, spend over a third of their day hunting for information to complete a task (60% of this across more than four different IT systems, according to our analysis), can contribute significantly to their improved wellbeing – by reducing stress and enabling them to complete their work more efficiently and with a greater sense of achievement.

Keeping content infrastructures and platforms as flexible and open as possible will ensure the organization can keep embracing the latest AI advances as these continue to emerge and mature. The rest is down to company culture and the foresight of its management in wanting to be ahead of the curve on ESG both out of a sense of corporate responsibility, and as a means of attracting future talent.

Check Out The New Enterprisetalk Podcast. For more such updates follow us on Google News Enterprisetalk News.

Previous articleOsa Commerce and Acenda Partner to Create a Unified Source of E-commerce Support and Orchestration
Next articleGenai Subsidiary Gets Approval to Participate in Nvidia’s Inception Program and Makes Purchase of Hyperplane AI Compute Hardware
Although often described as a serial entrepreneur or CEO, Dr. John Bates is a former Cambridge Don who gave up the gown in 2000 when he became addicted to starting and growing businesses, often linked to his primary research field of AI, smart algorithms, and intelligent analytics. After more than a decade in the US, in Boston and Silicon Valley, he moved back to London, now the CEO of SER, a leader and major player in enterprise content management headquartered in Bonn, Germany.