Shifting the focus – the Data Driven Organization
As we look into the potentials of implementing AI into our organization, we go through the experimentation stage and the implementation stage and might realize some shortcomings. We have now implemented AI into a business unit, we might even have created a competence centre to support this change. But we might start to realize that there is so much more in data and AI than that actual project, although we have put significant amount of effort into implementing that. We have to find a way to deal with this question on an organizational level instead of a project level. We have to start thinking in the terms of organizational capabilities instead of technical implementations.
This is the stage when we enter into the 4th level of our maturity model: the Data-driven organization
Data-driven organization is a very fancy term and sounds like a progressive and good-for-all solution. The main desire usually comes from leaders who wish their people would ground their decisions to facts and rely on mathematically sound predictions rather than just guess work, or gut feeling. They might want to reduce human bias or improve the connection with reality or they are annoyed by flying half-blind. So, they start to swing the phrase of data-driven like a magic wand and expect everyone to reason more: why do you suggest what you suggest? What is the data that you build your assumptions on and what is the tool you used to make sense of the data, and have good predictions? Questions like:
whom should we promote?
which market should we enter next?
which product line should we invest in?
how should we schedule production?
whom should we assign which incoming mail?
But just saying that we should be data driven is not enough there is significant amount of work especially on the organizational and data capital management sides that are prerequisites for success.
Two approaches to data driven
When we talk about data-driven it is usually ambiguous what we mean, mainly because there are two main approaches to being data driven: being KPI driven or being technology driven.
The KPI driven approach
Key performance indicators (KPIs) and the OKR (Objectives and Key Results) are focusing on what we want to measure. How can we break down our goals into measurable artifacts instead of just wishes? The message of KPI-based data-driven is to find what matters and try to measure it. The emphasis is on KEY performance indicator. Not all indicators are good. It is a significant management effort to find the right KPIs that build a bridge between performance and the impact I want to make. A bridge between actions and desired outcomes. Not all that I can measure are key and often times we don't even measure what the indicator would really be.
This is a top-down approach where technology only plays a role to the extent that we have to have an app where we track the KPIs, but at the end of the day it could be a big whiteboard that gives us a sense of the connection with reality. This does not require AI on its own.
Technology driven approach
The other approach to being data-driven is to bring the most out of the data that we have. To provide everyone with the toolkit with which they can have a better grasp of the width of the information they should deal with. To expand the human information processing capability with the power of data collection and data analysis methods. To give them data enabled suggestion about what they can do next. The very high-end narrative is to provide everyone an oracle who can give insights to the individual that can enhance their decision-making abilities.
This approach usually requires significant investments into both the data capital and technology management and both the organizational skills.
On the organizational side we want people to thrive WITH the power of data. Although the first message is that "don't trust your gut feeling, ask the data!" This can be very restrictive and discouraging. People might end up feeling that data is to either control them or even worse, to replace their expertise rather than being a support in their jobs. Realizing, that using data and data related tools is a special HUMAN skill is a very important point in this journey and upskilling people about how they can be better with data and technology is a significant effort. Especially if we are starting the data-driven narrative with the hidden or explicit agenda, that we want to correct human errors with data we might end up in a very serious push-back, that can be avoided with strategic thinking. Creating a general attitude, leadership requirement and relationship towards data is a significant factor in this.
Suppose we are very successful in creating a buzz around the potentials of data-driven and the opportunities technologies can bring. This is generating a mounting need for accessing data and accessing tools for experimentation. This on the other side puts a heavy pressure on the team providing the data architecture and tools for data analysis. To be profound: when we start to encourage people to make more data-driven decisions, and they find that they don't have access to data, data is not collected, or the data quality is bad they lose their enthusiasm very fast. Thus, this is the stage where the company's wide data capital management becomes a strategic and critical issue.
Data capital management on this level needs to manage the width of data (what do we collect?) the quality of the data (can we use what we collect?) and the interoperability of data (can we connect different data sources, so we analyse them together?). This is the first big step, where the company realizes that data is a strategic asset for them, not just a byproduct of managing the processes that generate money. This is where data engineering becomes a multi-stakeholder task and data architecture usually shifts from data warehouses to data lakes because we realize that the data we collected for a special purpose originally can be used for several other use-cases that we could not have designed the data warehouse structure when we implemented it. But more on this in the "Data" dimension.
Transitioning from implementing AI to Data-driven
When AI shows itself successful in a big project we might say: let's do this on an organizational level, let's provide everyone the capability to use AI enabled tools on the data that matters to them and provide insights to make their jobs better. But as we've seen there is a double challenge we have to manage in the transition: a technical and a cultural one and both of them can sabotage the efforts. If we start pushing people to use data and AI and we can't provide them with the tools, or even worse we provide them with tools that do not work we lose all trust. So, framing the development and transition efforts around the data and technology parts that actually work is a critical strategic choice. At the same time if we start big investments into data capital with data merger, data cleaning or data lake project and nobody uses the data that is managed we quickly find ourselves in an empty space where huge budgets are burned with no immediate effects. This is very easily labelled as L'art pour l'art and these are great candidates to be shut down as cost-cuts and they are harder and harder to defend.
The transformation journey from a local implementation to organizational level capability building is an iterative balancing act. This is the first level, where AI is no longer a project, but an organizational enabler and the focus shifts from technology to the organization. Yet also the first stage where organizational level technical and architectural capabilities are going to define the success of the implementation. This is where Chief Data Officers rise from under IT, where data quality management processes emerge instead of data engineering departments and where data science/AI teams start to contemplate whether to be a centralized competence center or to sit close to business to understand needs better. Data capital management and AI becomes a strategic discussion with general, significant budget and everyone starts to understand that we have to leverage technology.
Risks and constraints of succeeding
This is already a very compelling stage. Most companies we have met up until last year usually aimed at this stage and pushed hard to get there. But striking the balance between culture and technology and initiate an upward going spiral is difficult. Pushing too hard on technology (even if succeeding in making everything measurable with good data) can be a real culture killer without good people management. At the end people may feel like gears monitored 360 degrees transforming into a performance number. Pushing too hard on people might result in long term, general disappointment in technology. Sorry.
But the good news is there are several companies who manage to support their operations with data and technology and are start to thrive. They have better customer insights, supply-chain planning, marketing segmentation, more effective customer service, production scheduling or research processes, more human than administration centric HR, better IT support... They start to use AI for what it really is: a general purpose technology that is implemented in all domains.
The constraint comes with the realization of something that is arriving to everyone this year: these technologies are changing faster than I can implement them! By the time I chose, purchase, integrate I might realize that I should have bought something different. With ChatGPT and generative AI technology every company leader faced with this problem. So, this year we already see a lot more companies who are aiming for the next level: the Adaptive organization.