Data is your base component. The building block on which the rest of your information and insight stands. We still don’t know how to use data properly. We use it only for indicating past performance through management reports, scorecards and dashboards.
There is a difference between data-driven and data-enables organization. Data-drive organization is the one that base their activities on data or insights from data, they are data companies to some degree, data is their major asset. Data enabled companies are the one that use data as enablers for transformation of their business. They can use data to better utilize their assets and to support their wider business strategy.
Some of the important terms of data-driven journey are:
- Assurance – activities to measure confidence in a given process, framework or data set
- Big data
- Data architecture – a discipline focused on the models and policies that describe how data is structured, looked after and used
- Data cleansing
- Data dictionary – a catalogue and definition of all data elements
- Data governance
- Data lake
- Data lineage – describes where the data comes from, what happens to it and where it moves over time, often mapped between systems, applications or data stores
- Data migration
- Data warehouse
- Digital – the electronic technology that generates and processes data
- Enterprise architecture – made up of four architectures: application, business data and systems. This is practice for analyzing, designing, planning and implementing enterprise wide changes.
- Information architecture
- Master data – a single source of common data used across multiple processes
- Master data management
- Meta data – data that provides information about other data; such as how long it is valid for, where, when and how it was created.
For an individual organization to undergo transformation of any kind there are risks and costs that are perceived to be out weighted by the benefits of the end state after the transformation. Transformation is painful, costly, requires change and needs great leadership. It is often easy for organization to talk convincingly and even passionately about transformation without embracing it or delivering the transformation end state. During any kind of transformation, data is a useful tool to help with it. It is used to build a business case.
Digitization – this is a conversion of analogue information into digital form. It is the starting point of digital journey.
Digitalization – is the actual process of technologically induced change. It is use of digital technologies to change a business model. The use of new platforms to change operating models and drive new revenue or operational efficiencies. In this process you are still really focusing on the tools rather than taking a more holistic approach.
“Digitization has enabled the process of digitalization, which resulted in stronger opportunities to transform and change existing business models. Digitization (the conversion), digitalization (the process) and the digital transformation (the effect) therefore accelerate and illuminate the already existing and ongoing horizontal and global processes of change in society.”
Digital projects are not digital business. Sometimes digital is not delivering the business outcome, the transformational outcomes, because it is still focus on a technology digital transformation rather than a data digital transformation. Technology in itself does not deliver value, it is how that technology is used, that is important.
Digital business also requires new set of attributes and skills that allow you to operate successfully in a continuously changing world, like more frequent complex decision making, continuous problem solving, rapid pattern recognition and exception handling. Companies have to transform their cultures and data architectures to adapt to new technologies, new types of data and new governance and compliance regulations. In the future it will be less about capital to buy digital technologies and more about the talent and skills to harness the power in the data.
Hurdles in organization when it comes to data:
- You have multiple data sources, but no control over them.
- You have different versions of data.
- Your tools seem to work against you, not for you.
- There is a lack of trust in the data.
- You don’t have clear roles and responsibilities around data.
- There are problems with the scale of data.
- You have integration complexity.
- A lack of governance.
- Unpredictability of data and availability.
- There is a need to work at different levels with the data.
Drivers that are pushing organizations into data-driven transformation are coming from different sources: competitive advantage, customer service and personalization, operational efficiency, regulatory pressures and requirements or other forms of transformation didn’t deliver required business outcomes.
In some ways the data is new oil analogy does work. The combustion engine is the platform, the technology, the petrol is the data. Put the data into technology and deliver right spark (the people) and energy is released.
When we are looking at data to enable our business, we should be careful not to fall into trap that businesses have fall into in 1980s, when IT was seen as magic tools, and businesses were focusing on what technology can do, instead of how will they use it to support their businesses. It could happen with data too, instead of connecting its use with business, companies can fall into trap of focusing on data only. Business should drive data-enabled transformation. Data enabling an organization involved the data leaders listening to the needs and the questions that the business is asking.
The business use data and insight from data to help shape and make decisions about the future end state vision. Once a future end state vision has been developed the technological requirements, people requirements and process changes are defined and the business transformation is delivered to reach that end state vision. It is triangle people, process and technology, supported with data that are building blocks of business ecosystem.
Data literacy is becoming game changer in today’s environment. It is the ability to understand and communicate data as information, focusing on the competencies involved in working with the data. In data enabled business, we have data specialist on one side, data aware people on other side and so-called data citizens between them.
We have four layers in DIKW pyramid in business: data, information, knowledge and wisdom. Data literacy sits across the top end of data layer, through information layer into the lower parts of knowledge layer. Information layer is about three Cs (collated, curated and contextual data).
Layer 1 – data – is about raw data, about operational technology.
Layer 2 – information, 3 Cs – this layer is managed.
Layer 3 – knowledge – where subject matter experts sits, those with domain knowledge.
Layer 4 – wisdom – where insights are being used to create body of wisdom to run the business.
Important element of data literacy is ability to tell story from the data. The story is drawn out by subject matter experts who is data literate. Four enablers of data literacy are: easy access to data leadership onboarding a data culture, a platform for sharing data, most importantly trusted, governed and consistent data and critical thinking.
All companies have data ecosystem. It consists of business, operational technology, data technology and people.
- Business: need, leadership, culture, data literacy.
- Operational technology: data collection, data access, data flow, security and data storage.
- Data technology: data governance, data analytics, data management, data ETL, data storage, data visualization and data science.
- People: processes, data literacy, location, tools and skills.
In practice transformation may be a series of smaller changes that collectively deliver the transformation. What’s important is that they are all heading in the same direction, each of these individual changes need to be adopted. In order to do it properly, you need business to lead transformation, you need to understand difference between digital, technology and data transformation and you need to understand data ecosystem. You need to set up proper team that will design, drive and deliver the transformation – mix of business, domain experts, technology and data and some visionaries in this blend of skills. There are two transformations, disruptive that is changing business models and innovation that is changing only the way you do things in the same business model.
The organization that best adopt transformation are those that start the process expecting things to be different at the end of process. They may not fully understand the exact shape of things to come, but they are prepared to trust the process.
Blockers of transformation:
- Failure of adoption
- Not having a clear vision of end state
- Lack of adequate funding
- The wrong people
- The wrong skills
- No culture of transformation or no data culture
- Transformation is too hard for the organization
- Disruption elsewhere: internal or external
First step in transformation journey is to do a data maturity assessment, to help you define starting point. It is also helpful to do it, because you can use this baseline to show progress in later phases, with stakeholders. When you conduct your maturity model, have in mind that every interaction is a chance to engage, not all of them will be supporters, if somebody is positive, don’t assume it will stay that way, take every opportunity to change people’s attitude to data.
Main elements of the maturity model are:
- Corporate governance
- Leadership and sponsorship
- Framework, process and tools
- Information risk
- Organization, roles and responsibility
- Information guardianship behavior
When marking scale on which company is, we are using marks:
- 0 – unaware – old unknowns
- 1- aware
- 2 – reactive
- 3 – proactive – things like master data management programs are started and governance of unstructured data begins to emerge
- 4 – managed
- 5 – optimized – nearly all information assets are inventoried, including knowing external sources, information advocated across the business, enablement and value generation. True partnerships with customers and suppliers are embedded and the data landscape is trusted and agile.
Strategy – before putting data strategy in place, you need to know what your business strategy is and align data strategy with it. You need to understand also where on risk adverse versus value-add scale the organization is. And then you need to estimate how this is reflected in your data strategy. You have to include work with suppliers and partners into it. Timing is everything. Strategy needs to incorporate external governance and regulation and it needs to be communicated properly.
Corporate governance – is a company way of checking – just in case. It is needed to set the rules of engagement. You need to check if it already exists. Look for risk-framework and decision-making meetings. You need to check also where data governance is wrapped in – check if it is in IT governance.
Leadership and sponsorship – companies have lots of different priorities constantly competing for attention. You just have to make sure you are aware of where the data sits in this sea of noise.
Framework, process and tools – framework is how your data space hangs together. It is vital to make sure you have the ability to discuss important issues at the right level and most important actually make decisions. You are looking to check if you have the right people in the right place to make the right decisions in a timely fashion. Another area you have to check is how information architecture impacts the enterprise architecture and vice-versa.
Policies – it is not only about policies, it is about all documents, that cover standards, procedures, guidelines and whatever else company uses to make sure it gives clear instructions to everyone who works with them and around them. They can’t be longer than 5 pages. You look for volume of documents, what kind they are, where they are stored, if they are used. Check how long it was since they were reviewed.
Information risk – you should talk about risks not in a sensational way and you should understand levels of them and how do they fit into operational model. Look for ownership of them.
Architecture – some organizations believe that they already have this sorted but it is easy to confuse data architecture as defined by TOGAF, which is part of the enterprise architecture framework for information architecture – that itself is concerned with the information map of your organization. It is about how information flows through an organization, rather than what data sits in what IT system. But have in mind not to go too much into details. It is about balance between details and still being able to operate at a manageable level. Think granularity, distribution, latency and physicality of data. Also check if you are buying data from external sources.
Organization, roles and responsibility – a RACI (responsible, accountable, consulting and informed) model can be used to describe participation by various roles in competing tasks or deliverables. It is especially important to get it right in cross-functional processes.
Skills – if your HR department has any kind of skills matric, database, training programs or record of skills and experience, use it.
Metrics – people do what gets measured. People brain can only concentrate on five things at the time, so don’t try to measure to many things. Look for links between data measurement and the business outcome. Making improvements in your data is only useful if it delivers real business benefits.
Behavior – do words match behavior. It is about understanding the culture. Looking if organization values data. If they spend money on data, if they are discussing about data.
Technology – it is about understanding the technology environment. Being careful with funds, since there is a lot of investment already placed in this area and you need to protect it if possible.
Data maturity assessment can be time intensive, especially if you do it for the first time. Information gathering should be face to face if possible. Be transparent what do you want information for and what you are going to do with it. When you are marking areas, try to get as much arguments about a mark and if possible common agreement of a score. Some areas are subjective so you want be able to get tangible evidence. You can use radar diagram for presentation of your assessment. Put your ambition (expected state) and current state status on diagram. Baseline maturity assessment will also be used as part of your governance and assurance activities later.
Carruthers and Jackson data model consist of:
Proper planning prevents poor performance. Focus is on driving an improvement in the quality of decisions. Every small decision has a cumulative effect. Identify the critical path items that must happen in a specific order and then look at what else can take place in parallel. You should work in agile fashion. You need to prioritize. You can’t do anything at once. Think about yours and your team capacity. Start with strategy.
By looking at strategy, governance and risk you are definitely getting a solid foundation for the transformation you are taking your company on. Central pillar to direct and support the data-driven business transformation is a data strategy. This is a document that sets out the vision of the end state of the data transformation and provides the route map to get there. There should be a document on a business level that describes business transformation and how it will be driven by data and what will be the outcome. It should cover all three elements: people, processes and technology and all operational, support, service, back-office functions and supply chain, sales, marketing, compliance and risk. Data strategy and business goals should be aligned and inputs from both of them drive data-driven transformation.
Three should be three concurrent tracks, with IDS (immediate data strategy) and TDS (target data strategy) being main tracks and UDS (urgent data strategy), being in place in order to tackle urgent cases. IDS start the cultural change towards being data driven and data centric. TDS is intended to deliver a more strategic and complex end state, that is only achievable over longer period of time. Even if all three tracks are concurrent, the level of activity in each will change over time.
And when we talk about timeline, we have stages of evolution:
- T1 – at this point the data strategy is early in its delivery. Most of the activity is on the UDS. Some initial data governance, data lineage, reducing the over dependence on spreadsheet and shadow IT, tackling the high-profile burning issues. TDS in this stage is in communication mode.
- T2 – at this stage the UDS is winding down in activity and the focus will be shifting to the more structured and strategic IDS. New methods of data integration, data transformation, data storage and analytics. TDS in this stage is in trying and sometimes failing mode.
- T3 – by this stage UDS should have been completed. Data strategy at this stage will be disruptive. Spending will be at his highest.
- T4 – this is crucial stage of data strategy. And it should be starting to deliver significant data-driven business transformation. It may require the greatest skill and business change management.
- T5 – this is peak business transformation. Data culture should be integrated into business and at the heart of decision making and operations.
Data strategy should:
- Be owned by the CDO (chief data officer)
- Be endorsed and supported by the board
- Be published and shared
- Be in a form
- Be organic document with integrated flexibility to change
- Be aligned with business goals
- Address people, processes and technology
- Describe the end state of dynamic data-driven transformation
- Have technical document being placed in appendices
Your UDS is a reaction to what you face, when you walk into company.
Your IDS need to address:
- Stabilization and rationalization of existing data environment
- Data culture and governance
- Existing and immediate data and IT development initiatives
- Data exploitation and integration
- Data performance, quality, integrity, assurance and provenance
- Data security
We have to be careful how we approach data-driven transformation and how we tackle digital transformation hype. We have been here before. If we have a simple model of organization and the broader shared-enterprise within which it operates, we can divide it into:
- Inside-in: internal workings of organization
- Inside-out: presentation of organization to outside world
- Outside-in: view of outside world of organization
- Outside-out: overall enterprise-story, within the further context of the wider world.
At the inside-in level, probably classic early example of digital transformation is business-process reengineering. Outside-out level was covered with rise of web-technologies. Outside-in with the rise of so-called digital transformation. All of those processes were presented as deus ex machina, that will solve all the problems. But we need to do it properly, we need to acknowledge Conway’s Law and we need to change a way the organization communicates with itself.
When you run workshops try to limit their number and use them to explain why you want a change and what is it you are hoping to achieve. Identify in all areas of data model three categories: pain points, initial changes and ambition. What is written in the ambition column provides the details for target strategy.
You need to write down vision and have it supported by up to three statements.
If you are supporting self-service model, make sure that data quality and control are in place. Areas that you should also cover inside data strategy are data science ambitions and data information ethics.
In order to help people, do their job easier, you should have clear corporate governance. Check how it is delivered, used and understood within an organization. You should run audit checks, see if governance model is proper and how it is impacting on data governance. You should also check if governance is producing expected results. You can use color-coded system, like red, amber and green to show different levels of maturity and compliance across the organization.
“Centered round clear governance that establishes data ownership, quality standard, regulatory compliance and oversight of value-driven exploitation, the foundation will see the implementation of clear strategy for managing data flows through the organization. Authoritative sources for customer and operational data, a single data model and a clear strategy for acquisition of data sit at the core of an effective data strategy.”
Data strategy on its own doesn’t add value. It does not exist in a vacuum. It must be embedded within the overall business strategy. The culture of the organization must embrace data as an asset alongside other assets that they more readily recognize.
When we have a clear picture about strategy, we should have a clear picture about risks, especially information risks. In order to really understand them, you should look at number of elements, like:
- Clear short description of them
- Early warning indicators
- Risk assessment, safety, performance, finance and reputation (political) overall
- Risk assessment rationale – why?
- Existing controls, causes and consequences
- Improvement actions
A good risk assessment is a living, breathing document.
Skills, behavior and leadership form the people-focused elements. The main challenge is how we attract, build, position, deploy and retain teams within a market that is still shaping itself. Market alignment is key. If all actors share a common view and a common language, it will be far easier to build, attract and identify roles and skills within data.
Another challenge will be also that profiles of needed people change during the development. The initial problem solvers don’t tend to wait to stick around when it becomes business as usual. This is also why documentation must be top notch and fully agreed.
Have in mind that organization will need to adjust to differences between operational IT and data, reporting and analytics IT.
When you are building teams and skill sets, have in mind also external roles that impact data like suppliers, consumers, modifiers and accessors. Organization can also think about using external resources in the first phase. In the build phase change management is one of the prime skills needed. And even if agile approach is needed for dynamic development, project managers are needed in build phase. In build phase data teams should have three core elements: data delivery, data exploitation and data governance. You should also establish framework for demand management. And you should not forget about training. It is vital to start building data skills across the business during the build phase.
Don’t assume that what works at the beginning of the process is what will continue to work as the company evolves into a great data-enabled company.
Check people behavior. It is important to distinguish between culture, values, ethos and behaviors. We are looking for behavior that shows data value. Appreciating that data have value. But you also should look for behavior that is valuing data. Valuing data is when individuals make decisions, or take actions, based on data. But how often do we see business fail to take this action? Perhaps because they are too busy or it would disrupt “the way they have always done it”.
Data-mature companies that have undergone a data driven transformation demonstrate collaborative behaviors. Immature are afraid of collaboration and openness since they don’t understand their data. Mature ones also tend to be more innovative. Not to mention open and ethical.
With leadership you have three positions. Leadership you get, the one that you project and the one you deserve. Trying to implement an organizational wide data-driven business transformation without committed leadership and sponsorship in place puts you in an incredibly difficult position. Even worse is if you end up with a data villain sitting on the board, who could be incredibly damaging to the program overall. For each stakeholder you need a clear understanding whether he/she is supportive or not. And when you talk with senior stakeholder make sure your focus is on a big picture.
With the method we incorporate the organization, framework and policy elements. Secret for success is to group the data engineering team with the business intelligence and analytics function, along with the insight and science team. Moving from the start state of maturity to the desired end state will inevitably require some decisions about the roles and responsibilities and organizational design. For an organization that is starting on a data-driven transformation from a very immature state it may be favorable to build a centralized data organization. For some more mature, you can choose decentralized version. But you need to have central function that is focus in your data organization. This could be CDO.
To the business there is one data team. Getting the right information to the right place at the right time is the data team’s responsibility, so the rest of the business can use it. Talking about citizen roles in organization is more and more important, inside organization you will find people that will be data champions and will look for data driven activities. You have to find balance between data communities being linked together and clear role and responsibilities setup.
Key part of the framework is understanding how all the different pieces of your data organization make decisions. It is about the data lifecycle and operation in a company. Initial starting point is to put data council or information steering group. This will be the highest decision-making body across the organization. It should have members from all important data domains.
Policies enable efficiencies by providing guidelines to data users around data availability, usage and sharing. You have to make sure that you have policies that work and that everybody can access them. When you putting them together, try to keep the level of detail appropriate. They should be simple, relevant and consistent. With policies you should cover:
- Policy for policies
- Data protection
- Data science
They should be factored into framework.
It is about information architecture, metrics and technology.
Data architecture is composed of models, policies, rules and standards that govern which data is collected, how it is stored and arranged and, basically, how it uses this data. Data architecture is where data sits in a system. Information architecture is about information flow. It is important that the corporate information model reflects the organization and can be recognized and related to by business at all levels. On the highest level we should not have more than ten different data domains. Once you have your big domains, you can start to look at how the data flows through that and at what points that data changes into information and beyond.
The definition of the conceptual information model, decomposed to the appropriate level, provides the foundation for the information asset register, a common representation of the corporate information and data along with definitions, ownership and so on. The software can keep your data lineage up to date in a simpler fashion, but it does not automatically keep your assets owners up to date. Information and data architecture allow you to define, catalogue and share the structures and associated definitions that enable integration and sharing of trusted data sets.
Metrics are an essential part of assessing progress towards your goals by providing information about what is going on. The measures should be part of your assurance activities. You have two types of metrics: early warning indicators and the measurement of your progress.
Some metrics to estimate your data value are:
To apply the knowledge from data, organizations need to bring knowledge and insights production closer to the business, where deeper and adhoc questions are being asked. Often these questions are answered with gut feel and “how we do it here” approach – because it takes so much time to get answer from specialist. We should be seeking compromise between emotion and data, technology and experience, a more data informed approach.
Don’t invest heavily on the perfect solution, perfection is a disease.
The secret of change is to focus all of your energy not on fighting the old, but building the new. The beginning of wisdom is a definition of terms (data dictionaries), know thyself (maturity assessment), let him that would move the world, first move himself (lead by example). Someone has to lead the change. We should really focus on the value. We can begin with the risk, since it activates resources, but move through it quickly and invest into value, since that really excites people. Use advocates, people listen to people they know and trust. An advocate is someone who shares your vision, believes in the future you are creating and want to work towards that future. Don’t assume that everyone learns, communicates or engages in the same way; people absorb information differently. Remove obstacles to help the advocates on their journey.
Concept of data-driven business transformation. It has start, middle, but not the end. True data-driven businesses don’t stop. The present end state is limited by our current imagination. We need to create a culture of D3. A culture that sets the corporate state of mind into continuous improvement, innovation and transformation. Mindset of D3 is:
- Never satisfied
- Always questioning
- Always constructively challenging
- Always believing that processes and decision making can be better
- Always believing that the customer can be given better service, better products
- Fast moving
- Capable of pivoting
- Achieving results on minimal spend
- Celebrating the successes of the journey
A startup culture is a workplace environment that values creative problem solving, open communication and a flat hierarchy. Kaizen is an approach to creating continuous improvement based on the idea, that small ongoing positive changes can reap major improvements. Ten principles that address the Kaizen mindset are commonly referenced as core to the philosophy:
- Let go of assumptions
- Be proactive about problem solving
- Don’t accept the status quo
- Let go of perfectionism and take an attitude of iterative, adaptive change
- Look for solutions as you find mistakes
- Create an environment in which everyone feels empowered to contribute
- Don’t accept the obvious issue; instead, ask “why” five times to get to the root cause
- Cull information and opinions from multiple people
- Use creativity to find low-cost, small improvements
- Never stop improving
To truly land a D3 an organization needs to restructure and respect their people along the lines of the Kaizen methodology, behave like a start-up and develop their own version of this culture.
Tendayi Viki identified three human barriers to transformation: inertia or tendency for people to do nothing or remain unchanged, doubt and cynicism. People who experience the first two types of human reaction to transformation can be persuaded by good leadership. Those that fall into the type of “cynic” need to be addressed head on, they are either with the transformation or against the transformation.
To achieve D3 an organization needs to be less project driven and more driven by continuous improvement and agility to respond to business needs and outcomes. Data-driven business transformation should not be confused with research and development.
Another consideration in new approaches is role of DataOps (methodology similar to DevOps in software development). DevOps is the combination of cultural philosophies, practices and tools that increases an organization’s ability to deliver applications and services at high velocity.
Tariq Bhatti proposed that DataOps is focused on six points:
- Governance and education
- Process and automation
- Focus on data
“DataOps are process-oriented solution that would revolves around collaboration between operations (the business), governance and technology to bring data solutions that are integrated and automated for the business as well as delivering Value.”
DataOps works is focused on: business understanding, data acquisition and understanding, data modelling and deployment.
For technology in an organization to support D3 it should be flexible, responsive, forward looking and capable. Open architecture is a type of such technology.
Data-driven business transformation is sometimes wrapped up in the story of the 4th industrial revolution, or digital-driven transformation or simply digital transformation.
Seven elements do data-driven business transformation are:
- Change should be dynamic and continuous
- It will deliver a fundamental shift in culture
- It will need a particular form, style and leadership
- Will move away from focusing on technology and digital with a growing focus on data
- Will need right skills
- Will demand from organizations to know their starting point
- It will be about people
will be using data to drive change, transform and build the future.
 Shahyan Khan in the book on page 7
 Kevin Fletcher in the book on page 137
 Tariq Bhatti in the book on page 241