More and more companies are coming to realise that “Data Management” is playing an evolving role in their day-to day business. But, at present, it is hard to pinpoint a clear, singular definition of this term. When looking up the definition of Data Management one is likely to stumble across different proposals and ideas regarding how to approach Data Management at your company. In this blog we will summarise our key-findings and experiences to provide you with some guidance.
Today, companies are generating more data than ever before. In 2020 up to 2.5 quintillion bytes of new data are being created every single day[1]. To keep up with this huge amount of generated data it needs to be properly managed and stored. In 2006 Clive Humby, an UK Mathematician and architect of Tesco’s Clubcard is credited with coining the phrase: “Data is the new oil. It is valuable, but if unrefined it cannot really be used. It must be changed into gas, plastic, chemicals, etc. to create a valuable entity that drives profitable activity; data must be broken down, analyzed for it to have value.”
This statement should encourage companies to consider their data as an asset. Therefore, proper management is a pre-requisite for both ensuring trust, as well as for incorporation into business processes. To go even further, according to a Gartner survey, nearly 80% of executives agree companies will lose competitive advantage if they do not effectively utilize data, and 49% say data can be used to decrease expenses and create new avenues for innovation.[2] Hence, lacking adequate Data Management could not only lead to disorganised data swamps, poor data quality, and incomplete data sets, but could also limit organisations in the scope of their business analytics and long-term planning. Data has also grown in importance as businesses are subjected to an increasing number of regulatory compliance requirements, including data privacy and protection laws such as GDPR.
Taking this into account, it should become clear why Data Management applies not only to the IT department but represents a structural issue for an entire company. We will do our best to give a high-level overview of which areas to take into consideration when (re-)structuring your company’s Data Management strategy.
“Top-Down” Data Management
Firstly, organisations should work on aligning their Data Strategy with their business strategy. A business strategy refers to the actions and decisions that a company takes to reach its business goals and be competitive in its industry. As data handling becomes more important, it is crucial that data strategy closely supports business strategy, as opposed to being built on “latest and greatest” technology aspirations. Data which is purposefully collected and analysed with respect to the pains your business might have serves the overall scope of your business. As an example, if your business objective is to double sales, your business strategy may be oriented towards building more appealing products. To find out how to optimize the appeal of your products, your data strategy should focus on collecting and analysing data which describes how your customers use and benefit from your products. Do not forget to consider external data as well.
To evaluate and extend your company’s Data Management helps to be aware of which data is used in day-to-day operations, as well as potentially required data for future projects. This is one reason why input from the business departments is crucial. Not only should the IT department be involved in setting up a Data Management structure, but also management and selected business leaders. Jointly they should cooperate on company-wide data oversight structures – you might want to consider creating a steering committee and a Chief Data Officer (CDO) position for this activity. The business provides the input needed to rank and prioritize data-related activities: a method we have seen to work well, both to get started as well as an iteratively repeated activity, is Use Case clustering. Here, “Use Cases” are the data-related activities of concrete business initiatives. By ranking these you prioritize them according to business relevance/value which enables your organization to be data-driven. This compiled list should provide guidance regarding overall data strategy as well as Data Management structure.
Essentially, the setup of “top-down” Data Management is not only relevant for IT, but also impacts the entire enterprise through collaborative future planning.
Data Governance
Data Governance is another central component of Data Management. More than half of organizations, however, lack a formal Data Governance framework.[3] It comprises the processes and responsibilities relevant to the quality and security of the data used in an organisation. These considerations are especially relevant these days due to GDPR regulations.
Data Governance processes and responsibilities entail a number of distinct roles. These roles, such as data steward or data owner, are clearly defined to effectively ensure Data Governance. When setting up a framework, having a clear definition of Data Governance is only the first step. Equally important is implementing the framework into daily business. The real challenge here is for the processes to be used and lived, and roles to be owned. Gartner analyst Nick Heudecker estimates that 85% of data projects fail[4] due to difficulty in culture and challenges in change management, as well as management blocking progress. These statistics show that top-down guidance and support is key. If management sets a good example, others in their organisation are more likely to follow.
Data Leveraging
An additionally important task in working towards true Data Management is making data easily accessible to users. You can compare this to logistics: the required material (data) should be available in sufficient volume, necessary quality, via the correct channel/tool at the needed time. This is not only true for BI and reporting. Data Scientists often find themselves spending a huge amount of time (up to 60% of their time according to a Data Science report[5]) in allocating the right data for analysis, whereas they could preferably go straight to digging for insights.
The key here is efficiency and convenience: to achieve these, enabling self-service is the way to go. This entails building data infrastructure which enables, for example, Data Scientists to retrieve the right data in the right format for their analysis. There are plenty of ways to make this possible – be it via data lakes, catalogues, repositories, warehouses and probably more – this is individual to every company and their situation. You need to find out what works best for your organisation.
Together with proper Data Governance you can make effortless accessible data for authorized users happen. Plus, with accessibility comes easier collaboration for your team.
Data Source Integration
Enabling self-service of data is already a big efficiency improvement. Nevertheless, companies usually deal with a huge diversity of data types and sources, which can also be extremely dynamic, updating often. Hence, flexibility is hugely advantageous.
It is easy to underestimate the work required to correctly manage Data Source Integration. For a given project, the effort spent on integrating data sources and formats can represent 60% of the entire effort[6].
The key here is to reduce the complexity of data integration. Tools can help with increasing flexibility. If you are not specialized in building system integrations, and you have other business focusses, we recommend using standard solution systems such as Ab Initio, Cloudera, Cloud native services, etc. In-house integration solutions are usually higher maintenance and changes in the data system landscape requires too much effort.
Benefits of Proper Data Management
To summarize, for one to gain value from their data it needs to be managed (or, along the lines of the oil analogy, refined) accordingly. A well-executed Data Management strategy can boost your entire business, not only IT.
Data Management can help companies to gain a competitive edge over their business rivals, both by improving operational effectiveness and enabling better decision-making. Organisations which make extensive use of their data can also become more agile, making it possible to spot market trends and quickly adjust towards taking advantage of them.
Ultimately, a solid approach to Data Management results in more effective use of data, a better understanding of customers, as well as value creation through providing new (data-) products and offerings. This can open up new sources of revenue and game-changing opportunities in an increasingly competitive business world.
Links, References
[1] https://www.dihuni.com/2020/04/10/every-day-big-data-statistics-2-5-quintillion-bytes-of-data-created-daily/#:~:text=Every%20Day%20Big%20Data%20Statistics%20%E2%80%93%202.5%20Quintillion%20Bytes%20of%20Data%20Created%20Daily,-This%20post%20was
[2] https://www.gartner.com/en/newsroom/press-releases/2019-11-7-gartner-says-data-and-cyber-related-risks-remain-top-worries-for-audit-executives
[3] https://www.gartner.com/en/newsroom/press-releases/2019-11-7-gartner-says-data-and-cyber-related-risks-remain-top-worries-for-audit-executives
[4] https://www.techrepublic.com/article/85-of-big-data-projects-fail-but-your-developers-can-help-yours-succeed/
[5] https://visit.figure-eight.com/data-science-report.html
[6]https://www.researchgate.net/publication/293299036_Integrating_data_sources_from_different_development_environments_An_E-LT_approach
