Managing your BI data to ensure accuracy
For many companies, making better use of data will be how they differentiate their services in the future.
For retailers, this could be based on smarter algorithms for product recommendations; for manufacturers and logistics companies, it could be more effective supply chains that deliver better results for customers. However, giving people across a business access to data is only half the battle.
Alongside deploying data visualisation and dashboard tools that can make it easier for people to work with data, there's a big issue around governance to consider, as well. This management of data over time is a tough problem that has kept BI from delivering on its full potential.
So how can company IT teams deal with the issues around data governance? Here are four areas to bear in mind:
1. There's no single level of data governance that can be used for everything, all the time.
While it would be great to put one approach to data governance in place for all data across the business, this approach is not practical. Why? Because not all data is created equal.
For example, a marketing analyst may want to get some insight into which channels are performing best as part of a campaign to meet an ad hoc request from the company's management team. This may pull some information from the marketing automation system into a visualisation tool. Would this require the same level of governance as the finance team's reports on company performance across the whole business? Should it require that governance as well, or would that get in the way?
In looking at this problem, IT teams have to balance the speed at which low-level data can be supplied and used against all the necessary data security, protection and management requirements that might be in place. For small decisions like this, strict levels of governance would be overkill.
However, it's important to recognise that once people start making more use of visualisation, it's likely that they will start wanting to create more in-depth reports that make use of data from different systems. At this point, data governance will come into play. As data needs evolve, so too should the level of governance support.
2. As data gets more joined up, data governance has to be joined up too.
To deal with this issue, it's worth identifying where governance is necessary for data and where it is not. The simple way to do this is around data consistency and trust.
Information that might be used across different departments for business metrics is the prime candidate for governance, as it will get used in more places than other departmental data. For instance, companies looking at metrics around how the business is performing can use lead-to-cash as a guide. This takes data from three departments – marketing, sales and finance – and from three different applications – marketing automation, CRM and ERP – to get a figure that can be used to show how effective the business is at selling and earning money on its investments.
Lead-to-cash should provide a reliable and consistent view on performance over time. However, calculating this figure depends on getting a common definition in place on what represents a "lead," as well as recognising revenue coming into the business. Without this agreement, it's very difficult for people to make decisions that are accurate over time. Similarly, any changes in reporting around leads, revenue recognition or sales performance can therefore have an impact on the lead-to-cash figure.
To make data governance easier, one approach is to create data sets that are "certified" as meeting specific standards and can be used for analysis by people across the business. For other analytics needs, other forms of data can be brought in with an appropriate level of understanding around how trustworthy the data is. Layering semantic models on top of data is an effective way to support common definitions that can be shared and reused across different departments.
3. There's a difference between central and local data
Another area that data governance programs have to bear in mind is where and how data sources are stored. Are these central services where data can be controlled, or are they brought in by individuals as and when they need them? Are the sources of data managed by the central IT team, by departments or by those individuals?
Individuals and departments can use their own applications and data without relying on the central IT team for guidance on governance. This can improve decision-making, but the data collected may not get used outside. Equally, it may not be managed to the same level of consistency that central data receives.
To understand this, it's worth looking at how often those data sources are updated and used for reporting. In most cases, departments such as sales and marketing will tend to use data as and when it is needed. This may be during everyday activities or for ad hoc reporting. On the other side, departments such as finance will have set times during each month when reports for accounting and billing are created.
These different models for data can affect how reports are put together. For analytics around sales or marketing, using the latest data can help show progression over time. However, wider metrics such as lead-to-cash can bring together "real-time" data from sales alongside more "fixed" data from finance. Making sure that these metrics are consistent over time can be difficult, while the results can also affect how individuals make decisions.
Equally important is identifying when local data becomes central. An analysis conducted by an individual user may prove to have value for a broader audience, or even the entire organisation. Broader use of that data means a greater need for robust governance in order to avoid conflicting interpretations by a larger audience. In this situation, data that began locally must now fall under central governance policies.
4. Linking data together helps to ensure consistency
One of the big challenges for BI deployments is to build in the appropriate level of governance and consistency around data. Without central control, it becomes a free-for-all around what data is used and when. By contrast, with too much central control, people resort to using their own stored data. This "shadow data" can be out of date and inaccurate, jeopardising the quality of decision making.
However, there are more than these two options available. Rather than having either a centralized or decentralized approach, building up a data network instead can help bridge the gaps involved. This networked data approach relies on linking data sources together and presenting the results as one consistent whole, rather than duplicating data or moving it around. Instead, it provides departments with the ability to access the data, which needs full data governance around it alongside their own sources of information.
The rule of thumb for governance is that business-critical information such as financial data should have full governance processes in place. However, this trusted data can be pulled in at any time for ad hoc reporting or for users to perform their own self-service analysis. By networking data sources together, individuals can use data to meet their own needs without relying on central analysts to share that data with them.
As data plays a more important role across the business, the accuracy of that data will become more important too. From a governance perspective, helping individuals and departments manage their own requirements can help teams make better decisions, but also make them faster.