Lessons I didn’t read in a Book of Knowledge
Data black markets within an organization drive data policy.
Any organization large enough to have a centralized data source is bound to have a data black market that has sprung up around that source. This is especially true if there aren’t meaningful governance strategies already in place.
Maybe it’s a ‘sideload’ of ‘just a quick analysis of recent trends’ that someone bumps up against the data warehouse data, but it will quickly escalate into ‘all of our membership eligibility needs this Excel spreadsheet that Marcos does.’ Maybe it’s an Access database. Regardless of what it is, it’s sent via email attachments and departmental drives, and sooner or later, it will become a bottleneck for a key decision-maker.
Often the black market is even powerful enough to strangle fledgling data governance efforts. Newly-incepted governance strategies will often slow the data supply chain (much to everyone’s chagrin), and those with the data (power) will halt support of the project due to pressing timelines. Many governance programs have wilted and floundered due to such frustrations at the early stage.
Counteract the damage a data black market can do by instead bring these citizen data developers into the fold. It is vital that consideration on how to integrate governance strategies into all areas where data creation and development occur to be given early in governance projects. An organization needs to ensure that there are appropriate support and cross-coverage for any key data assets.
Data governance requires that business and IT already be effectively communicating about the data.
These programs are not replacements for complete data dictionaries, consistent change management practices, effective metadata tools, and clearly defined issue management procedures. Hiring a data governance director isn’t going to fix any bad blood that is rising between IT and the business, and it’s not going to get your data dictionary written any faster, either (it might add to your analysts’ annoyance, as now they have yet another ‘boss’).
Adding another layer into a contentious relationship is not helpful. Instead, strategize around data change management and data issue management before introducing a large comprehensive data governance program, because your data stewards and champions will naturally arise during that process.
Once there is healthy infrastructure in place that supports meaningful conversations instead of spiraling bickering, then it’s time to introduce formalized data governance programs.
Data projects suffer more from a lack of ‘lessons learned’ exercise.
Many places are starting to try to understand the ‘data lifecycle’ and acknowledge that there are nuances between how we handle, process, talk about, and manage data and how we do the same for software development. This adds to poor habits as data projects are already the projects most likely to skip phases, especially lessons learned. This is most unfortunate, as data projects have the most to gain from serious considerations of their lessons learned.
Data engineering and data pipelines often have a complicated web of cross-departmental initiatives that need to happen to ensure data availability before departments can build new information assets on that data. Understanding the timeline and resource impacts of these cross-departmental support efforts will be key to future successes.
The ‘style’ of technology and tools will always get more interest from stakeholders than the behind the scenes data infrastructure.
Data stakeholders and C-Level executives are far most often more interested in visual representations of data and analytics than having to hear about the infrastructure that makes it happen. The most apt comparison I’ve heard is that your data shop is more like the electric company — people don’t praise it, and they complain when things aren’t working.
As much as they may not want to admit, both big data and analytics are still dependent on the behemoth of the warehouse, and all depend on data governance to be effective
By Jamie Toth, The Somewhat Cyclops on .
Exported from Medium on December 5, 2021.