I have noticed over the last year or two that a lot of the focus for data management, data governance, metadata management, and lineage programs has been focused on new cloud technologies. Examples include – data mesh, data warehouses in the cloud, etc. While this makes absolute sense managing and governing this data from the beginning rather than when you identify issues or have a regulatory finding, in some cases their seems to be an attitude of that is all that is needed. But what about your transactional systems? Shouldn’t we need to manage and govern that data, understand how it flows, and is used by other systems?
Think about the type of systems and data we are talking about. These systems run your business. You may feel that these data assets are fully understood and described. While that may be true of the primary users of the data, what about people downstream who started using this data for say reporting or analytics? What about systems upstream of this critical system feeding it data – do those responsible understand how this system uses it, how critical it is, etc. Do they know to notify someone if they are making changes and who to notify? Do new potential users of this information know how they should treat it?
An additional complicating factor is if these systems are still on older technology with few subject matter experts remaining who know the systems, data, and technology. I would suggest that few if any of these remaining SMEs know all the ways data is passed or used between these critical systems. Migrating the technology might be a partial answer, but do you have the resources and the knowledge to do the migration? These issues continue to grow with time and evolution of your transactional systems.
In the end while I agree that reporting and analytics can be low hanging fruit for the beginning of your journey to being a data driven organization, without considering the critical systems that run your business I’m not sure how you can truly arrive at the desired destination.