Data Governance in the Cloud

It has been a quite a while since I’ve written about Data Governance and quite a lot has changed over the past few years.  From the rise of cloud computing to the unfortunate increase in data breaches, the landscape of data has changed swiftly over time.  Today I wanted to talk a little bit about how data governance is changing and evolving in this new age of cloud computing.

Regulation Mandates

There are regulatory mandates that come into play once a company decides to place data in the cloud, outside of their private server farm.  Depending on the industry, the cloud service provider may be required to have different certifications in place or adhere to specific industry standards.  While this isn’t an issue with the big players such as Amazon Web Services or Microsoft Azure, the smaller players who may be more flexible to meet your needs might not have the required certifications.

Security Consideration

While security is not typically in the domain of data governance, putting data in the cloud means that the data governance team needs to understand the tokenization and encryption in place. No private data should be stored in the cloud in plain-text, but rather it should be replaced with either a token value or strongly encrypted.  This security needs to be noted in the data governance documentation. Because of data governance’s involvement in cloud computing, cloud security firms now have data governance-specific options to help with this securitization and documentation process.

API Management

The cloud makes it easy to launch an API which will allow other enterprise software to easily integrate data into other apps.  In the old days, it would take large teams months of works and stacks of documentation to integrate systems, but now spinning up an API can be done in a week.  Cloud services make this data sharing easy, but API access also means the data can end up in other places and updated by third-parties.  Because of this, APIs and their access rights need to be clearly documented by the data governance team.

The cloud is exciting and is changing the way we all work, mostly for the better.  Because the cloud is almost a given for IT now, strong Data Governance processes and procedures need to be in place to ensure that the valuable data in an organization isn’t mismanaged or abused.

The Importance of Data Governance in the Financial Markets

It occurred to me today that, for some organizations, the repercussions of having a poor data governance program in place could be catastrophic.  I worked in the financial industry doing data governance about 5 years ago (student loans), but for some reason the magnitude of poor data governance for large financial institutions didn’t fully occur to me until recently.

Imaging running a data warehouse (or live data feeds) for a trading floor such as the New York Stock Exchange or NASDAQ and the immense amount of data flowing through the organization. I’ve recently become familiar with the Online Trading Academy, a highly rated & reviewed learning center where they teach you how to day trade. Trading stocks is very data dependent, and peoples’ livelyhood depends on it, and this data is used on a daily basis in realtime. Can you imagine if bad (or no) data governance is in place and how this could seriously impact a lot of people?  This would literally jeopardize people’s lives as they know it, and one bad move by a trading floor could mean more government regulations and overhead – something that they absolutely do not want.

Consider companies like MorningStar, whose job it is to compile massive amounts of data and make stock ratings and recommendations based on that data.  Can you image the intense amount of data governance and data quality that occurs in that organization? (incidentally, I can related as I saw their Data Quality Director speak at a conference in Vegas on the topic).  If MorningStar has a corrupt set of data that impacts just one small portion of their business, it could literally mean thousands of people get bad advice on their retirement investments.  If that mistake is discovered too late, we are talking about a catastrophic loss of money and potentially the credibility of their company going completely out the door.

I wanted to share this with you, because often times we just think of the worst case scenario being an ‘out of compliance’ finding, but it really can be so much worse than that.  Job loss, income loss, bankruptcy and more all all serious possibilities if data isn’t treated like gold with an organization.