Imaging running a data warehouse (or live data feeds) for a trading floor such as the New York Stock Exchange or NASDAQ and the immense amount of data flowing through the organization. I’ve recently become familiar with the Online Trading Academy, a highly rated & reviewed learning center where they teach you how to day trade. Trading stocks is very data dependent, and peoples’ livelyhood depends on it, and this data is used on a daily basis in realtime. Can you imagine if bad (or no) data governance is in place and how this could seriously impact a lot of people?  This would literally jeopardize people’s lives as they know it, and one bad move by a trading floor could mean more government regulations and overhead – something that they absolutely do not want.

Consider companies like MorningStar, whose job it is to compile massive amounts of data and make stock ratings and recommendations based on that data.  Can you image the intense amount of data governance and data quality that occurs in that organization? (incidentally, I can related as I saw their Data Quality Director speak at a conference in Vegas on the topic).  If MorningStar has a corrupt set of data that impacts just one small portion of their business, it could literally mean thousands of people get bad advice on their retirement investments.  If that mistake is discovered too late, we are talking about a catastrophic loss of money and potentially the credibility of their company going completely out the door.

I wanted to share this with you, because often times we just think of the worst case scenario being an ‘out of compliance’ finding, but it really can be so much worse than that.  Job loss, income loss, bankruptcy and more all all serious possibilities if data isn’t treated like gold with an organization.

While data governance has shifted from hard copy to software, you should realize that the automated processes will not be sustained on its own. This kind of data management still needs a point of contact to be held accountable to the accuracy and security of the records you keep.  Aside from that, you also want to be sure that all pertinent information can easily be shared with all employees when needed. This may not be a huge problem to small and medium sized businesses, but as your company starts to grow and you start to employ more staff, the records may grow to an extent that managing paper trails may not be the most efficient way to do business.

Since we have been working towards data governance for maximum efficiency, another advantage of being able to implement data management strategies would be the prevention of fraud. This is especially true for any type of company. Managing financial records and information is not limited to sales or banking organizations. Nothing can be worse than losing important financial or accounting information, and if not managed properly, you can end up losing hundreds to thousands of dollars in an instant. Being able to secure your company’s records will play a critical role to your company. The sooner you are able to implement data governance processes, the more time you can devote to other things that will make your business successful.

In order to achieve utmost data quality, it has to start from the bottom-up. An enterprise needs to solidify and integrate this information into one silo and create processes and profiles to collect the data upstream.  Effective Data Governance requires that customer information be available where need be at the most opportune time when it is needed. Data should be centralized in a way that it would be easy for the Data Stewards to look for information and make necessary changes where applicable.

We need to gain a complete and timely understanding of our customers in order to effectively gather information. We need to know which information is needed at a given time. When all the needed information has already been collected, it should be compiled and consolidated into one big effective structure that can easily be accessed by people within the organization.  To reduce operational risks, and to lessen situations where customers get irate because of wrong information given to them, Data Quality should (as always) be observed. This regulating body ensures that various processes are being met in order to provide a flawless data structure.

A unified front end system must be employed by every enterprise if they want to maintain the integrity of their systems and their data.  With the overwhelming amount of information from both internal and external customers, Global organizations should start to see Data Governance not only as a single entity in an enterprise but its vital importance to the welfare of the enterprise as a whole.

A Data Quality Team is tasked to use software, tools, analysis, and historical knowledge to check for errors within the data structure, ensure that the data being entered is foolproof and ultimately, ensures that data updates are correct and can be pushed live after new sets of information come in.

The following are the main focus areas of a data quality program:

• Raw Data
• Classified Data
• Collected Data
• Data needed by data stakeholders and data stewards

Because of the increasing demand for accurate data, another role of the Data Quality Team is to stay in constant communication with the various departments within the organization and come up with quality-related procedures or initiatives that will help the enterprise in a global way.

The Data Quality Team is accountable for the following:

• After gathering all data quality processes/initiatives from the various departments, it is the main responsibility of the Data Quality Team to consolidate these and make it into one large, fluid process that can be applicable to all
• This team needs to ensure that quality-related updates are being disseminated to concerned and affected departments
• This team must ensure that there are no quality gaps, inconsistencies, or  variances where data quality is concerned within the system
• Quality-focused goals should be streamlined to match that of the enterprise’s mission/vision

Feel free to leave thoughts and comments on this below!

1. Start a Data Initiative to fix a few specific problems
Before the words ‘data governance’ are ever uttered, start an initiative to fix a few problems that span the enterprise. Make sure these are problems that bother people so that you’ll get buy-in. As part of this initiative, build a business team that will work on the problems (they’ll later become your Data Stewardship Council).


2. Define the scope of a Data Governance program
You should determine what needs to be governed. Is it Enterprise Data, Master data, or the Data Warehouse?

3. Introduce Data Governance
Make sure that #1 above is a wild success. Then tell everyone, “Lets keep this momentum going. Lets tackle some of the other problems that are bothering everyone”.

4. Form your Data Governance Stewardship Council
It’ll likely be the same people who helped you in your first data initiative. Make sure that every business area affected by your scope is involved.


5. Identify a Chief Data Steward
Read my post “Building your Stewardship Council” and “More on Identifying the Lead Steward” for more information on this.

6. Develop a strategy to work your scope
Remember, this is a program, not a project. You need a plan that is ongoing and factors in Data Quality.

7. Kick-off with the Business and IT
Here are some tips for kicking off your meeting.

8. Work the strategy.
Read this blog for ongoing tips

A Carnival is a monthly publication of links to interesting articles on a topic – in this case data quality. It travels to different blog homes each month. We have some great DQ blogs for the first issue and at the bottom of the post we announce the host of issue #2.

Here are the DQ posts for issue #1. Please visit the blogs highlighted and leave comments to show your support.

An Amazon Post that’s not about the Kindle

Daragh O’Brien is busy in Data Quality. Why are there so many DQ gurus in Ireland? He is part of the The International Association for Information and Data Quality (IAIDQ) and writes the entertaining IQ Trainwrecks blog. He has submitted a great article from there called oops Amazon did it again… about how a search for a Spiderman wristwatch on Amazon returns something unexpected. Say hello to your friendly neighborhood Spiderman.

DQ for Business

Stephen Buck brings data quality solutions to small businesses. He got busy in November, he started the Address Quality blog and posted a long and interesting first entry Why do we insist on capturing unvalidated data??? His company DuoShare has interesting business applications – you can use it in QuickBooks and you can open a shop on their website to use the data quality validation on customer shipping addresses.

SOX SUX – compliance time bombs

You cannot talk about data quality for long without talking about Sarbannes-Oxley, and developing a facial tick. A post from Leon on the SOX first network called Thinking and sending about the emails being compliance time bombs when people put in content that shouldn’t be there.

Why Data Entry Errors are Inevitable

I received some submissions to the Carnival that had nothing to do with DQ and had to reject them. I decided this one should be included as it’s perfect for all the data entry folks out there: The Blasted Crossing brings us Uncluttering your mind with tips on focusing and handling distraction.

Death, Taxes and Lost Data

From the ask the CareerCounselor blog comes the post Three Things are Certain: Death, Taxes, and Lost Data which I thought would be about the episode of Star Trek Next Gen where Data triggers an evacuation of the ship and pisses off with the Enterprise, but it turned out to be full of lost data haikus!

Which DQ Leader joined the Carnival?

I didn’t get any posts from data quality leaders Informatica, DataFlux, Business Objects or Oracle. Whoops, did I call Oracle a data quality leader? I did get introduced to Data Governance & Data Quality Insider from Trillium’s Steve Sarsfield with the post Winners and Losers of Data Quality: Nominations. I’m guessing HP is not a Trillium customer because he lays into them! Posts like this and IQ Trainwrecks will expose companies with DQ problems and encourage better governance.

Join the DQ Aggregator

Beth from Confessions of a database geek submitted a few great articles but I had room for just two. In Update: New Information Quality Aggregator she builds an RSS aggregator of notable data quality blogs out there (and there will be some more from Carnival #1), you can put this feed on your own blog sidebar to promote data quality blogs. She also submitted >Stepping up to Responsibility for Data Migration which is a great post for data integration developers with six tips for better data quality in data migration projects.

Open Source DQ

At Darren Hartford’s Developer Blog you have someone who believes in metadata. From his title I can tell it’s a blog for developers by Darren Hartford. In Data Improvement – Addresses Darren avoids CASS and DQ software fees by standardizing US addresses using the open source Pentaho Kettle tool and USPS Publication 28. I wonder whether this code can be used by the KETTLE community.

Data Quality Gobbledygook

And finally we come to my own submission for the Carnival – Data Quality Gobbledygook, a post that has had better feedback then just about any blog post I’ve done in two years! It’s the first blog post that has been forwarded internally in my company to our information management community with a positive review from my boss.

With great feedback and lots of conversations with data quality bloggers this has been a very good month for me. I look forward to visiting the next Carnival…
Announcing Carnival of Data Quality #2 – Happy New Year
The issue #2 Carnival has a home at the IQ Trainwrecks blog for the last Friday of December. Now that issue #1 has been completed you can use the DQ Carnival submission form for blog posts for issue #2.

The suggested theme of issue #2 is Happy New Year!

We encourage posts looking back at 2007 – highlights, lowlights and major DQ events. Posts looking forward to 2008. What are your predictions, resolutions and trends in DQ for 2008? If you are a DQ vendor what product enhancements do you have planned? If you are a customer what do you want to do with your data in 2008?

Since we are reviewing 2007 and looking forward to 2008 we welcome discussions of things that impact the DQ world such as BI and Enterprise 2.0. You can submit your blog post at any time before December 26.

• Why MDM needs to be deconstructed into discrete functional areas

• Why understanding business requirements means understanding functional requirements

• Why successful MDM depends on a solid data governance framework

The webcast is Today at 2PM EST. To register, click here.

I also wanted to point out a service that I’ve been working with lately on some new data management projects. Medical Billing Service – Medical Electronic Data-Processing is an industry leader in medical billing services. They state that, “Your practice will benefit from our experienced billing staff and IT experts who utilize top-tier technology to increase billing efficiency”. I’ve found their work to be incredibly efficient, and they have fully embraced the data governance and mdm methodologies.

This is an un-sponsored post.

Bible Printing Errors

Humans have had centuries to mess this up and they’ve certainly used this time to make some humorous mistakes. Considering that the text of the Bible is the main product, glaring mistakes such as these are evidence that they didn’t have a handle of their data!

• The dubbed “Wicked Bible”, printed in 1631, omitted the word ‘not’ from the 7th Commandment, and thus it read “Thou shalt commit adultery”. The Bibles were recalled, the printer was fined $400 for the mistake (a lifetime’s wage), and only 11 copies remain today. Current value – $90,000.
• The “Murderer’s Bible” declared “Let the Children be killed” instead of “Let the Children be filled”.
• The “Sin On” Bible reads “Go and sin on more” rather than “Go and sin no more”.

Baseball Card Printing Errors

Baseball cards are no longer the staple toy for young boys that they used to be. Still, it appears that the most valuable cards are the ones that have mistakes (or, for some other reason, are extremely rare). Like the Bible examples above, whats printed on the card is the product – so these companies have a lot of work to do to get their product in shape. It isn’t just older cards that have errors, it seems the new batches have just as many mistakes. Here are some of the most interesting:

• On the back of the 1964 Rookie card of Dave Bennent it says “the 19 year old curve baller is just 18 years old”
• Johnny Lipon’s 1954 Topps card lists his team name as the Baltimore Orioles, he is shown in the big portrait picture wearing a Boston Red Sox hat and the smaller action photo has him wearing a St. Louis Brown hat. Turning the card over to see what team he actually plays for is no help either because the back has him listed with the Chicago White Sox.
• Claude Raymond is seen to have not ‘zipped up’ in both his 1966 and 1967 baseball cards. The reason for the mistake two years running is likely because of the reuse of an images from the previous year’s photoshoot.

Minnesota Mining & Manufacturing

In a similar vein, Minnesota Mining & Manufacturing is a company that promotes risk and failure as they have found that this ultimately leads to innovation. You probably know them as 3M, and you can read their very interesting story here.

I tried to find other examples of where companies have made mistakes that led to valuable and/or collectible products but haven’t found a ton. If you have any fun tid-bits, please share with other readers in the comments!

It is really well written and focuses on data quality, data governance, and data management.

“Data is the foundation of practically everything that goes on in today’s organizations, and so managing the quality of data is reaching “business-critical” status.”