Value of Verification: Is This Happening To You?

One speaker at ITC 2012 in London spoke of how the false economies of an earlier systems decision came to light during a subsequent data migration exercise. It seems the performance of an underwriting system had been speeded up by disabling its validation. This simple intervention improved performance by 20%, at no cost. Even better, underwriters using the system found they could put whatever they wanted in its fields. Instead of asking the IT team for enhancements to cope with new data items, they simply bent existing fields to new uses. This meant everyone was even happier, not only was the system faster, but development costs went down.

Two or three years later and the need for data migration after integration exposes the folly of this decision. The systems' fields were filled with disparate data that was hard to interpret. Dispensing with quality control brought apparent short-term benefits but created massive problems further down the line.

The company concerned dealt with the problem, bringing its systems discipline back in line and insisting on validation – and thereby data quality. The speaker went on to say that organizations across the industry must hold themselves responsible for the quality of the data they send each other and not try to offload the responsibility on to intermediate handlers of the data. Whatever you do inside the organization may be okay, bit at the interfaces you must have the discipline of standards.

It's interesting that the time delay between the poor decision to unhook validation and the integration blow-up was short – only a couple of years. Since people don't often share war stories like this, you might think data quality issues like this are only relevant to aged systems. But it seems that, perhaps under the pressure to perform cost miracles, and with a lack of direction about the value of data assets, IT folks can use the kinds of strategies we associate with an era of less frequent change and more expensive storage costs.

More important is the message that data quality is today more likely to be a concern of the business than the IT professional. After all, we migrate data because it's of interest to the business, not for the  sheer fun of it. We capture data because it's fundamental to the business – at the end of the day, it is the business.

We need to educate our IT colleagues on the value of data. It's not just “content”. Or rather, “content” is more valuable than such a generic term suggests. This data we generate, store, modify and share is the stuff of business.