| IntroductionWhen viewed from a high
| |
| | added work towards market share growth.
|
| level, the cost of poor quality data can
| |
| | Scarce human resources are often a
|
| affect a company's bottom-line in two
| |
| | bottleneck towards progress, like running
|
| ways. First, there's the cost of scrap
| |
| | one more marketing campaign, delivering
|
| and rework, and second, missed
| |
| | insight in a product portfolio's
|
| opportunities.An example of scrap and
| |
| | performance, etcetera.2. Information
|
| rework costs might be when an agent errs
| |
| | quality assessment or inspection costs-
|
| in recording a customer's address
| |
| | People spend time in assessment processes
|
| details, and consequently a marketing
| |
| | when they are aware of suspect data
|
| premium is sent to the wrong address.
| |
| | quality; in any database project, each
|
| Later, the customer calls to complain.The
| |
| | and every file of questionable quality
|
| complaint needs to be handled (extra call
| |
| | needs to be inspected for data quality
|
| center time), the address details then
| |
| | problems first.This time is
|
| need to be entered a second time
| |
| | irreplaceable, forever lost and never
|
| (rework), and a second premium needs to
| |
| | recouped in any way. Merely assessing if
|
| be sent. The initial premium is
| |
| | data is of sufficient quality is
|
| scrapped.An example of missed opportunity
| |
| | specialist work. This requires access to
|
| costs might be a credit card that is not
| |
| | scarce resources that are often a
|
| granted because the calculated credit
| |
| | bottleneck towards progress.3.
|
| score (erroneously) falls below the
| |
| | Information quality process improvement
|
| cutoff score, and the customer is
| |
| | and defect prevention costs- Development
|
| rejected. The opportunity to make a sale
| |
| | costs to rework existing front-end
|
| is lost, when marketing costs were
| |
| | applications; data entry applications
|
| already incurred.In this whitepaper, I
| |
| | need to enforce data quality by
|
| attempt to supply a comprehensive list of
| |
| | performing validity checks, and
|
| potential data quality costs.Cost
| |
| | minimizing keystrokes and eye-hand
|
| Categories of Information QualityThe
| |
| | movements. On the basis of usability
|
| costs of data quality can be broken down
| |
| | findings, interface improvements
|
| in 3 categories:1. Immediate costs of
| |
| | invariably lead to both higher efficiency
|
| non-quality data. This happens when the
| |
| | and better data quality.- Management
|
| primary process breaks down as a result
| |
| | attention to redefine accountabilities
|
| of erroneous data. Or, information scrap
| |
| | and monitor improved information quality;
|
| and rework, when immediately apparent
| |
| | steering the organization towards higher
|
| errors or omissions in the data need to
| |
| | data quality requires changing
|
| be circumvented in support of the primary
| |
| | accountabilities and continuously
|
| business process. For example, data entry
| |
| | monitoring improvement. This topic will
|
| of a non-valid ZIP code requires
| |
| | need to stay high on management's agenda
|
| back-office staff to look this up again
| |
| | to create lasting
|
| and correct it before sending out a
| |
| | improvement.ConclusionProblems in data
|
| product.2. Information quality assessment
| |
| | quality often go unnoticed. It can be
|
| or inspection costs. These are costs
| |
| | both a source of process inefficiencies
|
| efforts expended for (re)assuring
| |
| | (timeliness), as well as operational
|
| processes work properly. Every time a
| |
| | costs (direct and indirect losses). In
|
| 'suspect' data source is handled, the
| |
| | neither of these cases is it apparent
|
| time spent to seek reassurance of data
| |
| | that improvement is possible from
|
| quality is an irrecoverable expense.3.
| |
| | enhancing data quality.One of the
|
| Information quality process improvement
| |
| | pernicious consequences of suboptimal
|
| and defect prevention costs. Broken
| |
| | data quality is that the cost of poor
|
| business processes need to be improved to
| |
| | quality data is usually hidden. Lack of
|
| eliminate unnecessary information costs.
| |
| | data quality is not obvious to those not
|
| When a data capture or processing
| |
| | deliberately looking for it. Quantifying
|
| operation malfunctions, it requires
| |
| | costs isn't always easy. What makes the
|
| fixing. This is the long-term investment
| |
| | indirect costs of poor data quality so
|
| needed to avoid further losses.1.
| |
| | pernicious is that the relation between
|
| Immediate costs of non-quality
| |
| | data quality problems and its
|
| dataProcess failureFor example, capturing
| |
| | consequences is non-obvious, and often
|
| erroneous customer data like address,
| |
| | only occurs with a substantial time
|
| contact information, account details.-
| |
| | delay. Therefore, the connection between
|
| Irrecoverable costs; e.g. premiums sent
| |
| | downstream consequences and poor quality
|
| in vain to non-existing customer
| |
| | data is often not made, and the problems
|
| addresses.- Liability and exposure costs;
| |
| | are not attributed to their true
|
| for instance credit risk losses when data
| |
| | cause.The cause of many downstream data
|
| quality problems cause erroneously
| |
| | quality costs can easily remain largely
|
| offering credit to a customer who is not
| |
| | hidden (e.g. data quality), and therefore
|
| considered creditworthy on the basis of
| |
| | insufficiently subject to management
|
| self-supplied information.- Recovery
| |
| | attention and intervention. Also,
|
| costs of unhappy customers; time spent
| |
| | progress after improvement efforts is
|
| handling complaints. Information Scrap
| |
| | gradual, relatively slow, in large part
|
| and Rework- Redundant data handling;
| |
| | 'cultural', and therefore difficult to
|
| because many processes are 'known' to
| |
| | monitor and track.Another, and probably
|
| rely on inaccurate data, it is customary
| |
| | the most significant problem caused by
|
| for front-line and back-office staff to
| |
| | poor-quality information, is that it
|
| maintain little private "lists" of all
| |
| | frustrates the most valuable resource of
|
| sorts. These serve merely as a backup or
| |
| | the company: its employees. Non-quality
|
| improved version of what is available in
| |
| | information prevents knowledge workers
|
| the primary database. Apart from further
| |
| | from performing their job effectively. On
|
| problems like 'maintenance' and
| |
| | top of that, it alienates customers
|
| 'recovery' not being possible for these
| |
| | because of wrong information about them,
|
| private lists, such activities are
| |
| | and to them. Customer data is the raw
|
| redundant, and non-value adding.- Costs
| |
| | material that needs to be managed for
|
| of chasing missing information; a field
| |
| | what it is: a strategic resource.Data
|
| that has not been filled out properly, or
| |
| | quality is far more than accurate data
|
| not at all, needs to be looked up later
| |
| | entry. It stems from monitoring
|
| on in the process. Excess time and costs,
| |
| | downstream data usage, maintaining
|
| inefficiency, and not in the least place
| |
| | comprehensive and up-to-date meta data,
|
| an aggravation factor. Time spent looking
| |
| | and nurturing a corporate culture of
|
| up missing information is not being spent
| |
| | naturally doing things right at the first
|
| servicing the customer better.- Business
| |
| | attempt. Only then will knowledge workers
|
| rework costs; e.g. reissuing a credit
| |
| | learn to expect data quality, and enforce
|
| card that was sent out with a misspelled
| |
| | it because it's the natural thing to do.
|
| customer name.- Workaround costs; when a
| |
| | Letting data quality slide will promote a
|
| primary key is missing or faulty,
| |
| | culture of negligence, and disdain for
|
| laborious fuzzy matches need to be
| |
| | the use of one's most precious assets:
|
| performed to match records. This kind of
| |
| | customer information.The case for
|
| work is challenging, and eats up precious
| |
| | accurate source data is further
|
| time of the most highly skilled database
| |
| | underlined when one realizes that the
|
| workers.- Data verification costs; e.g.
| |
| | source in and of itself does little more
|
| costs of reworking data entry. But also,
| |
| | than support primary processes, which is
|
| analyses by knowledge workers must begin
| |
| | fine. However, the greater value to the
|
| by checking the correctness of data
| |
| | organization comes from enhancing these
|
| available before beginning analysis.-
| |
| | data, from deriving new information from
|
| Program rewrite costs; rewriting programs
| |
| | source data.The investment in improving
|
| that fail to run because of invalid
| |
| | information quality is recouped several
|
| entries found in the data. E.g.:
| |
| | times in decreased costs, and improved
|
| sometimes pre- or post-conversion scripts
| |
| | value of information to accomplish
|
| needed to be written to deal with the
| |
| | strategic business goals.Rapid access to
|
| content of source systems prior to
| |
| | high quality data is the decisive factor
|
| loading in a Data Warehouse environment.-
| |
| | in an organization's ability to assess
|
| Data cleansing and correction costs; when
| |
| | and adapt it's business model to changing
|
| feeds are processed to load into the Data
| |
| | market conditions. As corporations become
|
| Warehouse, these data need to be
| |
| | ever more 'digitized', those that get a
|
| transformed for reasons that stem from
| |
| | grip on their data quality assurance
|
| quality issues. Any data cleansing and
| |
| | processes can reap great rewards. In a
|
| scrubbing that needs to be performed in
| |
| | highly turbulent market this may well be
|
| the ETL process is essentially redundant
| |
| | the critical factor in determining the
|
| and unnecessary insofar this is caused by
| |
| | survivors in a competitive business, and
|
| faulty initial data entry. For example,
| |
| | therefore prove to be ultimately
|
| when a mailing is done on the basis of a
| |
| | priceless.ResourcesLarry P. English
|
| problematic customer file, dedicated
| |
| | (1999) Improving Data Warehouse and
|
| scripts need to be run to deal with the
| |
| | Business Information Quality: Methods for
|
| (known!) errors in the address fields.
| |
| | Reducing Costs and Increasing Profits.
|
| This process needs to be repeated for
| |
| | Wiley, ISBN 0- 471-25383-9Jack E. Olson
|
| every mailing. Since such customer files
| |
| | (2003) Data Quality: the Accuracy
|
| are often shared across departments and
| |
| | Dimension. Morgan Kaufman, ISBN
|
| systems,source changes need to be
| |
| | 1-55860-891-5Sid Adelman, Larissa Moss &
|
| negotiated with all end users of these
| |
| | Majid Abai (2005) Data Strategy. Addison-
|
| data.- Data cleansing software costs;
| |
| | Wesley, ISBN 0-321-24099-5Article
|
| data cleansing software (like Vality,
| |
| | download "How Non-Quality Data Can Cost
|
| Ascential, etc.) is usually very
| |
| | Money"XLNT Consulting - Turning Data Into
|
| expensive. However, there's a tradeoff
| |
| | Dollars.Tom Breur: Biographical SketchTom
|
| between scarce labor doing this 'by
| |
| | Breur is a consultant out of deep passion
|
| hand', and the fact that ETL data quality
| |
| | for his work. He can be profoundly
|
| software to help with such tasks
| |
| | analytic, in his passionate quest to
|
| typically has very high license costs.
| |
| | drive out the deepest business issues and
|
| Purchase may sometimes prove remarkably
| |
| | the nexus point of a business model. It's
|
| economical when related to (often unseen)
| |
| | all about finding where the least effort
|
| labor costs for manually improving data
| |
| | will generate the most results.Once the
|
| quality.Lost and missed opportunity
| |
| | business challenge becomes clear Tom
|
| costs- Lost opportunity costs; when e.g.
| |
| | loves to roll up his sleeves and get his
|
| misspelling customer name on the card
| |
| | 'hands dirty'. Be it data analysis,
|
| causes the customer to not use their card
| |
| | market research, data mining or database
|
| (instead of calling up to complain about
| |
| | work. Once the hands-on work gets
|
| this) the business looses their future
| |
| | started, his eyes begin to flicker, and
|
| revenue.- Missed opportunity costs; when
| |
| | he has a tendency to get carried away.Tom
|
| unhappy customers directly influence
| |
| | has an academic background in Psychology,
|
| their social environment, they generate
| |
| | an education he took up twice. Initially
|
| negative publicity. This will make it
| |
| | he majored in Clinical Psychology (1986),
|
| harder to sell to people in the social
| |
| | years later he went back to college to
|
| network of displeased customers.- Lost
| |
| | study Economic Psychology (1996) with an
|
| shareholder value; information quality
| |
| | emphasis on quantitative methods.Tom is
|
| puts a drain on precious resources
| |
| | fluent in Dutch, English, French and
|
| (scarce database experts), preventing
| |
| | German.
|
| knowledge workers from performing value
| |
| |
|
