“There are three kinds of lies: lies, damned lies, and statistics.” A couple of posts on the American Evaluation Association’s blog Genuine Evaluation have had me thinking about that quote and what it means for the development of court ADR programs.
The quote, of course, refers to the twisting of data to fit one’s argument. Sometimes, though, data is twisted inadvertently, leading to a misinterpretation of outcomes. I recently came across a minor example of this in an AP article that is unfortunately no longer available online, about the first six months of the Foreclosure Mediation Program in Nevada (“Nevada Court: Homeowners Use Foreclosure Mediation Program,” January 19, 2010). The article states that 10% of homeowners facing foreclosure had taken advantage of the program between July and December 2009. This is based upon two pieces of information: almost 3,900 homeowners requested mediation and 38,277 foreclosures were filed during that time period. That sounds straight forward enough. However, a quick check on Nevada’s judiciary web site tells us that only owner-occupied residential mortgages are eligible for the program, and that the number of filings includes commercial and non-owner-occupied properties.
This calls into question the assertion that only 10% of homeowners have taken advantage of the mediation program. A simple misinterpretation of two numbers could easily lead to inaccurate conclusions about the success of the program in recruiting homeowners. This example in particular doesn’t seem to be too harmful. However, misinterpretations or misrepresentations of data could lead to the discontinuance of good programs, or, perhaps worse, the replication of bad ones. The two posts on Genuine Evaluation I linked to above provide just such examples in other fields.
This underscores the importance of getting the data right, both when writing and reading about statistics. Good policy and good programs rely on the correct interpretation and representation of data.
[On a side note, I almost committed the egregious error of spreading false information by attributing the opening quote to Mark Twain. Apparently, he was quoting British Prime Minister Benjamin Disraeli. But even that’s not the whole story. Later research found no evidence of Disraeli saying that and the origin of the quote is still not settled. Another example of the need to check the facts before drawing conclusions.]
**This post has been revised. It previously linked to “Nevada Court: Homeowners Use Foreclosure Mediation Program” on Business Week. It was changed to reflect the expiration of its online availability.
Tags: foreclosure mediation, statistics
Thanks for the shoutout to the Genuine Evaluation Blog! However, it is not the American Evaluation Association’s Blog, but rather a new blog from two of our members, E Jane Davidson and Patricia Rogers.
AEA’s blog is aea365, which is a tip-a-day blog for evaluators. It may be found at http://aea365.org/blog/.
Susan Kistler, AEA Executive Director