*Or what I learned when I was investigating something else
I was enjoying a rare vacation from "the bubble" in Mexico when the BusinessWeek article by Peter Coy, which discussed my work on MLS accuracy, came out–and I did not have the opportunity to see it. Now that I have, I stand by both my comments and my data provided to BusinessWeek. I would also like to lay out for my readers what I explained to BusinessWeek–i.e. the difficulty with relying on data from various MLS services. I shared with BusinessWeek the results of my studies on the reports of three local MLS services: Phoenix, Tucson and Las Vegas. It appears to me that "adjustments" in MLS data may be indicative of errors in bookkeeping, and there may be an excessive dependence by consumers and Wall Street on the accuracy of this data. Here are the results of my studies:
The source of my data for Phoenix is the ARMLS. Assembling the data was a rather tedious project, but if you care to follow the link to their website, you will see a list of reports. I opened all of them and entered the sales and listing data into Excel. I tried to "balance" the data, much as one might in an accounting statement. I assumed that the following formula was essentially correct:
"THIS MONTH’S INVENTORY" = "LAST MONTH’S END-OF-MONTH LISTINGS" – "THIS MONTH’S SALES " + "THIS MONTH’S NEW LISTINGS" – "DELISTINGS*"
(*"Delistings" is a term coined by our reader "Rebel," who clued me into the accuracy issue. Delistings would be estimated cancelled/expired and temporarily-off-market [TOM] listings. While different MLS services around the country vary in the amount of info they give the public, I know of none that report this information on a regular basis. While I have seen claims as to the "actual" number of delistings in Phoenix, these figures do not jive with the ARMLS reports.)
Admittedly, there are a couple of problems with using this formula. One is that nailing down MLS numbers is like trying to hit a moving target. There is no single source for data entry into the MLS–the data base updates as realtors update listings. Any report is a "snapshot" of the time at which it was done, and two reports of the same time period can look markedly different.
The other problem with the formula is the status of pending sales. When a listing is marked as "pending" it leaves inventory, and will not show up under "sales" until closing. This lag can last for months. In spite of these problems, though, as the ARMLS seemed to be consistent in their methodology, and this lag seems to shift only gradually over time, I believe the result here, while imperfect, is accurate enough for our purposes. This study will allow us to look at the reports of the ARMLS and assess their merits in general terms.
You can see my Excel chart by clicking here.
One of our sources has been pointing out to me how more homes were leaving the market due to "delistings" than sales. I assumed that the market would be less efficient over time, and the ratio of sales:delistings would gradually lessen over time. I thought I would graph that ratio, over time, to see what it looked like, and I was really surprised when I got a graph that looked like this:
As I expected, ratios in general lowered over time. However, there were huge unexpected anomalies in the data. I went back to the Excel chart. In December of 2002, where the first anomaly occurred, excessively low "new listings" in December appear to be "balanced" by excessively high "new listings" in January. This caused "delisted" to be skewed in the opposite direction- something most easily caught with a ratio. As one moved on to 2004-2005, it’s apparent that these anomalies, which appear to be attempts to "balance" the reports, show up more often.
I became curious and decided to compare ARMLS data against the numbers from Jay Butler of the Arizona Real Estate Center. Butler’s monthly report is used by the Arizona Republic and others as a kind of a "State of Phoenix Housing" report. As Butler does not track inventory in his reports, I could not look at sales:delistings. I did however, graph his sales numbers vs. ARMLS sales numbers:
While in general, Butler’s numbers track lower than those of ARMLS, in the 2004-2005 period Butler’s and the ARMLS’ numbers differ markedly. I emailed Butler and asked if he had an explanation for this difference, but received no reply. It appears, though, that one explanation may be the differences resulting from difficulties with MLS data during that period.
I’m not the only one that has questions about the data during this time. Ultimate New Homes is a newsletter that is widely read by local realtors. In their November newsletter, they posted a graph of Phoenix listings vs. sales, but they omitted Jan.-Mar. 2005 with the following explanation:
Our sources of data for these displays are the ARMLS reports. In these, there are three months, January – March 2005, for which we have not shown listing data because of apparent discrepancies. Additionally, ARMLS notes that the listing information for March through August 2002 may contain errors, but we have chosen to display this reasonable data above.
I asked long time realtor L for his explanation of the discrepancies in this time period. He said:
I don’t know where Butler gets his data, but the NAR, ARMLS and the local Associations get their data from the MLS sold report. Tons of agents had exclusive listings that never went in the MLS. When the MLS reported 5400 listings for Phoenix, there were probably 24,000 or more for sale. I would bet 75% never made it to the MLS- all you had to do was tell another agent or put a sign up and it sold. Some agents turned in listings and sold reports in the same day, so it never came up as listed, so days on the market was "0." Most never reported the sale to the MLS. Not only that, there were duel escrows sold twice in the same day- some went pending to one person and while pending someone else would buy it, so two agents turned in sold reports on the same house. It went and on and on and on like that. The only way to get numbers for that time period would be to go to the Recorders Office and lay out all the sales and see how many times a house sold, and what sold. That could take
The source of my data for Tucson is the Tucson Association of Realtors (TAR). As with the Phoenix study, I calculated "delistings" and looked at the "sales:delistings ratio." Here’s what that graph looks like for Tucson:
You can note two interesting things about the Tucson graph. One is that there appears to be a large number of anomolies in the 2004-2005 time period, as there was in the Phoenix market. This brings the accuracy of data of this period into question, as it did in Phoenix. The other is the sudden jump in the ratio that occurs every December. It appears likely that this jump indicates that the TAR reviews and adjusts their data in December to "balance" the year. That in itself, however, shows that these reports are not the result of simply pushing a button to mine the data–they are reviewed and adjusted.
What has happened in Las Vegas is what caused me to start considering the implications of MLS inaccuracies and reliance on their reports. I did a post December 9th where I used my "delistings" calculation. Reader "Rebel" indicated that the numbers in the report by the Greater Las Vegas Association of Realtors (GLVAR) had to be wrong, and that they were celebrating a decrease in inventory that didn’t happen. I was quite frankly skeptical of his claims. After we worked together though on the Las Vegas data, I was convinced that he was right, and there was a serious problem with the reports. To see our worksheet, click here.
Here is what the ratio chart for Las Vegas single family sales:delistings looks like, as well as the more dramatic condo ratios (please click below on page 2 to continue):