= Subscriber content; or subscribe now to access all American Banker content.

How Online Home-Price Estimators Distort the Market

One of the more attractive tools on real estate search websites these days for prospective homebuyers and sellers alike are those providing automated house value estimates. In today's world the thirst for real-time information is almost unquenchable. Yet arming consumers with such tools without providing them with disclaimers on the limitations of these models can magnify price volatility in housing markets and hurt house sales.

Lying behind online home value estimators are a host of complicated statistical models that attempt to predict a home's price based on comparisons with similar properties (comparable sales) using county property record data on a variety of attributes such as house square footage, number of bedrooms and bathrooms, among a number of features unique to that property. Aggregating and updating property record data from thousands of offices around the country has improved over the years. However, significant data lags of six months or more still exist in many areas depending on the record office's capabilities to update this information.

Moreover, a danger in using this data is that there is a great deal of variation in the quality of information captured by these recording offices. Square footage can be wildly off, as well as the characteristics of the properties. These models have been around for a number of years and during the boom years were used extensively behind the scenes by banks (including those I worked for) as well as Fannie Mae and Freddie Mac as a way of reducing processing costs. During these years, regulatory agencies identified a number of significant limitations surrounding the use of these models. They tend to work better on homes of similar type and quality but break down quickly on custom homes, homes in rural areas, small neighborhoods and nonstandard property features. Further, no statistical model can understand the market and condition issues of a particular property. For example, these models cannot factor in property upkeep or whether the property sits directly across the street from a gas station, which could reduce its value.

Of some concern for regulators is the tradeoff between the percentage of homes where an automated valuation model could be used (known as AVM coverage) and the accuracy of the estimated value. The greater the use of the model across property types, the higher the valuation errors, typically. Those tradeoffs remain, and as we have learned from the crisis, anytime we become overly reliant on modeled outcomes it leads to a misunderstanding of risk.

In the specific application of online valuation engines, buyer offers can be highly influenced by the estimates generated from these models. In forming an offer, a buyer needs to obtain a reasonable view of what a home is worth. The values displayed by online real estate sites provide an easy way for a buyer to develop an offer.

Consider a scenario where, unknown to a buyer, the comps used by a valuation model could be as old as six months and the county had not recorded the 1,000 square feet added to the home the year before. Even though prices were picking up briskly in the last couple of months on this hypothetical property, the data feeding the online valuation model was not reflective of current market conditions. Further, the valuation model used by the buyer further sets a low accuracy tolerance for its use. In other words, to spit out a number it has to be only 65% sure that the property estimate generated doesn't exceed the actual value of the property by more than 15%. Such a low threshold allows more properties to be evaluated by the model, but at the expense of a higher error rate on the valuation. Granted, that particular quirk might partly offset the downside bias from the data lag. Even so, the net effect is a valuation estimate that is 20% less than true market value due to these data limitations of the model. The buyer has no idea of these issues and so a low bid would be the natural outcome.

Such outcomes create an artificial drag on house values during recovery periods and amplify price appreciation trends during boom periods given potential data lags in market pricing. In addition, the use of such valuations in forming bids can lengthen or prevent real estate transactions from being consummated given large potential gaps between sale and offer prices using these estimates.

AVMs may have a limited place in the real estate and mortgage business, but they remain a black box to homebuyers and most market participants, susceptible to a host of data weaknesses despite their technical sophistication. Proper disclosures and limitations on the use of such models should be made to consumers beyond the meager ones that exist today. This should be one area of focus by the Consumer Financial Protection Bureau.

Clifford Rossi is the Professor-of-the-Practice at the Robert H. Smith School of Business at the University of Maryland.


(11) Comments



Comments (11)
I recently bought a home and frequently looked at home values using Zillow, Trulia and some other AVM's. I found the data on price, square footage, number of rooms and other items to be occasionally inaccurate. These sites were very helpful, however, in providing a broader sense of the range of home values in a specific neighborhood, or even a specific street. I double-checked the inaccurate data with online county tax records and found that the county tax records were also wrong. So it's not like using county tax records, instead of an AVM, would give you better information. Also, for what it's worth, my Realtor strongly condemned my use of Zillow, Trulia and other AVM's, saying that the data was inaccurate. Perhaps the Realtors industry feels competitive pressure from the AVM's? --Andy Peters, regional & community banking reporter, American Banker.
Posted by Andy Peters | Monday, September 30 2013 at 12:15PM ET
It is an interesting hypothesis that Zillow and other free consumer AVMs are distorting the market but without some empirical data to support this I am going to say that this is highly unlikely. First, a quick check of Zillow does include a disclaimer on their 'zestimate' as well as a valuation range. Of course, I understand that most people will not read the disclaimer and will pick the price from the range that is most aligned with their own interests. However, most consumers still use real estate agents to assist in their transaction and most competent RAs have both a good understanding of market values and in dealing with consumers that have unrealistic expectations on home prices.

Finally, having worked extensively with a large number of commercial AVMs, I know that the mean error on AVMs is close to zero - slightly low in an appreciating market and slightly high in a declining market. The error distribution certainly varies significantly from FIPS to FIPS but in high population areas with decent data the PPE10 (% of valuations within +/- 10%) can exceed 80% for good AVMs (I have never seen accuracy stats on Zillow).

I think that if I were to highlight a major shortcoming in AVMs I would point out that there are more than a few commercial AVM vendors that include MLS listing data as an input to the valuation estimate. This can produce exceptionally misleading performance statistics as accuracy is highly inflated when benchmarked against sales transactions (how AVMs are measured for accuracy). However, AVMs are primarily used for refinance transactions where the MLS crutch is not available; and as expected the performance of those AVMs that are dependent on MLS data falls apart for these transactions.
Posted by R H | Monday, September 30 2013 at 5:16PM ET
AVM's are here to stay. True, they are not a perfect solution for every transaction but are a good starting point and in aggregate can be considered reliable (if some properties are slightly undervalued and some slightly overvalued taken in sum they are a reliable predictor of value). As a lender I could have 3 certified appraisals done on a given property that would, almost certainly, return three different values. It's all an opinion of value. What a home is worth remains what an informed buyer and seller agree the value is. The same case was originally made against automated underwriting engines which have proven to be highly predictive. AVM models will continue to be refined and improved enabling more streamlined closings especially in refi markets.
Posted by Chinook Mortgage Ltd | Wednesday, October 02 2013 at 4:37PM ET
I wonder how much E&O insurance AVM's carry (none). I guess you cannot take them to court, only physical appraisers are lible for values.
Posted by mr.ed | Wednesday, October 02 2013 at 6:15PM ET
OK People, Get your head out of the sand! I am with you Clifford...I am a certified residential real estate appraiser (20 years) and I can tell you first hand that AVM's are not only worthless for reliable valuation, but they are a large factor in deceptive valuation, and could be detrimental to the country's economy if left uncheck since so many people who are uneducated in the valuation process take them as gospel! I realize that my words mean nothing to AVM supporters, but I can't tell you how many times that a borrower has told me at the inspection what they feel the value is on their property based on Zillow! Once I educate them and provided a "substantiated" appraisal with a full inspection, verification of "true" comparables, market analysis, scope of work, etc., They are humbled. Does Zillow inspect and measure the subject property? Does Trulia drive by the comparables to see if there is a land fill in the back yard or even if the place is still standing? Do they take into consideration if the sale was an arm's length transaction? Absolutely NOT! AVMs may be here to stay, but in what capacity? The only way an AVM would even be close to being acceptable is if all of their data was obtained from Appraisals. But even that would be a great feat considering the data would age and not be reliable. Chinook Mortgage Ltd, It is a known fact that if you had 3 appraisals done on a given property that you would almost certainly return 3 different values. Not because because it is an opinion of value, but because it is not an exact science! The opinion of value is based on hard data which is not available to an AVM! What a home is worth IS what an INFORMED buyer and seller agree that the value is, and if they both used an AVM to determine that then they could really be in trouble! Why don't you run your home on Zillow and sell it for that much. You might get lucky and have the inferior home for the neighborhood, or you might not be very happy if you have a higher end home.....
Posted by | Wednesday, October 02 2013 at 7:14PM ET
Add Your Comments:
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.