= Subscriber content; or subscribe now to access all American Banker content.

How Online Home-Price Estimators Distort the Market

One of the more attractive tools on real estate search websites these days for prospective homebuyers and sellers alike are those providing automated house value estimates. In today's world the thirst for real-time information is almost unquenchable. Yet arming consumers with such tools without providing them with disclaimers on the limitations of these models can magnify price volatility in housing markets and hurt house sales.

Lying behind online home value estimators are a host of complicated statistical models that attempt to predict a home's price based on comparisons with similar properties (comparable sales) using county property record data on a variety of attributes such as house square footage, number of bedrooms and bathrooms, among a number of features unique to that property. Aggregating and updating property record data from thousands of offices around the country has improved over the years. However, significant data lags of six months or more still exist in many areas depending on the record office's capabilities to update this information.

Moreover, a danger in using this data is that there is a great deal of variation in the quality of information captured by these recording offices. Square footage can be wildly off, as well as the characteristics of the properties. These models have been around for a number of years and during the boom years were used extensively behind the scenes by banks (including those I worked for) as well as Fannie Mae and Freddie Mac as a way of reducing processing costs. During these years, regulatory agencies identified a number of significant limitations surrounding the use of these models. They tend to work better on homes of similar type and quality but break down quickly on custom homes, homes in rural areas, small neighborhoods and nonstandard property features. Further, no statistical model can understand the market and condition issues of a particular property. For example, these models cannot factor in property upkeep or whether the property sits directly across the street from a gas station, which could reduce its value.

Of some concern for regulators is the tradeoff between the percentage of homes where an automated valuation model could be used (known as AVM coverage) and the accuracy of the estimated value. The greater the use of the model across property types, the higher the valuation errors, typically. Those tradeoffs remain, and as we have learned from the crisis, anytime we become overly reliant on modeled outcomes it leads to a misunderstanding of risk.

In the specific application of online valuation engines, buyer offers can be highly influenced by the estimates generated from these models. In forming an offer, a buyer needs to obtain a reasonable view of what a home is worth. The values displayed by online real estate sites provide an easy way for a buyer to develop an offer.

Consider a scenario where, unknown to a buyer, the comps used by a valuation model could be as old as six months and the county had not recorded the 1,000 square feet added to the home the year before. Even though prices were picking up briskly in the last couple of months on this hypothetical property, the data feeding the online valuation model was not reflective of current market conditions. Further, the valuation model used by the buyer further sets a low accuracy tolerance for its use. In other words, to spit out a number it has to be only 65% sure that the property estimate generated doesn't exceed the actual value of the property by more than 15%. Such a low threshold allows more properties to be evaluated by the model, but at the expense of a higher error rate on the valuation. Granted, that particular quirk might partly offset the downside bias from the data lag. Even so, the net effect is a valuation estimate that is 20% less than true market value due to these data limitations of the model. The buyer has no idea of these issues and so a low bid would be the natural outcome.

Such outcomes create an artificial drag on house values during recovery periods and amplify price appreciation trends during boom periods given potential data lags in market pricing. In addition, the use of such valuations in forming bids can lengthen or prevent real estate transactions from being consummated given large potential gaps between sale and offer prices using these estimates.

AVMs may have a limited place in the real estate and mortgage business, but they remain a black box to homebuyers and most market participants, susceptible to a host of data weaknesses despite their technical sophistication. Proper disclosures and limitations on the use of such models should be made to consumers beyond the meager ones that exist today. This should be one area of focus by the Consumer Financial Protection Bureau.

Clifford Rossi is the Professor-of-the-Practice at the Robert H. Smith School of Business at the University of Maryland.


(12) Comments



Comments (12)
Do you need a loan or capital we offer loans to both individuals and firms at low and affordable interest rate. Contact us today at ( or to get your loan today.
Posted by carren ben | Tuesday, November 03 2015 at 11:04PM ET
@Stan Humphries

Your Zillow Zestimate algorithm is fundamentally flawed and the fact that 17% of 100 million Zestimates are more than 25% incorrect is evidence of this. In 7 years you have learnt nothing. You claim Zestimates are a 'starting point' but, when Zillow refuse ALL reasonable requests from homeowners to correct or delete what are obviously nonsense zestimates then it is apparent that Zestimates are an 'end point' too. Zillow has demonstrated that it cannot be trusted to act in a fair minded manner is causes nothing but misery and potential financial damage to millions of homeowners and the time has come for Zillow to be Regulated. As a mininimum our elected officials should introduce a DoNotZestimate website in the same way that we can opt out of unwanted phone calls with DoNotCall. I speak with the authority of having a substantially inaccurate Zestimate where Zillow had my home located in a city over 6 miles away. They called this a 3rd party geo-mapping error bit still refused to correct or deleted the erroneous valuation on my home. Shame on Zillow and even more shame on Stan Humphries who had zero housing experience before developing the Zestimate algoritithm which is patently obvious to the 17 million homeowners with Zestimates more than 25% incorrect.
Posted by Surbiton | Saturday, November 23 2013 at 9:35PM ET
@ Stan Humpries

If your consumers are only using the information as a starting point, or a general guide, then the information is fine. To use it on specific houses and generate a value for that houses which they then throw at Real Estate Agents or Appraisers, then that information is misleading and not accurate.

@ Petere007

I have been an appraiser for nearly a decade. My fees have dropped from an all time high of 500 (around the boom) to 300. Even though the amount of work has gone up with new forms to be filled out and new laws to follow. My cost has gone up, while my fees have not.

The high fee the consumer is paying is due to middlemen adding on additional charges. The bank adds a charge for ordering the appraisal. The company that hires the appraiser adds a charge for hiring the appraiser, and in many instances charges the appraiser as well.

So the fee the consumer pays is not the fee the appraiser gets.

@ R H

AVMs are not a threat to the appraisal industry. They were used before, caused all sorts of problems and headaches, were dropped and now they are coming back again. Lenders, consumers and so many others do not understand what appraisers do versus what an AVM does.

The only real threat to the appraiser industry would be lenders and companies that hire us taking our reports, using the comparable property information and adding that to their database which they then use to generate values. Not only is it against the confidentiality agreement that each lender/client enters into with the appraiser, but it is also as useless as an AVM.
Posted by TaoOfJester | Wednesday, October 30 2013 at 6:45PM ET
The automated models are fine in some circumstances.

The problem is that the traditional appraisal required by lenders has become so convoluted and complex and filled with nonsense that appraiser have to charge high fees to compensate for the new reports.
Appraisal reports could be a lot cheaper if the lender only required a value determined by an expert.
Simplify the traditional reports and fees should drop and reliance on computer models will decline.
Posted by Peter007 | Friday, October 04 2013 at 10:39AM ET
Stan Humphries, Zillow chief economist here. When we created Zillow and the Zestimate, we thought hard about how consumers would use it. Seven years in, there's a lot we know.

First, we constantly track (and publish here: our accuracy down to the county level by comparing actual sale prices with a home's Zestimate. We know, as R.H. above me suggested, that the median error on Zestimates is very close to zero (even while the median absolute error is about 8.4%). This implies that the Zestimate is an unbiased estimate of value and suggests that it does not cause significant distortions in the market (since, for half the sales that occur, the buyer pays more than the Zestimate, something that presumably would not happen if buyers were blindly using the Zestimate). You can read more about this here: I can't speak for all consumer-facing AVMs out there, but this is true of ours.

Second, I think it's important to look at how consumers use AVMs. We firmly believe that consumers are intelligent and want to educate themselves, and that they deserve access to the same kinds of information professionals have had for years. An AVM is one part of that, but it is only one part. We often say the Zestimate is a starting point in determining a home's value.
I often explain the home buying process as a funnel. Near the top, the consumer looks at a lot of data that we make available on Zillow - AVMs, recently sold properties, what is currently on the market, current mortgage rates and what they can afford - to determine where and when they want to buy. As they get closer to the transaction, they contact a real estate agent, something we encourage as having a professional guide one through the home buying process is incredibly valuable.

I hope this information is helpful. Consumers are clearly thirsty for information about the housing market, and we truly believe that giving them more information helps them make smarter decisions.
Posted by StanHumphries | Thursday, October 03 2013 at 1:37PM ET
AVM's are similar to pricing art by the square inch or sculpture by the pound. There is no consideration of taste, condition, use of space, etc. No adjustment made for one home impeccably maintained vs. another that burned down last week.

How much is a 3-year old Quarterhorse worth?

An 8 oz. Sirloin in Chicago?

AVM's are OK so long as you understand it is a ballpark for average properties without adjustment. If you can't tell whether you are looking at a $250K or $750K neighborhood an AVM can give you a clue.
Posted by IRonskonski | Thursday, October 03 2013 at 11:43AM ET
Chinook - I agree completely that AVMs are here to stay and that they can be a very valuable tool. My concern is that the performance statistics on many AVMs are misleading. They are benchmarked on sales transactions (where they can cheat) but primarily used for refi transactions where they cannot.

Mr. Ed - google AVM lawsuits. You sure can take the AVM vendors to court as the FDIC and others have shown.

Jerry - I think your response is in highly reflective of the fact that AVMs are a threat to your industry and have reduced the demand for your services. Nobody who is knowledgeable about AVMs would ever state that on an individual property and AVM is going to be more accurate than a physical appraisal. And yes, AVMs are know to occasionally produce spurious results (which are often trotted out by appraisers as proof that AVMs have no value). However, on an aggregate level AVMs are actually more accurate than appraisers (as measured by the mean error) and much more cost effective. If I have a portfolio of mortgages, the most cost effective and accurate way for me to get the current LTV of that portfolio certainly is not by using appraisers. Also, the quality and accuracy of appraisals is also highly dependent on the skill of the appraiser. An appraiser like you with 20 years of experience is going to produce more accurate valuations - especially on more challenging properties - than an appraiser that is new to the business.
Posted by R H | Thursday, October 03 2013 at 9:37AM ET
OK People, Get your head out of the sand! I am with you Clifford...I am a certified residential real estate appraiser (20 years) and I can tell you first hand that AVM's are not only worthless for reliable valuation, but they are a large factor in deceptive valuation, and could be detrimental to the country's economy if left uncheck since so many people who are uneducated in the valuation process take them as gospel! I realize that my words mean nothing to AVM supporters, but I can't tell you how many times that a borrower has told me at the inspection what they feel the value is on their property based on Zillow! Once I educate them and provided a "substantiated" appraisal with a full inspection, verification of "true" comparables, market analysis, scope of work, etc., They are humbled. Does Zillow inspect and measure the subject property? Does Trulia drive by the comparables to see if there is a land fill in the back yard or even if the place is still standing? Do they take into consideration if the sale was an arm's length transaction? Absolutely NOT! AVMs may be here to stay, but in what capacity? The only way an AVM would even be close to being acceptable is if all of their data was obtained from Appraisals. But even that would be a great feat considering the data would age and not be reliable. Chinook Mortgage Ltd, It is a known fact that if you had 3 appraisals done on a given property that you would almost certainly return 3 different values. Not because because it is an opinion of value, but because it is not an exact science! The opinion of value is based on hard data which is not available to an AVM! What a home is worth IS what an INFORMED buyer and seller agree that the value is, and if they both used an AVM to determine that then they could really be in trouble! Why don't you run your home on Zillow and sell it for that much. You might get lucky and have the inferior home for the neighborhood, or you might not be very happy if you have a higher end home.....
Posted by | Wednesday, October 02 2013 at 7:14PM ET
I wonder how much E&O insurance AVM's carry (none). I guess you cannot take them to court, only physical appraisers are lible for values.
Posted by mr.ed | Wednesday, October 02 2013 at 6:15PM ET
AVM's are here to stay. True, they are not a perfect solution for every transaction but are a good starting point and in aggregate can be considered reliable (if some properties are slightly undervalued and some slightly overvalued taken in sum they are a reliable predictor of value). As a lender I could have 3 certified appraisals done on a given property that would, almost certainly, return three different values. It's all an opinion of value. What a home is worth remains what an informed buyer and seller agree the value is. The same case was originally made against automated underwriting engines which have proven to be highly predictive. AVM models will continue to be refined and improved enabling more streamlined closings especially in refi markets.
Posted by Chinook Mortgage Ltd | Wednesday, October 02 2013 at 4:37PM ET
It is an interesting hypothesis that Zillow and other free consumer AVMs are distorting the market but without some empirical data to support this I am going to say that this is highly unlikely. First, a quick check of Zillow does include a disclaimer on their 'zestimate' as well as a valuation range. Of course, I understand that most people will not read the disclaimer and will pick the price from the range that is most aligned with their own interests. However, most consumers still use real estate agents to assist in their transaction and most competent RAs have both a good understanding of market values and in dealing with consumers that have unrealistic expectations on home prices.

Finally, having worked extensively with a large number of commercial AVMs, I know that the mean error on AVMs is close to zero - slightly low in an appreciating market and slightly high in a declining market. The error distribution certainly varies significantly from FIPS to FIPS but in high population areas with decent data the PPE10 (% of valuations within +/- 10%) can exceed 80% for good AVMs (I have never seen accuracy stats on Zillow).

I think that if I were to highlight a major shortcoming in AVMs I would point out that there are more than a few commercial AVM vendors that include MLS listing data as an input to the valuation estimate. This can produce exceptionally misleading performance statistics as accuracy is highly inflated when benchmarked against sales transactions (how AVMs are measured for accuracy). However, AVMs are primarily used for refinance transactions where the MLS crutch is not available; and as expected the performance of those AVMs that are dependent on MLS data falls apart for these transactions.
Posted by R H | Monday, September 30 2013 at 5:16PM ET
I recently bought a home and frequently looked at home values using Zillow, Trulia and some other AVM's. I found the data on price, square footage, number of rooms and other items to be occasionally inaccurate. These sites were very helpful, however, in providing a broader sense of the range of home values in a specific neighborhood, or even a specific street. I double-checked the inaccurate data with online county tax records and found that the county tax records were also wrong. So it's not like using county tax records, instead of an AVM, would give you better information. Also, for what it's worth, my Realtor strongly condemned my use of Zillow, Trulia and other AVM's, saying that the data was inaccurate. Perhaps the Realtors industry feels competitive pressure from the AVM's? --Andy Peters, regional & community banking reporter, American Banker.
Posted by Andy Peters | Monday, September 30 2013 at 12:15PM ET
Add Your Comments:
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.