Introducing Search Based Comp Sets

Back when I was pricing flights in the airline industry, my VP used to tell me “You can only be as good as your dumbest competitor.”  The strong implication was, that if your competitor has bad pricing or drops their prices you can be forced to do the same even if you don’t want to.  This is particularly true in a market like the airlines where there are only a few competitors and their products are very similar.  In less competitive industries, you may be able to get away with holding up prices when your competitor is making a mistake. 

An Economy class seat on most major airlines in the U.S. are incredibly similar, they all provide slightly different levels of discomfort.  The majority of leisure travelers do not see a difference between American and United when they are booking travel.  If United drops its prices, America is almost certainly going to respond and match.  Staying uncompetitive only works when demand exceeds supply, so it is driven by capacity.

Short-term rentals are different.  There are a lot more competitors.  When we think about a choice set for an airline trip, it is normally 1 to 3 possible options.  For a rental property, there could be thousands of options in a single city.  The quality of these options can also differ greatly, they are not nearly as similar as two economy-class tickets.  This distinction has made it difficult to determine who your actual competitors are in the market and which ones you want to stay competitive with. 

Quibble has just released a new way to determine a comp set and it is based on search results, only the competitors that end up in the same search as the property our software is setting price for are considered in the pricing.  This is a very refined set of properties that consists of only 10-20 listings and is much more representative of how consumers shop.  

Defining which listings, neighborhoods, regions, etc. make it into your comp set isn’t always static, it depends on the question you are trying to answer. Some questions are pretty broad, like: What impact will rising interest rates have on the short-term rental market? The data set to answer that question could be pretty broad as well, possibly nationwide since the interest rate will impact the entire country. Another question a comp set could answer: Where is the best place in the country to invest in a vacation home? The data needed to answer this question is more specific. Now you are looking at comps or comparable properties by state, city, or neighborhood. The question we, in pricing, are most concerned about answering is this: What comps should I consider when setting the nightly rates for my listing?

The industry standard answer to that question has been surprisingly broad, and it is normally in the 100’s or 1,000’s of properties. These properties are generally nearby and may share some of the same metrics as the listing being priced. A broad comp set like this has some positives and negatives from a pricing perspective. It provides reliable results because the dataset is so big. It is also very easy for the user to manage because it is tracking broad changes, so if one of the competitors gets booked it has almost no impact on the outcome. The biggest negative is the lack of specificity in the pricing. These models are in essence tracking the broad changes in the market.

The quibble model works differently, it is trying to predict the choices shoppers make when they are presented with a set of options online. Shoppers cannot consider hundreds or thousands of listings at the same time when making a decision. So, we started building our comp sets based on the human experience of shopping online for a rental. Instead of taking cues from a broad dataset, we are determining which properties they might consider booking. The rest of the comps do not matter to the quibble model, because they do not matter to the shopper.

There is a very good reason the early attempts at pricing models in this space used such broad comp sets. The forecast/optimization models that have traditionally (airlines and hotels) been used in pricing and revenue management don’t work very well in this space. The dominant model in airline pricing and revenue management can run completely on historical booking data, it doesn’t need any competitive pricing data at all. That type of model is way too risky to run for a single STR property because the historical booking data is too sparse to be reliable.

The models currently dominating this space are Base Price models. These models are the complete opposite of the traditional model and are 100% dependent on competitive pricing data. Since the historical data models didn’t work, the industry invented a model that would track and follow market pricing trends of future dates. These models take big groups of competitors and track their future rates. If they were to use a small comp set, like 10 properties, the model would start to struggle to keep prices from going off the rails.

When we designed the quibble model we made a few attempts to get the traditional airline pricing model to work by using some clever workarounds to solve the historical data problem. This did not work very well, so we took a step back and thought about the problem we were trying to solve. At the most basic and granular level, we were trying to predict what choice a shopper will make when they are looking for a vacation home and use that data to set our pricing.

Someone ready to book a rental is not considering between 1,000 or even 100 properties. By that point, they have weeded out the properties that do not fit their criteria and there are only 10 or so properties left on their list. When we create the comp set for the quibble model, those are the only comps we care about. These are the properties that the shopper may book instead of our property, all of the other properties have been eliminated from the compset. When the model solves for the price, it optimizes the expected revenue when compared against this set.

To find this refined comp set, we have to simulate a series of searches. We are looking for the properties that have the highest likelihood of being on that shopper’s list of considerations. So, we shop the OTA’s to find the properties that show up and then apply a series of filters to match property quality and price. The benefit of doing this through OTA search is that we can leverage the display algorithm that is going on behind the scenes which influences the choice set.

The search is designed to mimic human behavior. It starts with the broad city that the shopper wants to be in. Then, zooms into the specific area inside the city where they plan to stay. Next, we filter on quality and price. By this time, we are taking the first page results with high confidence that these are the real comps.

When I first came into the STR space and was meeting with property managers, I was surprised at how obsessed everyone was with tracking their competitors. In the airline industry, the world I came from, we have pricing models that can run and optimize revenue without any competitive information at all. Competitive pricing data was a secondary data source, not a primary source.

It all became clear to me when I saw the pricing models that STR managers were using. The models weren’t helping them compete effectively because they considered everyone their competitor with no regard to quality or pricing. Quibble’s release of Search Based Comp sets solves that problem for the managers.

Want to learn more about Search Based Comp Sets?