How Much is Fare Scraping Costing the Travel Industry?

Scraper bots make up the worst of bad bot traffic for the travel industry, with sites witnessing over 90% of traffic attributed to fare scraping. Whilst this activity can be benign or even used for positive means, if uncontrolled it can impact top line revenue, bottom line profits and customer experience.

Bots and Airline Ticket Sales

The airline industry is one of the most heavily targeted by bot operators. It’s common for automated traffic to account for over half of all activity on airline websites. In some cases, this can peak at over 90%. The bots that typically target airlines are also typically more advanced that those targeting other industries.

This is in part because pricing is so competitive in the airline industry, not just from airlines but also from third-party vendors and price aggregators. Airline tickets are also a valuable commodity, much like luxury goods or concert tickets. This drives up scraper bot activity, where bots continually send requests to airline websites to collect up-to-the-minute pricing and availability information.

Sometimes this is done to undercut a competitor, however scraping is not always malicious in intent and could be part of a commercial partnership with resellers. Whatever the intention of the scraper bot operator, this activity adds expensive overheads to infrastructure.

Another business logic attack commonly launched against airlines is denial of inventory using spinner bots. Spinning is the act of adding items to an online basket, thus removing it from being available to other customers. When this is done at high volume using bot automation, all the available stock appears to be depleted, disrupting sales and moving customers away to competitors. The spinner bot will then empty its basket without completing the purchase.

For example, on an airline’s or third-party online travel agent’s website, bad actors use bots to reserve seats on flights. The bot reserves the seats for up to 20 minutes, during which time genuine customers perceive there to be no availability left on the flight, and the perpetrator attempts to sell the seats on for a profit. And repeat.

This is done by bad actors for several reasons, including:

• Generating high and fast profit off the back of a fairly low risk opportunity
• Defeating the competition by sending customers to a rival website
• Disrupting availability by making an application unusable as part of an application-layer denial of service attack

What is the look-to-book ratio?

The look-to-book ratio is the number of requests made per booking on an online travel site.

Requests can be made by humans or bots, and the lower the look-to-book ratio, the better. A low look-to-book ratio means conversions are high from genuine customers browsing the website. However, scraper activity can cause look-to-book ratios to exceed several thousand. A high look-to-book ratio inflates the number of requests versus the number of conversions.

When scraper bots pull information from a website, they create excess web requests which, in turn, negatively impacts your look-to-book ratio. Increased competition driven from the pandemic, and the popularity of dynamic pricing, means this is fast becoming a top threat for the travel industry.

How does fare scraping work on travel booking websites?

In travel, web scraper bots are mainly used to collect fare and availability information by rival companies and aggregator sites, used for price comparison

But also targeting travel booking sites are scraper bots, used to discover and publicize the availability of products and services such as flights, hotels or car rentals.

Attackers advertise the scraped information at lower price points on a secondary site, motivated by the financial reward of charging commission, stealing personal data, or generating advertising revenue.

Scraping is also often used to gather the data needed for more sophisticated or damaging attacks such as ticket spinning or denial of inventory. Preventing malicious scraper bots can cut out these further attacks early as the attackers do not have the data they need to progress.

What damage is inflated look-to-book ratios causing to travel companies?

Excess traffic caused by aggressive scraping and high look-to-book ratios negatively impact airlines and travel companies both on their bottom line and on their technical performance.

Business costs

• Additional costs (up to millions per year) to third-party services like Metasearch engines and GDS booking fees, which charge based on traffic volumes
• Extra costs for SIEM and anti-fraud solutions, again which charge based on traffic volumes
• Loss of pricing visibility leading to competitive pricing disadvantage
• Misleading analytics from inaccurate number of website viewers interested in a certain product
• Loss of ancillary revenues (or “add-ons”) e.g., hotels, travel insurance and car hire

Technical costs

• Excessive infrastructure costs (up to 50%) used to serve bots
• IT teams stretched to deal with bots away from daily tasks
• Slowed website performance leading to negative user experience
• Costly downtime in extreme cases

How to prevent scraping, excess transactions and high look-to-book ratios

The travel industry is one of the most severely affected by bad bots and has been since the advent of online travel. As bots grow in sophistication and volume, it is crucial for travel websites to accurately detect and bad bots without affecting good bots necessary for the steady running of your website, and genuine users’ experience.

Netacea Bot Protection helps brands detect and prevent malicious web scraping. Using defensive AI our bot protection technology ensures brands have a constantly active bot detection and mitigation solution to prevent the most sophisticated forms of scraping attacks.