Why Travel Data Is the Hardest Data on the Web

David Martin Riveros · July 9, 2026 · 4 min read

Ask a travel site what a rental car costs and it will not tell you. It asks you things first. Which dates. Which pickup location. How many days. Where are you booking from. Then it generates a number that is true for you, right now, and might be stale twenty minutes later.

I have collected most kinds of web data by now: retail catalogs, real-estate listings, marketplaces, registries. Travel is the one that still forces us to think the hardest, and it comes down to that one property. Travel prices are not published. They are quoted. There is no page where the price lives waiting to be read; there is a question you ask and an answer with a shelf life.

Almost everything that makes this category difficult follows from that.

A travel dataset is a questionnaire, not a crawl

For a catalog site you can, in principle, enumerate the pages and go read them. For travel there is nothing to enumerate. A dataset has to start from a designed set of queries: these markets, these pickup dates, these trip lengths, these booking windows. That design is the dataset. Two vendors can claim to cover the same market and hold completely different data because they asked different questions.

The uncomfortable part is that badly designed queries do not produce errors. They produce plausible numbers. If your queries always come from the same country, always on the same weekday, always for one trip length, you will get a clean, internally consistent dataset that quietly fails to represent the market you sold it as. Nothing in the pipeline will ever complain. I have reviewed purchased travel data where the entire booking-window dimension was one value, and the buyer had not noticed for a year.

The combinatorics make this worse. Locations times dates times durations times booking windows explodes into millions of possible queries, and nobody can afford all of them, so every travel dataset is a sample. The honest question is never "how many prices do you have" but "which slice of the space did you ask about, and what did you deliberately skip."

The site is allowed to say no

Every travel search costs the site real money, because it fans out to suppliers and partners behind the scenes. That is why travel runs the most aggressive anti-automation defenses on the consumer web. I am not going to describe collection techniques here. The part that matters commercially is what blocking does to the data: the gaps are not random. Defenses tighten exactly where demand is heaviest, so the missing pieces cluster in the busiest markets and the biggest suppliers, which are precisely the places you most wanted to measure.

And gaps hide well. We once found a site quietly trimming a supplier list to its first thirty entries. Nothing failed, no error appeared anywhere; every average computed downstream was just built on a market missing its tail. You do not catch that kind of thing by monitoring your pipeline. You catch it by measuring completeness itself, query by query, against what the site shows a real user. That check has to be a first-class part of the product. If a travel-data vendor cannot tell you how they distinguish "prices fell" from "we stopped seeing the expensive supplier," walk away.

Prices also decay while you are collecting them. A quote captured at nine in the morning and one captured at noon are different observations, and for some analyses the difference matters more than the price itself. Serious travel data carries its timestamps everywhere, expressed relative to the travel date, because "how far in advance was this observed" is half of what a travel price means.

What actually changed with AI

For years the constraint on travel data was manpower. Sites redesign, defenses evolve, result pages change shape, and someone had to notice, diagnose, and repair each one. Breadth was a headcount problem.

Agentic systems changed the shape of that cost. A collection agent can plan a query portfolio the way an analyst would, notice that a supplier stopped appearing in a market and re-verify through a second path before anyone downstream sees the gap, flag a price that is implausible against history instead of shipping it. The repair loop that used to take an engineer most of a week increasingly closes in hours, with the human reviewing the fix instead of producing it.

I want to be precise about the claim. AI did not make travel sites easier to collect from; they are harder targets every year. What it changed is that carefulness now scales. Checking completeness on every query, attaching provenance to every row, re-verifying anomalies through independent paths: these were always the right practices, and they used to be economically impossible at breadth. Now they are just engineering.

None of that makes travel data easy. It makes it possible to do honestly at scale, which is new, and which is the entire difference between a dataset a fund or an operator can act on and one that merely looks complete.