What Makes Web Data Investable
The first time I sat through an institutional buyer's diligence on one of our datasets, the questions surprised me. Nobody asked how much data we had. They asked when each row had been observed. Whether the history had ever been edited. What happens, operationally, on the day a source breaks. It was less like a sales call and more like an audit, and it taught me more about the alternative-data market than anything I have read about it.
Web data is the largest category of what investors call alternative data: prices, inventory, listings, hiring, reviews, foot traffic's digital cousins. Most funds now touch it somewhere in their process. But there is a wide gap between web data that is interesting and web data an institution will actually pay for and put into production, and the gap is not about volume.
What the buyers are really buying
Timeliness is the obvious part: the web shows you the economy before official numbers do. Breadth matters too, though it is misunderstood. One retailer's prices are an anecdote; the same measurement repeated identically across hundreds of sources becomes an instrument, and the word that matters in that sentence is "identically."
The less obvious asset is history. You cannot go back and start collecting three years ago. Whoever started measuring early owns the back-history, and backtests eat back-history. I have seen buyers pass on technically superior datasets because a worse competitor had four more years of tape. It feels unfair. It is also rational, and it means the best time to start collecting a signal is well before the demand for it is proven.
And then there is the one that decides deals: provenance. Can the vendor say, for any row, when it was observed, from where, under what conditions, and prove it has not been touched since. Data is easy to have. Defensible provenance is the scarce thing, and it is getting scarcer as generated content floods the open web.
View data table
| Year | Market size | CAGR 2023–2030 |
|---|---|---|
| 2023 | $7.2B | 52.1% |
| 2030 (projected) | $135.7B | — |
Where datasets fail diligence
The failure patterns are surprisingly consistent.
Retroactive edits are the classic one. Collection history has bugs, and the temptation to quietly clean the past is constant. But a dataset whose January can change in March is useless for backtesting and dangerous in production, so the discipline has to be point-in-time: corrections happen forward, or as explicitly versioned restatements, never silently.
Undocumented methodology changes are subtler. You improve a collector, coverage jumps, and to any model consuming the feed your improvement is indistinguishable from a market event. A good chunk of the false alpha I have seen in web-derived signals traced back to a collection change nobody wrote down, not to anything that happened in the world. The fix is boring: a change log, published, tied to the series.
Then there is coverage honesty. Sources block, redesign, and break; gaps happen to everyone. The difference between vendors is whether gaps are declared or papered over. Institutions can work around missing data. Interpolated data presented as observed is a different thing, and discovering it ends relationships.
Compliance questions used to come last in diligence and now come first: where does this data's right to exist come from, is it public, does collecting it involve deceiving anyone. Precise answers here are a competitive asset. Vague ones are a red flag that experienced buyers treat as disqualifying.
View data table
| Year | Share |
|---|---|
| 2023 | 62% |
| 2024 | 67% |
| 2025 | 90% |
What the AI wave actually changes
On the supply side, the economics moved. Turning messy web signals into clean panels used to require expensive humans at every step, which kept most niche datasets uneconomical to build. Model-assisted extraction, entity resolution, and continuous anomaly detection bent that cost curve. Sources that were too small, too varied, or too unstable to justify a team are becoming feasible for a system with human review at the edges. The frontier of what counts as collectable is moving outward fast.
On the demand side, funds are pointing their own models at purchased data, and models are unforgiving consumers. A human analyst reads the vendor's caveats; a model happily learns a methodology artifact as if it were alpha. So the premium is shifting further toward exactly the boring virtues above: point-in-time discipline, change logs, declared gaps. The datasets that win are not the biggest. They are the ones a machine can consume safely without a human explaining the footnotes.
View data table
| Year | Share |
|---|---|
| 2023 | 37% |
| 2024 | 57% |
| 2025 | 56% |
The questions that sort the market
If you are evaluating web-derived data, in either direction of the table, a handful of questions does most of the work:
- Show me the methodology change log for the last two years.
- Is the history point-in-time? Prove no retroactive edits.
- What were your three worst coverage gaps last year, and how were clients told?
- For this row, right here: when was it observed, and what checks did it pass?
- What is the data's legal basis for existing?
Vendors who answer in writing, with specifics, are rare, and funds notice. I have watched procurement teams forgive weaker coverage because the provenance answers were airtight, and I have watched bigger datasets die in diligence because they could not produce a change log.
That, more than any AI angle, is the state of this market: production of web data is getting cheaper every quarter, and trust in it is not. The vendors and the funds that internalize which of those two is the bottleneck will be the ones that get the next decade right.
New essays by email, on AI for data collection and analysis.