Essays on AI and data
I am also writing The Data Collection Handbook, a free book on data collection best practices with a replicable experiment in every chapter. The first chapters are out, and the book is a free download as PDF or EPUB that grows as chapters publish.
The Web's Attack Surface Just Switched Sides
The same week a court ruled the bot is the user, Black Hat research showed every AI browser tested obeying instructions hidden in the pages it reads. A year of prompt-injection disclosures, why layered defenses failed, and why the fix is architecture, not smarter models.
Your Analytics Copilot Survived Opus 4.8. The Claude 5 Upgrade Is Where It Breaks
Anthropic shipped three Claude 5 models in seven weeks, and the dangerous changes never throw an error. Why a model upgrade under your analytics is a data migration: inventory the pinned IDs, baseline golden questions, remap tiers, and delete the calibration that now works against you.
The Ninth Circuit Just Said the Bot Is the User
The Ninth Circuit overturned Amazon's injunction against Perplexity's Comet shopping agent: it was the users, not Perplexity, doing the accessing. Why that inverts the 2016 Power Ventures rule, and why the agent fight now moves from courtrooms to identity rails.
Bots Are Getting Passports, and the Passport Is a Rate Card
Cryptographic bot identity went from a Cloudflare proposal to Visa, Mastercard, Google and OpenAI in about a year, while the IETF group chartered to standardize it has yet to adopt a draft. The passport's real function is pricing: named traffic can be billed, anonymous traffic can only be blocked.
Only 4 in 10 AI Licensing Deals Still Include Training Rights
Rob Kelly's ledger of public AI content licensing deals crossed 94 in July, and only about 4 in 10 still include training rights. Live-access deals went 2, 11, 18, then 34 projected this year. Why the money moved from archives to live feeds, from an operator who sells the feed.
The First Public Map of What AI Actually Crawled
The EU AI Act makes every general-purpose AI provider publish its top crawled domains. A year in, small labs filed straight-A summaries, the frontier labs cluster at C, and four flagship models filed nothing. Enforcement starts August 2.
Does llms.txt Work? 137,000 Domains Say Not Yet
Ahrefs read a month of server logs across 137,210 domains: 97% of llms.txt files got zero requests, and the AI search bots the file was written for sent just 1.1% of the traffic that did arrive. What that means before you ship one, from a site owner who just did.
Is Robots.txt Legally Binding? Two Courts Answered in the Same Week
A US court called robots.txt a keep-off-the-grass sign. A German court made machine-readable opt-outs the only ones that count. What the split means for AI training data.
Prime Day, Measured
Of 1.19 million European prices tracked through Prime Day 2026, 74.6% never moved and only 13.8% got a real cut. What the shelf data shows, and why price history has to be built before the question is asked.
What a Web Page Costs
Cloudflare's pay-per-crawl posted the first official price for machine access to the web. An operator's math on what a page really costs, and what the premium buys.
How AI Agents Collect and Reconcile E-commerce Data at Web Scale
Variants, pricing conditions, silent breakage: what actually determines e-commerce data quality at web scale, and how AI agents change its economics.
Why Travel Data Is the Hardest Data on the Web
Travel prices are quoted, not published. Why that makes travel the hardest web data to collect, and what AI-driven collection changes about doing it honestly.
What Makes Web Data Investable
What institutional buyers actually pay for in web data: provenance, point-in-time history, coverage honesty, and the diligence questions that sort the market.
New essays by email, on AI for data collection and analysis.