Bots Are Getting Passports, and the Passport Is a Rate Card

David Martin Riveros · August 4, 2026 · 6 min read

HTTP has carried a slot for a price tag since the nineties. Status code 402, Payment Required, was written into the protocol early and then reserved for a future nobody built. That future is now in production: on an average day, Cloudflare's customers send more than a billion 402 responses, most of them to automated visitors, each one an invoice looking for a payer. The plumbing for charging bots was never the hard part. The hard part is that you cannot bill traffic you cannot name. Which is why the most consequential piece of web infrastructure shipping this year is not a payment rail. It is a passport.

A key pair and a header

The passport is called Web Bot Auth, proposed by Cloudflare in May 2025 and now co-authored with Google engineers in front of the IETF. The mechanism is plain. A bot operator generates a cryptographic key pair and publishes the public half at a fixed address on a domain it controls. Every request its crawlers or agents make is signed with the private half, and a Signature-Agent header on the request says where the public key lives. Any website, or any firewall in front of one, fetches the key once and checks the math on each request. No IP ranges to maintain, no user-agent strings taken on faith. The signature verifies or it does not.

The checking side adopted fast. Cloudflare verifies signatures across its network, Vercel added support in August 2025, Akamai in November, DataDome in January. On the signing side, OpenAI signed its agent traffic from the protocol's public debut. Google published its own key directory this spring and calls its implementation experimental, covering a subset of requests from its Google-Agent, though not Googlebot itself. And in October 2025 the payment networks arrived: Visa built its Trusted Agent Protocol with Cloudflare around Web Bot Auth, Mastercard folded it into Agent Pay, and American Express committed its agentic commerce program to it. When Visa and Mastercard have implemented your identity scheme, it has stopped being a security experiment.

The standard keeps not passing

Here is the strange part. The paperwork keeps losing to the deployment. The IETF chartered a working group for this in October 2025 and gave it three milestones: two standards-track documents sent to the leadership by the end of April 2026, and an operational best-practices document by August 31. April passed with nothing sent. At the group's meeting on July 22, the room polled 22 to 6 that HTTP message signatures are the right general direction, and still declined to adopt the core draft, sending it back for more work on key discovery and rotation. The August milestone comes due this month with the group yet to adopt a single document. Every specification in play remains, in the datatracker's own language, an individual submission with no formal standing.

The dissent is not procedural. Three days before that meeting, Eric Rescorla, who spent years editing TLS, posted a rival draft with Richard Barnes for anonymous bot authentication: prove your crawler belongs to an approved class without revealing which crawler it is. His warning to the group went into the minutes. A voluntary signature system, he argued, "may become a de facto requirement to access web content, potentially leading to a discriminatory web." Cloudflare's Marwan Fayed answered with the operator's shrug: without a standard, sites will keep blocking unrecognized traffic anyway. Both of them are describing the same fact from opposite sides of the wall.

The meter needs a name

Read the deployment lane again and notice what the passport is bolted to. Pay-per-crawl, the Cloudflare scheme that answers a crawler with a 402 and a price instead of a block, works, in the company's own words, "in tandem with Web Bot Auth." The Monetization Gateway it opened to a waitlist on July 1 lets a site "require agents to authenticate with Web Bot Auth and apply usage-based pricing," with example rates in the announcement of a cent per request and a metered charge per megabyte. The identity layer and the billing layer are shipping as one product, because a meter is useless without an account to attach the reading to. Anonymous traffic can only be blocked or allowed. Named traffic can be priced.

Billing by the page is not exotic. It is how my own customers already buy: they pay per page collected, and disputes get settled from that ledger. What is new is which side of the wall the meter sits on, and what the wall can see. By Cloudflare's measurement, 52 percent of crawler requests were fetching content for AI training by June 2026, up from 22 percent in spring 2025. More than half of all traffic on its network is now non-human, and the most heavily crawled categories of sites have lost as much as 40 percent of their human visitors in under a year. A website in 2026 is a shop where most of what comes through the door is machines. The two responses that scale are locking the door and installing a till, and both begin with knowing who is who.

What signing buys, and who pays

My position, and it is one that can be checked against events: signing wins, and it wins before the standard does. Not because the IETF blesses it, but because both sides of the wall profit from it. Site owners convert bot traffic from a cost into a revenue line. The large agent operators get the thing they need even more than access, which is reliability, an end to being mistaken for the traffic nobody wants. Rescorla's de facto requirement arrives not by mandate but by default settings, one onboarding screen at a time, starting September 15. The cost lands on whoever runs automated traffic without the scale to negotiate, because a posted price is what you pay when nobody fears losing you. If the anonymous-credential approach wins adoption instead, or if commercial-scale crawling still moves mostly unsigned through the end of 2027, I will have read this wrong.

The thing to watch is not the working group's November meeting, where the drafts may finally be adopted after the market has finished deploying them. Watch for the first published per-operator price. The moment a network acting for millions of sites posts different rates for different named crawlers, the passport will have produced the thing it was built to produce: a market where the cost of reading a page depends on who is asking. Every operator of automated traffic is deciding right now, mostly by default, which side of that ledger it will stand on.