The IETF Adopted the Bot Passport. The Market Issued It Fifteen Months Ago

David Martin Riveros · September 17, 2026 · 4 min read

Web Bot Auth, the protocol that lets a crawler or an AI agent sign its requests, became official standards work on September 1. That day the IETF working group chartered to standardize bot identity adopted it as its own document. The IETF is the body that writes the internet's technical rules. Adoption means the draft now belongs to the group, not its authors. Cloudflare had checked these signatures for more than a year. Google, OpenAI and Shopify already sent them. Visa and Mastercard had built payment protocols on top. In August I argued that signing would win before the standard did. It did. What the group fixed is the format of the name a bot presents, as that name started to carry a price. Two weeks later, on September 15, Cloudflare's default block on Agent and Training crawlers took effect.

What a working group adopts

A bot operator creates a key pair and publishes the public half at a fixed address on its domain, /.well-known/http-message-signatures-directory. Each request the bot sends carries three headers: the signature itself, a list of what was signed, and a Signature-Agent header saying where the key lives. The math follows RFC 9421, the existing standard for signing HTTP messages. A site fetches the key once and checks every request. There is no central registry. The name is a URL.

Adoption is a smaller step than the headline suggests. The document is now draft-ietf-webbotauth-httpsig-protocol-00, a working-group draft on the standards track, written by Thibault Meunier of Cloudflare and Sandor Major of Google. It is a draft, not yet a finished standard, an RFC. The group owed two standards-track documents to the internet's steering group by April 30. It owed an operations document by August 31. All three milestones still read "not done". In July the group declined to adopt anything, on a poll of 22 to 6 with 5 undecided. Eric Rescorla, an author of the TLS standard, warned that signed identity could lead to a discriminatory web. Adoption came one day after the last milestone lapsed.

The market did not wait

Cloudflare proposed the protocol on May 15, 2025. OpenAI said in the same post that it signed every request from its Operator agent. Cloudflare began accepting signed agents that August. In October, Visa and Mastercard built agent-payment protocols on the same signatures. Akamai followed in November, DataDome in January. By May, Google was signing a subset of its Google-Agent traffic, still experimental. On May 7, Shopify put the first price on anonymity. Bots that do not sign "are subject to the strictest limits", and signing earns higher ones.

The second document is the rate card

A companion draft, not yet adopted, defines a "Signature Agent Card", a small file an agent publishes to describe itself. Its authors, from Cloudflare and Amazon, list the fields. Identity, purpose, rate expectations and keys, and whether the agent fetches for a person or crawls on its own. Read those fields as a form and it is an order form: who you are, why you are here, how much you intend to take.

Cloudflare's directory already asks the same questions. On August 28 it opened BotBase, its directory of known bots and agents, to operators. To be listed, an operator declares what the bot does, how it uses content and who runs it. It proves its identity by IP list, reverse DNS or a Web Bot Auth signature. And since July 1, being verified no longer means being allowed: "we are no longer viewing Verified as 'default allowed'". In June, Forrester renamed the vendor category to Bot and Agent Trust Management, because agents "can represent legitimate customers, business partners, or malicious actors". The wall's product is now sorting names, not blocking machines.

What anonymity costs since Tuesday

On September 15 the defaults changed for every domain newly onboarded to Cloudflare, which fronts more than 20% of web domains by its own count. Crawlers declared as Training or Agent are blocked by default on pages that show ads. Search stays allowed. A crawler that mixes search with training is judged by all of its behaviors, so the training half now costs it the search half. What stays open is the named lane. A site owner can allow a verified bot by name, and Cloudflare's paid access bills only agents that authenticate with Web Bot Auth. An unnamed crawler cannot be allowed by name and cannot be billed. It can only be blocked.

If you buy web data, ask your vendor whose name is on its requests. A vendor that will not sign is buying the anonymous lane, and that lane shrank on Tuesday. If you value the wall vendors, Forrester just told you their product changed from a filter to a registrar.

My position, dated so it can be graded: the signature is the name on the invoice, so the next step is a price per name. By the end of 2027, at least one major network posts different per-page prices for different named crawlers, keyed on these signatures. I am wrong if every posted price on the major networks is still per declared purpose then, with no line that names the crawler. I am also wrong if the group makes Rescorla's anonymous alternative, class without name, its primary track. The first sign will be a rate card with a crawler's URL in the left column.