Web Bot Auth
Web Bot Auth is a proposed standard that lets bots prove who they are by signing each request, so a site can tell real AI bots from fakes.
Web Bot Auth is a proposed way for bots to prove who they are. Instead of only giving a name, a bot signs each request with a private key. It publishes the matching public key at a known address, and any website can check the signature against it.
It matters because a bot's name, its user agent, can be faked by anyone. If you want to let one AI company's bot in, charge another, or block a third, you first need to know who's really knocking.
How it works
The bot's owner makes a pair of keys. The private key stays with the bot. The public key goes in a directory on the owner's own website. Every request the bot sends carries a signature and a pointer to that directory.
The website, or the network in front of it, fetches the public key and checks the signature. If it matches, the request really came from that bot. The signatures follow the HTTP Message Signatures standard from the IETF, and the parts specific to bots are still drafts. Cloudflare, one of the networks that checks them, explains the details in its Web Bot Auth documentation.
Why it's an improvement
The older ways of checking a bot both have gaps. Companies publish lists of the addresses their bots use, but lists go out of date. Looking up the host name of an address works, but not every company sets it up. A signature travels with the request itself, wherever the bot runs from.
An example
For example, Kestrel Roasters' server logs show thousands of requests a day from something calling itself an AI search bot. Some come from the company's published addresses. Many don't. With signed requests, Kestrel's network provider could tell the real bot from the copies without keeping track of any address list.
In AnswerOrbit
Bot visits in Website and AI traffic checks every request before counting it. A visit counts when its address is on the list its company publishes, or when its address looks up to the company's own host name. Requests that fail the check are shown as Unverified requests.