Signed agents
Signed agents are AI bots that sign each request with a cryptographic key, so a website can check who really sent it.
Signed agents are AI bots and agents that attach a cryptographic signature to every request they make. The website can check the signature against a public key the bot's company publishes, and know for certain who sent the request.
It matters because anyone can claim to be any bot. A request that says it's from a well-known AI crawler might be from a scraper borrowing its name. Signing gives sites a reliable way to tell, so they can let real AI bots in and keep impostors out.
Why names aren't enough
Every request carries a name, called the user agent. It's just text, and anyone can write anything there. So a site that lets in "ChatGPT" by name lets in anyone who types that name.
Sites have used two other checks. One compares the request's address with a list of addresses the company publishes. The other looks the address up to see whether it belongs to the company's own host name. Both work, but they break when companies change their networks, and they don't help with agents that run from shared cloud addresses.
How signing works
The bot's company publishes a public key on its own domain. Each request the bot sends is signed with the matching private key, which only the company holds. The site, or its CDN, checks the signature against the public key. If it matches, the request really came from that company.
The best-known approach is called Web Bot Auth. An IETF working group is standardising it, and Cloudflare uses it to verify bots.
For example, Fernhill Coffee lets AI search bots read its site and blocks scrapers. Its CDN verifies signed requests, so a scraper pretending to be a search bot gets turned away, while the real one gets in.
In AnswerOrbit
We check every bot visit before we count it. A visit counts when its address is on the list its company publishes, or when its address looks up to the company's own host name. For bots whose company publishes neither, we judge by how they behave. Requests that fail are shown as Unverified requests. See Which visits count.