meta-externalagent
meta-externalagent is Meta's web crawler for training its AI models and for indexing content for its products. You can block it in robots.txt.
meta-externalagent is one of Meta's web crawlers. Meta says it crawls the web "for use cases such as training foundation AI models or improving products by indexing content directly," in its list of web crawlers. In practice, it's the bot that collects your pages for Meta's AI.
If you're deciding which AI bots to let read your site, this is the Meta one most people mean.
How to recognise it
It identifies itself with a user agent that contains meta-externalagent/1.1, in lower case, though Meta's own headings write it Meta-ExternalAgent. Bots match robots.txt names without caring about case, as the robots.txt standard requires. If you're filtering your server logs yourself, check how your tool handles case.
Anyone can put a bot's name in their requests, so a request claiming to be meta-externalagent isn't proof it came from Meta. Meta itself calls allow-listing by IP address the more secure choice.
Should you block it?
It depends on what you want. Meta lists this bot for both training and indexing, so blocking it keeps your pages out of both. If you're happy for your content to train AI models, let it in. If not, block it, and decide separately about Meta's other bots.
For example, a recipe publisher might add this to its robots.txt to keep its recipes out of Meta's training, while leaving meta-webindexer free to read the site so Meta AI can still find it:
User-agent: meta-externalagent
Disallow: /Meta says changes to robots.txt can take up to 24 hours to take effect.
In AnswerOrbit
The Audit tab in Website reads your robots.txt every day and shows each AI bot we know with its purpose and whether it's allowed, partly allowed or blocked. Connect your server logs and Bot visits shows which bots visit, after checking each request so that a bot borrowing a known name doesn't count. See Website and AI traffic.