meta-externalagent: Meta's training crawler
meta-externalagent belongs to Meta. It downloads pages to build training data for AI models. Blocking it keeps your pages out of future training. It does not remove you from AI search results or from answers.
| Operator | Meta |
|---|---|
| Used for | Meta AI |
| Type | Training crawler |
| robots.txt name | meta-externalagent |
| User agent | meta-externalagent/1.1 |
| Published IP ranges | None published |
| Blocked by top sites | 12.6% of 740 (study) |
| Official documentation | developers.facebook.com/docs/sharing/webmasters/web-crawlers |
Meta describes it as crawling for uses such as training foundation AI models or indexing content. A separate agent, meta-externalfetcher, fetches links at a user's request and, per Meta, may bypass robots.txt.
Allow or block meta-externalagent in robots.txt
To block it on the whole site:
User-agent: meta-externalagent
Disallow: /To allow it explicitly (useful when a rule for all bots would block it):
User-agent: meta-externalagent
Allow: /Meta says it can be blocked with a robots.txt disallow; changes take up to 24 hours.
robots.txt is a request, not a wall. A CDN or firewall rule can turn a bot away even when robots.txt allows it, and nothing in the file will show that.
The free check requests your home page as a browser and as AI bots, reads your robots.txt and shows what gets blocked. No sign-up.
Check my siteIs it really meta-externalagent?
Anyone can send a request with this bot's name. Meta doesn't publish IP ranges for it, so the name alone can't prove a request is real.
See meta-externalagent on your own site
meta-externalagent doesn't run JavaScript, so analytics scripts never record it. AI Visibility in PageLens reads it from your server or CDN and shows which pages it requested, when, and which requests ended in a 404.