GPTBot: OpenAI's training crawler
GPTBot belongs to OpenAI. It downloads pages to build training data for AI models. Blocking it keeps your pages out of future training. It does not remove you from AI search results or from answers.
| Operator | OpenAI |
|---|---|
| Used for | ChatGPT |
| Type | Training crawler |
| robots.txt name | GPTBot |
| User agent | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
| Published IP ranges | https://openai.com/gptbot.json |
| Blocked by top sites | 14.7% of 740 (study) |
| Official documentation | developers.openai.com/api/docs/bots |
The version number in the user agent changes over time.
Allow or block GPTBot in robots.txt
To block it on the whole site:
User-agent: GPTBot
Disallow: /To allow it explicitly (useful when a rule for all bots would block it):
User-agent: GPTBot
Allow: /OpenAI says disallowing GPTBot means a site's content should not be used to train its generative AI models.
robots.txt is a request, not a wall. A CDN or firewall rule can turn a bot away even when robots.txt allows it, and nothing in the file will show that.
The free check requests your home page as a browser and as AI bots, reads your robots.txt and shows what gets blocked. No sign-up.
Check my siteIs it really GPTBot?
Anyone can send a request with this bot's name. OpenAI publishes the IP ranges its requests come from, so a request that carries the name from another address is an impostor. PageLens checks every request from this bot against that list.
See GPTBot on your own site
GPTBot doesn't run JavaScript, so analytics scripts never record it. AI Visibility in PageLens reads it from your server or CDN and shows which pages it requested, when, and which requests ended in a 404.