| GPTBot | OpenAI | Training data crawl | User-agent: GPTBot | No — server log only |
| OAI-SearchBot | OpenAI | Search index for ChatGPT search | User-agent: OAI-SearchBot | No — indexer |
| ChatGPT-User | OpenAI | Live fetch on user instruction ("go to X and tell me Y") | User-agent: ChatGPT-User | Sometimes — fires GA if it loads JS |
| ClaudeBot | Anthropic | Training data crawl | User-agent: ClaudeBot | No |
| anthropic-ai | Anthropic | Older training crawler (still in use) | User-agent: anthropic-ai | No |
| Claude-User | Anthropic | Live user-driven fetch via Claude.ai | User-agent: Claude-User | Sometimes |
| PerplexityBot | Perplexity | Index crawl | User-agent: PerplexityBot | No |
| Perplexity-User | Perplexity | Live user-driven fetch | User-agent: Perplexity-User | Sometimes |
| Google-Extended | Google | Training opt-out token (block to exclude from Gemini training) | User-agent: Google-Extended | No |
| GoogleOther | Google | Multi-purpose Google fetcher | User-agent: GoogleOther | No |