Live diagnostic / no result storage
Your policy says one thing. Does your edge agree?
Review 32 prominent, live-testable AI crawler, search, user-fetcher, and dataset identities in one ordered report.
Exposure scanner
32 live identities / prominence order01
GPTBot
Awaiting test
02
ClaudeBot
Awaiting test
03
OAI-SearchBot
Awaiting test
04
PerplexityBot
Awaiting test
05
Meta-ExternalAgent
Awaiting test
06
AmazonBot
Awaiting test
Plus 26 additional training, search, user-fetcher, dataset, and crawler identities.
What it checks
The effective site’s robots.txt policy and limited homepage requests using fixed crawler identities, with controlled concurrency.
Clear findings
Every listed identity can be requested over HTTP, so each row can report a meaningful gap, alignment, no-gap, or inconclusive result.
Limits and privacy
The test does not verify source IPs or signatures. Responses are analyzed in memory; content and results are not stored.