Audit AI crawler access and citability for a page — confirm retrieval/citation bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed and the Googlebot vs Google-Extended split is correct, classify training vs search/retrieval vs user-fetch user-agents, check the page is server-rendered e
Audit AI crawler access and citability for a page — confirm retrieval/citation bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed and the Googlebot vs Google-Extended split is correct, classify training vs search/retrieval vs user-fetch user-agents, check the page is server-rendered enough for non-JS AI crawlers, validate llms.txt / llms-full.txt (also covers M21), and generate a ch