User-agent: * Sitemap: https://www.scottmachinery.com/sitemap.xml ############## # custom # ############## Allow: / # AI training crawlers: no visitor value. Search agents (Googlebot, Bingbot, Applebot, OAI-SearchBot, # Claude-SearchBot, PerplexityBot) and link-preview fetchers stay allowed. User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: meta-externalagent User-agent: Meta-ExternalAgent User-agent: FacebookBot User-agent: Amazonbot User-agent: CCBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: Bytespider User-agent: PetalBot User-agent: MJ12bot User-agent: DotBot Disallow: /