# Default policy User-agent: * Allow: / Disallow: /api/ Disallow: /membership/checkout/ Disallow: /_archive/ Disallow: /agents/ Disallow: /go/ # Tracking-parameter URL variants -- canonical already points to the # clean URL, so block these so Google stops adding them to the # "Alternate page with proper canonical tag" report and reclaims # crawl budget. Each pattern handles ?param= (first) and ¶m= (later). Disallow: /*?tambid= Disallow: /*&tambid= Disallow: /*?ClickID= Disallow: /*&ClickID= Disallow: /*?siteId= Disallow: /*&siteId= Disallow: /*?cat= Disallow: /*&cat= # AI search and indexing bots -- explicitly allowed User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / User-agent: Google-Extended Allow: / User-agent: Bytespider Allow: / User-agent: Amazonbot Allow: / User-agent: CCBot Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: FacebookBot Allow: / User-agent: cohere-ai Allow: / User-agent: Diffbot Allow: / User-agent: YouBot Allow: / Sitemap: https://tradersagency.com/sitemap.xml