User-agent: * Disallow: /xmlrpc.php Disallow: /wp-admin/ Disallow: /s/ Disallow: /review-terms/ Sitemap: https://gradle.com/sitemap.xml # --- Major AI Crawlers --- # Explicitly allowing major AI systems to crawl the site # OpenAI (ChatGPT) User-agent: GPTBot Allow: / # Anthropic (Claude) User-agent: ClaudeBot Allow: / # Google AI (Gemini/Bard) User-agent: Google-Extended Allow: / # Perplexity AI User-agent: PerplexityBot Allow: /