User-Agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Claude-SearchBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: DuckAssistBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: YouBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: PhindBot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Bravebot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Andibot Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Claude-User Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: ChatGPT Agent Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: Manus-User Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: MistralAI-User Allow: / Disallow: /api/ Disallow: /login Disallow: /password User-Agent: GPTBot Disallow: / User-Agent: ClaudeBot Disallow: / User-Agent: Google-Extended Disallow: / User-Agent: CCBot Disallow: / User-Agent: Bytespider Disallow: / User-Agent: Meta-ExternalAgent Disallow: / User-Agent: Meta-ExternalFetcher Disallow: / User-Agent: anthropic-ai Disallow: / User-Agent: Claude-Web Disallow: / User-Agent: Amazonbot Disallow: / User-Agent: cohere-ai Disallow: / User-Agent: cohere-training-data-crawler Disallow: / User-Agent: Applebot-Extended Disallow: / User-Agent: DeepSeekBot Disallow: / User-Agent: Diffbot Disallow: / User-Agent: PetalBot Disallow: / User-Agent: AI2Bot Disallow: / User-Agent: Ai2Bot-Dolma Disallow: / User-Agent: img2dataset Disallow: / User-Agent: LAIONDownloader Disallow: / User-Agent: Scrapy Disallow: / User-Agent: FirecrawlAgent Disallow: / User-Agent: Crawl4AI Disallow: / User-Agent: ChatGLM-Spider Disallow: / User-Agent: WRTNBot Disallow: / User-Agent: FriendlyCrawler Disallow: / User-Agent: Brightbot 1.0 Disallow: / User-Agent: Timpibot Disallow: / User-Agent: SemrushBot-OCOB Disallow: / User-Agent: TikTokSpider Disallow: / User-Agent: * Allow: / Disallow: /api/ Disallow: /login Disallow: /password Sitemap: https://www.tradealgo.com/sitemap.xml