# ============================================================ # robots.txt for orchidhotel.com # Last updated: March 2026 # ============================================================ # ------------------------------------------------------------ # SITEMAP & LLMs REFERENCE # ------------------------------------------------------------ Sitemap: https://www.orchidhotel.com/sitemap.xml LLMs: https://www.orchidhotel.com/llms.txt # ------------------------------------------------------------ # TRADITIONAL SEARCH ENGINES # ------------------------------------------------------------ User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Video Allow: / User-agent: Googlebot-News Allow: / User-agent: AdsBot-Google Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Slurp Allow: / User-agent: YandexBot Allow: / User-agent: Baiduspider Allow: / User-agent: facebookexternalhit Allow: / User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: / User-agent: WhatsApp Allow: / # ------------------------------------------------------------ # OPENAI / CHATGPT # ------------------------------------------------------------ User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: GPTBot Allow: / # ------------------------------------------------------------ # ANTHROPIC / CLAUDE # ------------------------------------------------------------ User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: anthropic-ai Allow: / # ------------------------------------------------------------ # GOOGLE AI (GEMINI / AI OVERVIEWS) # ------------------------------------------------------------ User-agent: Google-Extended Allow: / User-agent: Googlebot-AI Allow: / # ------------------------------------------------------------ # PERPLEXITY AI # ------------------------------------------------------------ User-agent: PerplexityBot Allow: / # ------------------------------------------------------------ # META AI (LLAMA / META SEARCH) # ------------------------------------------------------------ User-agent: meta-externalagent Allow: / User-agent: meta-externalfetcher Allow: / # ------------------------------------------------------------ # APPLE (SIRI / APPLEBOT / SPOTLIGHT) # ------------------------------------------------------------ User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # ------------------------------------------------------------ # MICROSOFT COPILOT / BING AI # ------------------------------------------------------------ User-agent: MSNBot Allow: / User-agent: Bingbot-AI Allow: / # ------------------------------------------------------------ # AMAZON ALEXA / AWS # ------------------------------------------------------------ User-agent: Amazonbot Allow: / # ------------------------------------------------------------ # YOU.COM # ------------------------------------------------------------ User-agent: YouBot Allow: / # ------------------------------------------------------------ # COHERE AI # ------------------------------------------------------------ User-agent: cohere-ai Allow: / # ------------------------------------------------------------ # COMMON CRAWL (trains most LLMs globally) # ------------------------------------------------------------ User-agent: CCBot Allow: / # ------------------------------------------------------------ # DIFFBOT (knowledge graph and AI training) # ------------------------------------------------------------ User-agent: Diffbot Allow: / # ------------------------------------------------------------ # BYTEDANCE / TIKTOK AI # ------------------------------------------------------------ User-agent: Bytespider Allow: / # ------------------------------------------------------------ # SCREAMING FROG / SEO TOOLS # ------------------------------------------------------------ User-agent: Screaming Frog SEO Spider Allow: / # ------------------------------------------------------------ # BLOCK SEO COMPETITOR SCRAPERS # ------------------------------------------------------------ User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: PetalBot Disallow: / # ------------------------------------------------------------ # BLOCK MALICIOUS / VULNERABILITY SCANNERS # ------------------------------------------------------------ User-agent: Nikto Disallow: / User-agent: sqlmap Disallow: / User-agent: Acunetix Disallow: / User-agent: Netsparker Disallow: / # ------------------------------------------------------------ # BLOCK EMAIL HARVESTERS AND SPAM BOTS # ------------------------------------------------------------ User-agent: spbot Disallow: / User-agent: EmailCollector Disallow: / User-agent: AutoEmailCollector Disallow: / User-agent: Harvest Disallow: / User-agent: BlackWidow Disallow: / User-agent: MegaIndex Disallow: / # ------------------------------------------------------------ # DEFAULT all other bots: allow site, block private paths # ------------------------------------------------------------ User-agent: * Allow: / Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /private/ Disallow: /assets/img/T3%20Magazine_September_2023_LOW%20RES.pdf