# --- Training crawlers --- # These fetch pages for model training purposes. User-agent: GPTBot User-agent: ClaudeBot User-agent: CCBot Allow: / # --- Search / indexing crawlers --- # These fetch pages to power AI-assisted search results. # Not used for model training. User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: MistralAI-Index User-agent: DuckAssistBot Allow: / # --- Control tokens --- # These are robots.txt-only tokens; they have no separate HTTP user agent. # They control whether content already crawled by a parent bot # (Googlebot, Applebot) is used for AI training. User-agent: Google-Extended User-agent: Applebot-Extended Allow: / # --- User-triggered fetchers (informational only) --- # These are initiated by end users, not automated crawling. # Vendor docs state robots.txt rules may not apply to these agents. # Listed here for documentation; no active directives. # ChatGPT-User — OpenAI: "robots.txt rules may not apply" # Claude-User — Anthropic: user-initiated web access # Perplexity-User — Perplexity: "generally ignores robots.txt" # MistralAI-User — Mistral: user-triggered Le Chat fetches # --- Default --- User-agent: * Allow: / Sitemap: https://www.photocircleapp.com/sitemap-index.xml llms: https://www.photocircleapp.com/llms.txt llms-full: https://www.photocircleapp.com/llms-full.txt