# llms.txt # AI Crawling Access Policy for This Store # Purpose: # This file defines how AI agents and Large Language Models (LLMs) # may access and crawl this shopping mall website. # Default Policy: # All publicly accessible storefront pages are allowed for AI crawling, # except for administrative and API-related paths. User-Agent: * Allow: / # Restricted Paths Disallow: /admin/ Disallow: /api/ Disallow: /api/* Disallow: /admin/* # Sensitive or Internal Paths (if applicable) # Disallow: /member/ # Disallow: /order/ # Disallow: /cart/ # Disallow: /mypage/ # Example: Blocking Specific AI Crawlers # (Uncomment if needed) # User-Agent: GPTBot # Disallow: / # User-Agent: Google-Extended # Disallow: / # User-Agent: ClaudeBot # Disallow: / # Notes: # - Product pages, category pages, promotional pages, and other public # storefront content are accessible to AI crawlers by default. # - Administrative dashboards and API endpoints are restricted. # - Adjust the policy according to your operational and security strategy.