Machine Readiness
Stored receipt and evidence
20
65
0
0
0
Samples
No stored offer samples.
Samples
No stored action samples.
Samples
No stored product samples.
Document
# robots.txt for www.ncrs.org # --------------------------------------------------------- # SECTION 1: BLOCK SPECIFIC AI & SCRAPING BOTS # These bots are strictly prohibited from crawling any part of the site. # --------------------------------------------------------- User-agent: GPTBot User-agent: ChatGPT-User User-agent: CCBot User-agent: ClaudeBot User-agent: Amazonbot User-agent: Google-Extended User-agent: PerplexityBot User-agent: Bytespider User-agent: YouBot User-agent: anthropic-ai User-agent: Applebot-Extended User-agent: Meta-ExternalAgent User-agent: Omgilibot User-agent: Diffbot User-agent: cohere-ai User-agent: ImagesiftBot User-agent: Timpibot Disallow: / User-agent: GoogleBot User-agent: Bing Crawl-delay: 10 Disallow: /services/show-biography.php Disallow: /shop/includes/ Disallow: /forumtest/ Allow: / # --------------------------------------------------------- # SECTION 2: ALLOW SEARCH ENGINES & OTHERS (WITH RESTRICTIONS) # This allows Google, Bing, and others to index the public site # while protecting sensitive backend and private folders. # --------------------------------------------------------- User-agent: * Disallow: /_includes/ Disallow: /ad_peeps/ Disallow: /archives/ Disallow: /awmdata/ Disallow: /bb/ Disallow: /bb_dev/ Disallow: /build/ Disallow: /cache/ Disallow: /classifiedtest/ Disallow: /cron/ Disallow: /deceased_members/ Disallow: /deploy/ Disallow: /directors/ Disallow: /docs/ Disallow: /errordocs/ Disallow: /forums/batch/ Disallow: /forumtest/ Disallow: /helpdesk/ Disallow: /gepcpretest/ Disallow: /includes/ Disallow: /ioncube.test/ Disallow: /language/ Disallow: /LinkArt/ Disallow: /liam/ Disallow: /mysql/ Disallow: /numinix/ Disallow: /shopdev/ Disallow: /shoptest/ Disallow: /staging/ Disallow: /temp/ Disallow: /tmp/ Disallow: /training/ Disallow: /video/ Disallow: /vote/ Disallow: /admin/ Disallow: /wp-admin/ Disallow: /*.sql$ Disallow: /*.log$ Disallow: /*.bak$ Disallow: /*.env$ Disallow: /services/show-biography.php Disallow: /shop/includes/ # --------------------------------------------------------- # SITEMAP LOCATION # --------------------------------------------------------- Sitemap: https://www.ncrs.org/sitemap.xml
Document
# ========================================= # llms.txt — AI model training permissions # ========================================= # This file communicates your site's AI usage policy. # Compliant AI agents agree not to use disallowed content # for training, dataset creation, or AI-generated responses. User-agent: * Disallow: /
Document
Not stored for this site.